Add element parameter to font engine string methods

[TriangulumDesire]

Hello! First, let me thank you (and all of the other contributors) for such an amazing library!

This pull request adds an element parameter to the GetStringWidth and GenerateString methods of the font engine interface. I am creating such a pull request since it can add extra context to text rendering that would otherwise be unavailable or difficult to acquire.

For example, I am using a custom font engine with RmlUi to render text. It uses the HarfBuzz text-shaping library to properly shape the glyphs when text is rendered.

In order to perform accurate text shaping, HarfBuzz requires three bits of information: language, script, and text direction. It is possible to set a global language in my custom font engine, but there is an issue with documents that have multiple languages — such as a language-selection menu has the native names of each language, which would have several scripts and therefore may require different shaping rules to render each string correctly.

Adding the element parameter to the aforementioned functions will give the additional context needed to render text correctly. For instance, you can add a lang="en" attribute (changing "en" to whichever language the element is) to the elements whose text has a constant language/language different to the global one. You can then check the lang attribute in the font engine's methods and configure the appropriate text-shaping flags therefrom.

[mikke89]

Hey, and thank you for the kind words!

I'm very interested to hear more about how well it works for you to combine the library with HarfBuzz. That is something that has been requested before, see in particular #211. And something I'd be interested in better integration with.

I think what you are trying to do here is very good, but I believe it is not the right solution. In fact, we've had several requests in passing elements to the interfaces. However, this creates a dependency between the interface and elements that I'm not comfortable with. The reasoning is very similar to the one in this comment: #540 (comment), with the main difference being the system interface instead.

To me, it would make a lot more sense to modify the font engine interface to make it pass whatever is needed for a good HarfBuzz integration.

[TriangulumDesire]

Hi; thanks for your response!

I can definitely understand that point of view. In hindsight, this PR does feel like an ad-hoc solution.

I am currently using my own font renderer with a custom FontEngineInterface implementation. My font renderer makes use of FreeType (for loading and rasterising the fonts), FreeType-GL (for texture atlases and Unicode utilities with FreeType), and HarfBuzz (for text shaping). It generates geometries for each glyph, which I then send to the custom font engine's geometry list output. I also use other auxiliary libraries for bidirectional text, UTF-8 iteration, and Unicode functions (such as case-conversion and glyph trait querying).

As for integrating text-shaping:
With HarfBuzz, you can create a shaper from an existing FreeType font/face. From here, all you really need is an buffer, and HarfBuzz will do all of the heavy-lifting for you regarding text-shaping.

HarfBuzz buffers need a few bits of information to help with context:

• Language: This is the BCP-47 language code of the text within the buffer ("en", "en-GB", "nl", "es", "es-419", et cetera).
• Script: This is the main script of the text inside a buffer (Latin for English/Dutch/German/et cetera, Arabic for Arabic/Urdu/et cetera, Cyrillic for Russian/Ukrainian/et cetera).
• Direction: This is the text-flow direction of the text in the buffer (left-to-right, right-to-left, et cetera).
• Flags: These are some additional flags that can help with edge-cases and other situations. Most of these are advanced and won't really be needed for basic text-shaping, but some can be useful depending on the language (for example: HB_BUFFER_FLAG_PRODUCE_SAFE_TO_INSERT_TATWEEL can help with making Arabic languages more pleasant to read, but it isn't strictly necessary).

Most of these can simply be inferred from the language. If the language is set to "en", then you can determine the script (Latin), direction (left-to-right), and that no shaping flags are required. If the language is set to "ar", then you know it will be Arabic script, right-to-left direction, and you could add the tatweel/kashida flag if desired. Therefore, the only extra bit of context that the font engine needs for text-shaping is the language.

One suggestion I came up with on-the-spot is to add a language parameter (instead of the Rml::Element* parameter) to the font interface's methods. The language could then be determined from a lang attribute (similar to HTML's), which would be queried (with Closest or similar element method) before the text's geometries are computed. The parameter could be a const char*, with it being null if the element has no set language attribute — using the default language/shaping. I'll await your judgement to see if this is the right way to go about this, or I'll see if I can come up with a different suggestion in the meantime.

[mikke89]

Thanks for elaborating, it's very interesting.

I certainly think passing the language as a parameter is a much better solution. I wonder if there is any chance that one might later want to expand with access to the other parameters you mention? In particular, I can imagine there might be some use for e.g. right-to-left. If yes, then to avoid having to break this API repeatedly, then I suggest that we create a new struct for this kind of data. Even if we only support language to begin with.

What would be the proper term for this data, TextShapingContext? Maybe include the letter-spacing parameter as part of that?

As for implementation of the lang attribute. I think we would take a great performance penalty if we called out to Closest every time we call into the font engine. Instead, I believe this attribute works very much like inherited CSS properties, so I suggest that we piggy-back onto that feature. In particular, we could add it as a computed value. Then, we can set a "dummy" CSS property representing the lang attribute any time it is changed, and have that apply the computed value and inherited like other properties. Maybe we could even add a proper property, if it could be useful at all to set it with CSS. At least it wouldn't be much more work.

In any case, let me know if this sounds reasonable to you, or if you have some other ideas. This was a very brief/rough description, so please let me know of any questions, and I can also help out around the implementation of this.

[TriangulumDesire]

Passing a structure is a great idea. TextShapingContext is a good name for it. Another suggestion could be something akin to TextLocalisationContext, but I feel that this isn't self-documenting enough (a name like TextLocalisationContext says that it provides text localisation context but doesn't explain why it's there to begin with; someone unfamiliar with text shaping might not know why text localisation would be needed to properly shape text) — whereäs a name like TextShapingContext clearly states that its members are there to give context to help with text shaping. letter-spacing would also be a good fit as a member of the structure with the name TextShapingContext.

The performance impact of calling Closest is one of the reasons I was tentative about the idea. Regarding making lang an RCSS property, I am very on board with this idea. I believe the reason why lang is an HTML attribute is because it is important for search-engine optimisation and accessibility features (like screen readers), so it does a lot more than style a webpage like a CSS property would (and the CSS lang selector also has a few extra powers). However, for RML, the reason for adding the language as a property is to help with text shaping, which is a form of styling the document, so I believe that adding it as an RCSS property would be the right choice here rather than making a dummy property based on attributes.

Making it a property will also help with providing other context without relying on the language. For example, text direction and text script. An idea I had would be the three following properties:

text-language: <string>;
text-direction: auto | left-to-right | right-to-left;
text-script: auto | <string>;

The text-language property would be the language code/name as a string ("en", "nl", "ar", et cetera). The text-direction property would state the direction of the text, or auto if it is to be determined from the language. The text-script would state the script of the text (for example, an ISO-15924 code, wherefrom HarfBuzz has functions to convert to its own script enumeration), or auto if it is to be determined from the language. The defaults for these could be "en", auto, and auto, respectively.

There could also be a shorthand property that combines the three:

text-localization: <text-language> <text-direction> <text-script>;

All of these could be added to TextShapingContext. Here's a basic outline of what the structure could be:

enum class TextDirection
{
    Auto,
    LeftToRight,
    RightToLeft,
};

struct TextShapingContext
{
    String language;
    TextDirection text_direction;
    TextScript text_script; // TextScript being a custom class similar to NumberAuto but for strings.

    float letter_spacing;
};

Let me know what you think about this.

[mikke89]

Great, I think we're starting to carve out a direction here.

Generally, I do think that we should follow HTML/CSS when we can, unless there are very good reasons not to. In particular, there is already the direction property in CSS. Thus, I think we should name this property as such.

On MDN, I see it is quite strongly encouraged to use the HTML attributes rather than CSS:

As the directionality of the text is semantically related to its content and not to its presentation, it is recommended that web developers use this attribute instead of the related CSS properties when possible.

Also, there is no CSS-equivalent of the lang attribute, I would guess with a justification as above. I see that this attribute also takes a script subtag. So with that in mind, I don't believe we need the separate script property.

My suggestions:

• Add direction property, but not language or script properties.
  • As a difference from CSS, we should note that we don't use this to affect layout, only as a hint to the font engine.
• Add dir attribute, which sets the direction property.
• Add lang attribute, which sets the language, Make it a computed value, acts like an inherited property.
  • Pass the lang attribute directly on to the font engine, and let them deal with parsing the language, script, and region subtags as they see fit.

The struct would then become something like this.

struct TextShapingContext
{
    String language;
    TextDirection text_direction;
    float letter_spacing;
};

Some implementation thoughts. Maybe take language as a const-reference? Do some measurements to see what is fastest. Also, some basic calculations tells me that the language is maximum 12 characters (excluding zero-terminator), so we could use that to pack it tightly in the computed values. But maybe a std::string is worth it.

Even better, if we actually had a Harbuzz font engine to show all this off with :)

[TriangulumDesire]

That all sounds really good!

So, to summarise everything:

• A new lang attribute for elements (similar to HTML's).
• A new dir attribute for elements (similar to HTML's).
• A new internal, inherited property called lang that is only set via the lang attribute. It is a string that is usually the BCP-47 code of the language or an empty string representing an unknown language (similar to how HTML's lang attribute works). Its default value would be an empty string.
• A new inherited property called direction that can be set by the dir attribute, or in RCSS. It can have the values auto, ltr, or rtl. Unlike HTML and CSS, the text direction in RmlUi does not affect the layout (they only exist as hints for text shaping). Its default value would be auto. One thing to note is that the direction property in CSS can't be set to auto, whereäs the dir attribute in HTML can. Did we also want to make the direction property in RCSS also support auto, remove auto as an option from the dir attribute, or only allow auto in the dir attribute to keep congruence between CSS and RCSS? Alternatively, we could forego implementing the direction property entirely and only make it an internal one, similar to lang (since use of direction in CSS is strongly discouraged in place of the dir attribute).

These properties are then sent to the font engine (in a structure) from the element whenever the text's width or geometries are to be computed. This structure will contain the language string, text direction enumeration, and letter spacing. From here, the font engine can determine what else is needed to shape the text properly.

One other thing to mention is the lang and dir CSS pesudo-class selectors. They can mostly be emulated with attribute selectors, but there are some subtle differences; however, I don't think implementing them is a priority right now if we wish to add them.

My intuition thinks that taking the language string as a constant reference would be the fastest, but I'll perform some benchmarks to properly verify this. Passing the text-shaping data to the font engine shouldn't have a lot of overhead if the user doesn't wish to use it.

Even better, if we actually had a Harbuzz font engine to show all this off with :)

I'm thinking that I will create a new sample project (similar to the bitmap-font one) that uses a HarfBuzz-based text renderer with buttons to change the language.

If that's everything, then I might have a go at implementing this soon (though in a new pull request, I'd say).

[mikke89]

Yup, this sounds all good to me!

I am fine with not adding the direction property too. I don't really have a strong opinion on that, and we can always add it later too. I'll leave it to you whether you find it useful. And I agree about :lang and :dir selectors, let's rather add those later if we see the need for them.

I'm thinking that I will create a new sample project (similar to the bitmap-font one) that uses a HarfBuzz-based text renderer with buttons to change the language.
If that's everything, then I might have a go at implementing this soon (though in a new pull request, I'd say).

Very cool, a separate sample sounds like a good idea, looking forward to seeing that! And then we could think about taking it into the default font engine later on.

[TriangulumDesire]

Alright, thanks for your guidance! I'll close this PR and get to work on implementing them. Expect a new pull request sometime in the near future!