Support for bidirectional text and glyph shaping

[nyanpasu64]

I added NotoSansArabic-Regular.ttf as a fallback font to see how RmlUi rendered Arabic:

• Arabic text is rendered backwards. "اَلْعَرَبِيَّةُ" is supposed to be rendered right-to-left, but is instead drawn left-to-right. I think the glyphs are also not connected as they should be, but I don't think there's a "proper" way to connect glyphs if they're drawn in the backwards order.
• Lato's "fi" ligature is not used. If ligature/shaping support is implemented, the "i" will not have a dot.

Is internationalization a priority for this project?

[mikke89]

Yes, we don't currently support bidirectional text (neither Unicode-based or in RCSS) or ligatures.

I don't think it's realistic for us to implement text rendering and Unicode support as well as Firefox and other web browsers. Here's a great article on the complexities they need to deal with. And for text rendering changes we also need improved text editing logic which can be surprisingly complex as well.

With that said, I'm all for adding ligatures and RTL support. However, RTL support in particular should involve somebody who needs it and uses it on a regular basis so they can maintain it and fix bugs that pop up.

[ZombieRaccoon]

Since I really appreciate the library and all the effort that has went into it, I'd like to give my two cents in hopes of furthering its reach in terms of internationalization.

First of all, I perfectly agree with @mikke89 in that implementing such functionality would be relatively complex and would require significant insight and possibly even first-hand experience with such languages. Generally speaking, I advocate the use of well-established technologies in such cases, mainly because of the sheer amount of domain-specific knowledge and experience that has been accumulated within them.

As such, it'd be worth considering whether integrating a text shaping library that supports RTL out of the box would better serve this project, whose main focus - from my understanding - is not centered around text shaping. In my opinion, making such a move would both eliminate a potential maintenance burden and help pave the way for broader adoption.

I'd recommend taking a look at HarfBuzz, for example.

[mikke89]

You make a great point, and I completely agree. Integrating HarfBuzz should be the way forward rather than trying to make something along these lines ourselves.

I wonder how much work it will be to integrate it? It doesn't seem overly complicated, but of course we all know these things are bigger projects than they seem like at first. I'd love to see someone do a proof-of-concept or start the integration process. I would be very much interested in following up on that.

I'm a bit worried that it might bloat the size of RmlUi applications, especially for those who mainly care about English and some occasional symbols. I think an integration should be optional if possible. What would be cool is perhaps the ability to select between two different font engines, one simple based on an integrated stb_truetype, and another based on FreeType+HarfBuzz.

[ZombieRaccoon]

It'd be quite difficult for me to predict the effort required, as I'm not particularly well-versed in font rendering. The Rml::FontEngineInterface interface looks quite simple, but the infrastructure behind the default font engine and maybe fonts in general seems to be on the heavier side. I'd probably look to simplify things a little, if it was possible. Perhaps HarfBuzz could take over certain duties that FreeType couldn't perform, thus eliminating some of the complexity.

Moving on, I do understand the motive behind trying to keep the resulting binary compact, but I'd personally refrain from maintaining multiple implementations until both measurements - quite simple ones, in this case - and demand from users show that it actually is an issue. If this was the case, then I could absolutely imagine having an alternate implementation.

[zakalawe]

Came here wondering about BiDi and similar shaping features in RmlUi. (Eg Arabaic and CJK scripts) From other research, HarfBuzz is essentially an extension to FreeType (was originally the same code-base) but for full text-shaping you would also need to consider using Pango which is unfortunately large and depends on GLib. (There's also FreeBiDi in there as a dependency, I think). Avoiding Pango dependency would be nice for those reasons but means you have to separate text into runs of consistent style, direction, font, etc before handing them to HarfBuzz.

[mikke89]

We now have initial support for harfbuzz in one of the samples, see #568. There are surely improvements to be made, and in the end we would like to move this font engine directly into core as an optional component.

However, I am closing this issue, further improvements related to this topic should be continued in new and more focused issues.