Rearrangements #2387

brucemiller · 2024-08-09T15:24:11Z

This PR continues the previous TeX-pool reorg PR in preparation for more extensive refactoring.

move several helper functions and ligature patterns to more appropriate "Engine" modules
recognizes more font names
more accurate initialization of various \mathcode, \sfcode
simulate TeX-plain's register allocation better.

… stored as-if 'at'; make mathDefault font use OT1 encoding; make relativeTo also compare encoding; new Font->asFontinfo to return simulated fontinfo for eventually better integration

…otherwise use the cmd itself; similar for \mathchardef

…ialized \mathcodes; removed helpers and ligatures to core Engine pool files

dginev

Nice progress!

I left comments on the pieces I understand already, and you have my "fingers crossed" for the TeX-native transitional changes I only understand in spirit.

dginev · 2024-08-09T15:39:11Z

lib/LaTeXML/Common/Font.pm

-    $size *= $scaled if defined $scaled;
+    $size        = 1 unless $size;    # Yes, also if 0, "" (from regexp)
+    $size        = $at           if defined $at;
+    $size        = $at * $scaled if defined $scaled;


Maybe the 3 size assignments here can be compacted to avoid edge cases:

$size = (defined $at) ? (defined $scaled ? $at * $scaled : $at) : 1;

Sample test:

sub trysize { my ($at, $scaled) = @_; my $size = (defined $at) ? (defined $scaled ? $at * $scaled : $at) : 1; return $size } print trysize(undef, undef),"\n"; print trysize(undef, 0.5),"\n"; print trysize(0.5, undef),"\n"; print trysize(0.5, 0.5),"\n"; # output: # 1 # 1 # 0.5 # 0.25

That said, the new code will not scale a previously available $size, it will only scale with respect to the $at value. Is that change intended?

Essentially a typo, from backing off a change that wasn't ready. The 2nd line should indeed have been ```$size=$size * $scaled if $scaled;``. Probably those lines will disappear, or be handled differently in the next PR, so not worth optimizing. commit on the way...

dginev · 2024-08-09T15:41:34Z

lib/LaTeXML/Common/Font.pm

@@ -266,6 +276,7 @@ sub stringify {
  no warnings 'recursion';
  my ($self) = @_;
  my ($fam, $ser, $shp, $siz, $col, $bkg, $opa, $enc, $lang, $mstyle, $flags) = @$self;
+  # !!!!!


What could be some words to describe that exclamation?

that got added? Hmm...

dginev · 2024-08-09T15:42:53Z

lib/LaTeXML/Common/Font.pm

+  my ($fam, $ser, $shp, $siz, $col, $bkg, $opa, $enc, $lang, $mstyle, $flags) = @$self;
+  return { family => $fam, series => $ser, shape => $shp, size => $siz,
+    color    => $col,          background => $bkg,  opacity   => $opa,
+    encoding => $enc || 'OT1', language   => $lang, mathstyle => $mstyle }; }


$flags are not returned?

I think they'll only show up in the CSS style of specifying fonts, so shouldn't be needed for "fontinfo", but the whole synthesization idea is a bit up-in-the-air.

dginev · 2024-08-09T15:43:14Z

lib/LaTeXML/Common/Font.pm

@@ -328,6 +349,7 @@ sub relativeTo {
  my ($self, $other) = @_;
  my ($fam,  $ser,  $shp,  $siz,  $col,  $bkg,  $opa,  $enc,  $lang,  $mstyle,  $flags)  = @$self;
  my ($ofam, $oser, $oshp, $osiz, $ocol, $obkg, $oopa, $oenc, $olang, $omstyle, $oflags) = @$other;
+  # !!!!


[2] What could be some words to describe that exclamation?

dginev · 2024-08-09T15:46:21Z

lib/LaTeXML/Common/Font/Metric.pm

@@ -152,12 +153,15 @@ sub process_lig_kern {
      $prognum = 256 * $op + $remainder; $firstinstr = 0;
      next; }
    $next = $$fontmap[$next];
+    $next = $$next[0] if ref $next;


The changes handling dual type between arrayref vs scalar happen a bit often in this subroutine. This will be somewhat painful to port to Rust, but also makes me wonder why both are allowed in the first place.

Something that will be on my plate a little later, jsut noting it here.

Indeed, there's a few such changes that leaked into this PR, but are really about the upcoming one. The idea is for the FontMap to store not only the equivalent Unicode, but some set of properties/attributes (esp. math related ones). Ie. all still up for discussion.

dginev · 2024-08-09T15:47:39Z

lib/LaTeXML/Core/Definition/CharDef.pm

@@ -47,13 +47,17 @@ sub invoke {
  my $mathglyph = $$self{mathglyph};
  # A dilemma: If the \chardef were in a style file, you're prefer to revert to the $cs
  # but if defined in the document source, better to use \char ###\relax, so it still "works"
-  if (defined $mathglyph) {    # Must be a math char
+  my $src   = $$self{locator} && $$self{locator}->toString;
+  my $local = $src && $src !~ /\.(?:sty|ltxml|ltxmlc)/;    # Dumps currently have undefined src!


Is .ltxmlc a new extension? We don't have them in the repository at the moment. If it's dump related, why c and not d? Or even better .ltxml.dump or .dump.ltxml

Exactly for dumps and your naming suggestion sounds good

dginev · 2024-08-09T15:50:20Z

lib/LaTeXML/Engine/LaTeX.pool.ltxml

@@ -4889,6 +4889,7 @@ DefConstructor('\@framebox[Dimension][]{}',
          $document->setAttribute($c[0], $k => $v); } } } }
 );

+AssignValue(allocated_boxes => 0);


Maybe the AssignValue should be 'global' scoped? Likely identical, until it isn't :>

dginev · 2024-08-09T15:52:25Z

lib/LaTeXML/Engine/TeX_Fonts.pool.ltxml

+# DefLigature(qr{ffi},              "\x{FB03}", fontTest => \&nonTypewriterT1);
+# DefLigature(qr{ffl},              "\x{FB04}", fontTest => \&nonTypewriterT1);
+
+DefLigature(qr{\.\.\.}, "\x{2026}", fontTest => sub { $_[0]->getFamily ne 'typewriter'; });  # ldots


this test can also be \&nonTypewriter

Ah the benefits of good modularity! will do

dginev · 2024-08-09T15:53:38Z

lib/LaTeXML/Engine/TeX_Paragraph.pool.ltxml

+    $document->insertElement('ltx:text', $line, class => 'ltx_align_' . $alignment);
+    $document->insertElement('ltx:break'); }
+  else {
+    $document->absorb($line); }


should this case emit an Info or Warning message? Silently absorbing while dropping the alignment may be worth logging.

not a bad idea...

dginev · 2024-08-09T15:54:37Z

lib/LaTeXML/Engine/plain.pool.ltxml

+\outer\def\newwrite{\alloc@7\write\chardef\sixt@@n}
+\outer\def\newfam{\alloc@8\fam\chardef\sixt@@n}
+\outer\def\newlanguage{\alloc@9\language\chardef\@cclvi}
+EoTeX


Dropping Perl definitions in favor of RawTeX? Scary. Hopefully more reliable, but something to keep an eye on while testing...

Yeah, scary, but really the underlying theme: most of plain.pool.ltxml could disappear if (some incarnation of) plain.tex can be read in.

which really was changes to Fontmap that weren't yet ready for inclusion This reverts commit 8144738.

brucemiller · 2024-08-11T20:35:34Z

Last commit reverts all the FontMap tweaks that were in LaTeXML::Common::Font::Metric, since those were too early, and probably won't even happen in the end.

* Recognize more TeX font names; note that fonts loaded as 'scaled' are stored as-if 'at'; make mathDefault font use OT1 encoding; make relativeTo also compare encoding; new Font->asFontinfo to return simulated fontinfo for eventually better integration * Have \chardef revert to \char only for definitions from main source, otherwise use the cmd itself; similar for \mathchardef * Slightly more robust coding * Corrections to INITEX initializations of mathcode * Corrected \sfcode default * Moved dash & quote ligatures and helpers from plain.pool * Moved cdots,ldots ligatures from plain.pool * Moved alignLine helper from plain.pool * More TeX-like allocation,\newcount, etc; initialized more fonts; initialized \mathcodes; removed helpers and ligatures to core Engine pool files * accommodate change to allocators * Retract two changes that are too early * Fix typo for scaled font * Assign allocated_boxes globally * use the utilities that we've just defined * Give clue when alignment of a line is dropped * Revert "Slightly more robust coding" which really was changes to Fontmap that weren't yet ready for inclusion This reverts commit 8144738.

brucemiller added 11 commits August 8, 2024 22:28

Recognize more TeX font names; note that fonts loaded as 'scaled' are…

67d4198

… stored as-if 'at'; make mathDefault font use OT1 encoding; make relativeTo also compare encoding; new Font->asFontinfo to return simulated fontinfo for eventually better integration

Have \chardef revert to \char only for definitions from main source, …

e216ff0

…otherwise use the cmd itself; similar for \mathchardef

Slightly more robust coding

8144738

Corrections to INITEX initializations of mathcode

32a185e

Corrected \sfcode default

892b625

Moved dash & quote ligatures and helpers from plain.pool

bfadb1b

Moved cdots,ldots ligatures from plain.pool

8b60aa0

Moved alignLine helper from plain.pool

ad34ec6

More TeX-like allocation,\newcount, etc; initialized more fonts; init…

5298d1e

…ialized \mathcodes; removed helpers and ligatures to core Engine pool files

accommodate change to allocators

289f3c2

Retract two changes that are too early

cb8721e

brucemiller requested a review from dginev August 9, 2024 15:24

dginev reviewed Aug 9, 2024

View reviewed changes

brucemiller added 4 commits August 9, 2024 12:44

Fix typo for scaled font

4059925

Assign allocated_boxes globally

4283924

use the utilities that we've just defined

895e80b

Give clue when alignment of a line is dropped

f4e6559

brucemiller requested a review from dginev August 9, 2024 21:56

dginev approved these changes Aug 9, 2024

View reviewed changes

Revert "Slightly more robust coding"

1669a34

which really was changes to Fontmap that weren't yet ready for inclusion This reverts commit 8144738.

brucemiller merged commit bc2d8f6 into master Aug 12, 2024
26 checks passed

brucemiller deleted the rearrangements branch August 12, 2024 13:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rearrangements #2387

Rearrangements #2387

brucemiller commented Aug 9, 2024

dginev left a comment

dginev Aug 9, 2024

dginev Aug 9, 2024

brucemiller Aug 9, 2024

dginev Aug 9, 2024

brucemiller Aug 9, 2024

dginev Aug 9, 2024

brucemiller Aug 9, 2024

dginev Aug 9, 2024

dginev Aug 9, 2024

brucemiller Aug 9, 2024

dginev Aug 9, 2024

brucemiller Aug 9, 2024

dginev Aug 9, 2024

brucemiller Aug 9, 2024

dginev Aug 9, 2024

brucemiller Aug 9, 2024

dginev Aug 9, 2024

brucemiller Aug 9, 2024

dginev Aug 9, 2024

brucemiller Aug 9, 2024

brucemiller commented Aug 11, 2024

Rearrangements #2387

Rearrangements #2387

Conversation

brucemiller commented Aug 9, 2024

dginev left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brucemiller commented Aug 11, 2024