extra guards for empty lines for Mouth::readToken #2215
Conversation
|
I should also mention that I have tested this on the way the regression manifested itself on |
lib/LaTeXML/Core/Mouth.pm
Outdated
| @@ -214,7 +214,7 @@ sub handle_escape { # Read control sequence | |||
| # Bit I believe that he does NOT mean within control sequences | |||
| my $cs = "\\" . $ch; # I need this standardized to be able to lookup tokens (A better way???) | |||
| if ((defined $cc) && ($cc == CC_LETTER)) { # For letter, read more letters for csname. | |||
| while ((($ch, $cc) = getNextChar($self)) && $ch && ($cc == CC_LETTER)) { | |||
| while ((($ch, $cc) = getNextChar($self)) && (length($ch) > 0) && ($cc == CC_LETTER)) { | |||
There was a problem hiding this comment.
Is this really supposed to be positive length, rather than defined?
There was a problem hiding this comment.
True, the length check is extra-paranoid (I believe I was triple-checking I am not misunderstanding the return values).
My fear was that the empty string "" is defined, but empty. But studying getNextChar, there is no code path that could return "". So I can switch this to a "defined" test, thanks.
There was a problem hiding this comment.
Done. Also rebased to master - the English babel load continues to succeed.
dginev
force-pushed
the
babel-english-regression
branch
from
29cb020 to
9f2645f
Compare
|
Slightly scary, but seems right, and doesn't cause any problems that I can detect. Thanks! |
Fixes #2175
The regression is really subtle, so this PR definitely deserves a careful look-over. As Tim helpfully diagnosed in the issue, the underlying cause were the improvements from commit acaa51d .
The remaining subtlety seemed to be revealed when babel.sty loads its english
.inifile on a recent texlive (babel-en.ini). Every time that file read encounters a completely empty line, it seems to terminate the read early. Explaining how that happens is not something I can do with sufficient clarity at the moment, but it has to do with the interplay ofGullet::readUntil,Mouth::readTokenandMouth::getNextChar, where theundeffrom the empty line needs to be handled at just the right context point - or the assembled token list for the argument is incorrect, raising the higher level Errors of the regression.I arrived at the PR contents by experiment - trying to tweak in a variety of contexts and settling on the first reasonable combination that worked out. The key change was adding a
$$self{nchars} == 0check to the branch ofMouth::readTokenthat creates an EOL marker if a line is completely empty - tests continue to pass, and the regression is resolved.I also changed a couple of checks relying on
$chto instead rely ondefined $chalong the way, as I kept worrying about0chars.