Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not possible to use mylatex.ltx with xelatex #15

Open
jfbu opened this issue Jun 1, 2018 · 20 comments
Open

Not possible to use mylatex.ltx with xelatex #15

jfbu opened this issue Jun 1, 2018 · 20 comments

Comments

@jfbu
Copy link

jfbu commented Jun 1, 2018

Consider this file abc.tex

\documentclass{article}
\begin{document}
abc
\end{document}

and try to generate a format:

$ xetex -ini \&xelatex mylatex.ltx abc
This is XeTeX, Version 3.14159265-2.6-0.99999 (TeX Live 2018) (INITEX)
 restricted \write18 enabled.
entering extended mode
(/usr/local/texlive/2018/texmf-dist/tex/latex/carlisle/mylatex.ltx
LaTeX2e <2018-04-01> patch level 5
) (./abc.tex (/usr/local/texlive/2018/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
(/usr/local/texlive/2018/texmf-dist/tex/latex/base/size10.clo)) )
Beginning to dump on file mylatex.fmt
 (preloaded format=mylatex 2018.6.1)
71708 strings of total length 90452
59166 memory locations dumped; current usage is 348&56891
4228 multiletter control sequences
\font\nullfont=nullfont
\font\OMX/cmex/m/n/5=cmex10
\font\tenln=line10
\font\tenlnw=linew10
\font\tencirc=lcircle10
\font\tencircw=lcirclew10
\font\OT1/cmr/m/n/5=cmr5
\font\OT1/cmr/m/n/7=cmr7
\font\OT1/cmr/m/n/10=cmr10
\font\OML/cmm/m/it/5=cmmi5
\font\OML/cmm/m/it/7=cmmi7
\font\OML/cmm/m/it/10=cmmi10
\font\OMS/cmsy/m/n/5=cmsy5
\font\OMS/cmsy/m/n/7=cmsy7
\font\OMS/cmsy/m/n/10=cmsy10
\font\TU/lmr/m/n/10=[lmroman10-regular]:mapping=tex-text;
! Can't \dump a format with native fonts or font-mappings.
<*> &xelatex mylatex.ltx abc
                            
? H
You really, really don't want to do this.
It won't work, and only confuses me.
(Load them at runtime, not as part of the format file.)

? X
No pages of output.
Transcript written on mylatex.log.
@davidcarlisle
Copy link
Owner

davidcarlisle commented Jun 1, 2018

It probably works to set to OT1 in the format and back to TU in everyjob, combined with not preloading fonts: could you test

diff -c mylatex.ltx~ mylatex.ltx


*** mylatex.ltx~	2018-06-01 11:17:54.186705500 +0100
--- mylatex.ltx	2018-06-01 11:36:38.729753400 +0100
***************
*** 117,122 ****
--- 117,124 ----
  % Save the original definitions.
  \let\MYLATEXdocument\document
  \let\MYLATEXopenout\openout
+ \def\encodingdefault{OT1}\fontencoding{OT1}\selectfont
+ 
  
  % The version of \document to use on the initex run.
  % Just preloads some fonts, puts back \document and \openout,
***************
*** 125,130 ****
--- 127,133 ----
  % skipped on normal runs with the new format.
  \def\document{\endgroup
  % Force some font preloading.
+ \ifx\Umathchar\@undefined
   {\setbox\z@\hbox{%
      $$% math (not bold, some setups don't have \boldmath)
      \normalfont% normal
***************
*** 134,144 ****
      {\itshape}% italic
      \ttfamily% monospace
      \sffamily% sans serif
!     }}% 
    \let\document\MYLATEXdocument
    \let\openout\MYLATEXopenout
    \makeatother
    \everyjob\expandafter{\the\everyjob
       \begingroup
        \listfiles
        \expandafter\MYLATEXcustomised\@dofilelist
--- 137,149 ----
      {\itshape}% italic
      \ttfamily% monospace
      \sffamily% sans serif
!     }}%
! \fi
    \let\document\MYLATEXdocument
    \let\openout\MYLATEXopenout
    \makeatother
    \everyjob\expandafter{\the\everyjob
+   \def\encodingdefault{TU}\fontencoding{TU}\selectfont 
       \begingroup
        \listfiles
        \expandafter\MYLATEXcustomised\@dofilelist


@jfbu
Copy link
Author

jfbu commented Jun 1, 2018

The patched latex.ltx appears to work: generation of format defining macro \foo worked, and I could use it with another file using \foo in document body.

@jfbu
Copy link
Author

jfbu commented Jun 1, 2018

The original had not issue with lualatex. The patch seems to treat xelatex and lualatex engine alike. Is anything lost on lualatex side by so-doing?

@davidcarlisle
Copy link
Owner

luatex may not give an error but I don't think it can work, to load an opentype font it would go through luaotfload and that lua state is not saved in the format so even if you don't get an error it's probably in an unstable state when it restarts. No time to test now, If you do any tests feel free to comment here. I'll look later.

@jfbu
Copy link
Author

jfbu commented Jun 1, 2018

I was going to test luatex with OpenType fonts, but already xetex fails, even with patch from your comment

\documentclass{article}
\def\abc{abc}
\usepackage{fontspec}
\setmainfont{Hoefler Text}% I am on a Mac
\begin{document}
\end{document}

produces (with patched mylatex.ltx)

$ xetex -ini \&xelatex mylatex.ltx abc
This is XeTeX, Version 3.14159265-2.6-0.99999 (TeX Live 2018) (INITEX)
 restricted \write18 enabled.
entering extended mode
(./mylatex.ltx
LaTeX2e <2018-04-01> patch level 5
(/usr/local/texlive/2018/texmf-dist/tex/latex/lm/ot1lmr.fd)) (./abc.tex
(/usr/local/texlive/2018/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
(/usr/local/texlive/2018/texmf-dist/tex/latex/base/size10.clo))
(/usr/local/texlive/2018/texmf-dist/tex/latex/fontspec/fontspec.sty
(/usr/local/texlive/2018/texmf-dist/tex/latex/l3packages/xparse/xparse.sty
(/usr/local/texlive/2018/texmf-dist/tex/latex/l3kernel/expl3.sty
(/usr/local/texlive/2018/texmf-dist/tex/latex/l3kernel/expl3-code.tex)
(/usr/local/texlive/2018/texmf-dist/tex/latex/l3kernel/l3xdvipdfmx.def)))
(/usr/local/texlive/2018/texmf-dist/tex/latex/fontspec/fontspec-xetex.sty
(/usr/local/texlive/2018/texmf-dist/tex/latex/base/fontenc.sty
(/usr/local/texlive/2018/texmf-dist/tex/latex/base/tuenc.def))
(/usr/local/texlive/2018/texmf-dist/tex/latex/fontspec/fontspec.cfg))) )
Beginning to dump on file mylatex.fmt
 (preloaded format=mylatex 2018.6.1)
82519 strings of total length 333172
310190 memory locations dumped; current usage is 364&304877
14995 multiletter control sequences
\font\nullfont=nullfont
\font\OMX/cmex/m/n/10=cmex10
\font\tenln=line10
\font\tenlnw=linew10
\font\tencirc=lcircle10
\font\tencircw=lcirclew10
\font\OT1/cmr/m/n/5=cmr5
\font\OT1/cmr/m/n/7=cmr7
\font\OT1/cmr/m/n/10=cmr10
\font\OML/cmm/m/it/5=cmmi5
\font\OML/cmm/m/it/7=cmmi7
\font\OML/cmm/m/it/10=cmmi10
\font\OMS/cmsy/m/n/5=cmsy5
\font\OMS/cmsy/m/n/7=cmsy7
\font\OMS/cmsy/m/n/10=cmsy10
\font\OT1/lmr/m/n/10=rm-lmr10
\font\c__fp_exp_intarray=cmr10 at 0.00002pt
\font\c__fp_trig_intarray=cmr10 at 0.00003pt
\font\g__regex_charcode_intarray=cmr10 at 0.00005pt
\font\g__regex_catcode_intarray=cmr10 at 0.00006pt
\font\g__regex_balance_intarray=cmr10 at 0.00008pt
\font\g__regex_state_active_intarray=cmr10 at 0.00009pt
\font\g__regex_thread_state_intarray=cmr10 at 0.0001pt
\font\g__regex_submatch_prev_intarray=cmr10 at 0.00012pt
\font\g__regex_submatch_begin_intarray=cmr10 at 0.00014pt
\font\g__regex_submatch_end_intarray=cmr10 at 0.00015pt
\font\TU/lmr/m/n/10=[lmroman10-regular]:mapping=tex-text;
! Can't \dump a format with native fonts or font-mappings.
<*> &xelatex mylatex.ltx abc
                            
? 

@jfbu
Copy link
Author

jfbu commented Jun 1, 2018

For info, minimal testing of lualatex with non-patched mylatex.ltx works.

File abc.tex:

\documentclass{article}
\def\abc{abc}
\usepackage{fontspec}
\setmainfont{Hoefler Text}
\begin{document}
\end{document}

then

$ luatex -ini \&lualatex mylatex.ltx abc
This is LuaTeX, Version 1.07.0 (TeX Live 2018)  (INITEX)
 restricted system commands enabled.
(/usr/local/texlive/2018/texmf-dist/tex/latex/carlisle/mylatex.ltx
LaTeX2e <2018-04-01> patch level 5
(using write cache: /usr/local/texlive/2018/texmf-var/luatex-cache/generic)(usi
ng read cache: /usr/local/texlive/2018/texmf-var/luatex-cache/generic /Users/jf
b/Library/texlive/2018/texmf-var/luatex-cache/generic)
luaotfload | main : initialization completed in 0.110 seconds) (./abc.tex
(/usr/local/texlive/2018/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
(/usr/local/texlive/2018/texmf-dist/tex/latex/base/size10.clo(load luc: /usr/lo
cal/texlive/2018/texmf-var/luatex-cache/generic/fonts/otl/lmroman10-regular.luc
))) (/usr/local/texlive/2018/texmf-dist/tex/latex/fontspec/fontspec.sty
(/usr/local/texlive/2018/texmf-dist/tex/latex/l3packages/xparse/xparse.sty
(/usr/local/texlive/2018/texmf-dist/tex/latex/l3kernel/expl3.sty
(/usr/local/texlive/2018/texmf-dist/tex/latex/l3kernel/expl3-code.tex)
(/usr/local/texlive/2018/texmf-dist/tex/latex/l3kernel/l3pdfmode.def)))
(/usr/local/texlive/2018/texmf-dist/tex/latex/fontspec/fontspec-luatex.sty
(/usr/local/texlive/2018/texmf-dist/tex/latex/base/fontenc.sty
(/usr/local/texlive/2018/texmf-dist/tex/latex/base/tuenc.def))
(/usr/local/texlive/2018/texmf-dist/tex/latex/fontspec/fontspec.cfg)(load luc: 
/usr/local/texlive/2018/texmf-var/luatex-cache/generic/fonts/otl/lmroman10-bold
.luc)(load luc: /usr/local/texlive/2018/texmf-var/luatex-cache/generic/fonts/ot
l/lmroman10-italic.luc)))(load luc: /usr/local/texlive/2018/texmf-var/luatex-ca
che/generic/fonts/otl/hoefler-text-1.luc)

LaTeX Font Warning: Font shape `TU/HoeflerText(0)/bx/n' undefined
(Font)              using `TU/HoeflerText(0)/m/n' instead on input line 5.


LaTeX Font Warning: Font shape `TU/HoeflerText(0)/bx/it' undefined
(Font)              using `TU/HoeflerText(0)/bx/n' instead on input line 5.


LaTeX Font Warning: Font shape `TU/HoeflerText(0)/m/it' undefined
(Font)              using `TU/HoeflerText(0)/m/n' instead on input line 5.

(load luc: /usr/local/texlive/2018/texmf-var/luatex-cache/generic/fonts/otl/lmm
ono10-regular.luc)(load luc: /usr/local/texlive/2018/texmf-var/luatex-cache/gen
eric/fonts/otl/lmsans10-regular.luc))
Beginning to dump on file mylatex.fmt
 (format=mylatex 2018.6.1)
16343 strings using 292376 bytes
684245 memory locations dumped; current usage is 322&340752
14898 multiletter control sequences
\font\nullfont=nullfont
\font\OMX/cmex/m/n/10=cmex10
\font\tenln=line10
\font\tenlnw=linew10
\font\tencirc=lcircle10
\font\tencircw=lcirclew10
\font\OT1/cmr/m/n/5=cmr5
\font\OT1/cmr/m/n/7=cmr7
\font\OT1/cmr/m/n/10=cmr10
\font\OML/cmm/m/it/5=cmmi5
\font\OML/cmm/m/it/7=cmmi7
\font\OML/cmm/m/it/10=cmmi10
\font\OMS/cmsy/m/n/5=cmsy5
\font\OMS/cmsy/m/n/7=cmsy7
\font\OMS/cmsy/m/n/10=cmsy10
\font\TU/lmr/m/n/10=[lmroman10-regular]:+tlig;
\font\c__fp_exp_intarray=cmr10 at 0.00002pt
\font\c__fp_trig_intarray=cmr10 at 0.00003pt
\font\g__regex_charcode_intarray=cmr10 at 0.00005pt
\font\g__regex_catcode_intarray=cmr10 at 0.00006pt
\font\g__regex_balance_intarray=cmr10 at 0.00008pt
\font\g__regex_state_active_intarray=cmr10 at 0.00009pt
\font\g__regex_thread_state_intarray=cmr10 at 0.0001pt
\font\g__regex_submatch_prev_intarray=cmr10 at 0.00012pt
\font\g__regex_submatch_begin_intarray=cmr10 at 0.00014pt
\font\g__regex_submatch_end_intarray=cmr10 at 0.00015pt
\font\TU/lmr/bx/n/10=[lmroman10-bold]:+tlig;
\font\TU/lmr/m/it/10=[lmroman10-italic]:+tlig;
\font\l_fontspec_font=HoeflerText
\font\TU/HoeflerText(0)/bx/n/10=HoeflerText:mode=node;+tlig;
\font\OT1/cmr/m/n/10=cmr10
\font\OT1/cmr/m/n/7=cmr7
\font\OT1/cmr/m/n/5=cmr5
\font\OML/cmm/m/it/10=cmmi10
\font\OML/cmm/m/it/7=cmmi7
\font\OML/cmm/m/it/5=cmmi5
\font\OMS/cmsy/m/n/10=cmsy10
\font\OMS/cmsy/m/n/7=cmsy7
\font\OMS/cmsy/m/n/5=cmsy5
\font\OMX/cmex/m/n/5=cmex10
\font\TU/HoeflerText(0)/m/n/12=HoeflerText:mode=node;+tlig; at 12.0pt
\font\TU/HoeflerText(0)/m/n/8=HoeflerText:mode=node;+tlig; at 8.0pt
\font\TU/lmtt/m/n/10=[lmmono10-regular]:
\font\TU/lmss/m/n/10=[lmsans10-regular]:+tlig;
43 preloaded fonts
warning  (pdf backend): no pages of output.
Transcript written on mylatex.log.

then consider file abctest.tex:

\documentclass{article}
\begin{document}
\abc
\end{document}

It works

$ lualatex \&mylatex abctest
This is LuaTeX, Version 1.07.0 (TeX Live 2018) 
 restricted system commands enabled.
(./abctest.tex
LaTeX2e <2018-04-01> patch level 5
(using write cache: /usr/local/texlive/2018/texmf-var/luatex-cache/generic)(usi
ng read cache: /usr/local/texlive/2018/texmf-var/luatex-cache/generic /Users/jf
b/Library/texlive/2018/texmf-var/luatex-cache/generic)
luaotfload | main : initialization completed in 0.114 seconds
CUSTOMISED FORMAT. Preloaded files:
        .
 article.cls    2014/09/29 v1.4h Standard LaTeX document class
  size10.clo    2014/09/29 v1.4h Standard LaTeX file (size option)
fontspec.sty    2017/11/09 v2.6g Font selection for XeLaTeX and LuaLaTeX
  xparse.sty    2018-05-12 L3 Experimental document command parser
   expl3.sty    2018-05-13 L3 programming layer (loader) 
expl3-code.tex    2018-05-13 L3 programming layer 
l3pdfmode.def    2018-05-13 v L3 Experimental driver: PDF mode
fontspec-luatex.sty    2017/11/09 v2.6g Font selection for XeLaTeX and LuaLaTeX

 fontenc.sty
   tuenc.def    2017/04/05 v2.0i Standard LaTeX file
fontspec.cfg
        .
(./abctest.aux) [1{/usr/local/texlive/2018/texmf-var/fonts/map/pdftex/updmap/pd
ftex.map}] (./abctest.aux)

LaTeX Font Warning: Some font shapes were not available, defaults substituted.

)
 393 words of node memory still in use:
   3 hlist, 1 vlist, 1 rule, 2 glue, 4 attribute, 50 glue_spec, 4 attribute_lis
t, 1 write nodes
   avail lists: 2:17,3:3,4:1,5:21,6:1,7:18,8:2,9:8
</Library/Fonts/Hoefler Text.ttc>
Output written on abctest.pdf (1 page, 6780 bytes).
Transcript written on abctest.log.

Side note: I tried with

%&mylatex
\documentclass{article}
\begin{document}
\abc
\end{document}

but then lualatex abctest fails. This surprises me but may be a TeXLive issue with lualatex not handling first line format. (analog with pdflatex does work).

@davidcarlisle
Copy link
Owner

davidcarlisle commented Jun 1, 2018 via email

@jfbu
Copy link
Author

jfbu commented Jun 1, 2018

How do I trigger luacode with luatex? simply loading Hoefler Text.ttc or STIXGeneral.otf (these are system wide fonts on my mac os) via \fontspec plus \setmainfont so far worked in my testing.

@davidcarlisle
Copy link
Owner

davidcarlisle commented Jun 1, 2018 via email

@jfbu
Copy link
Author

jfbu commented Jun 1, 2018

I see, thanks for explaining, not being a practitioner of lua, the obvious escaped me...

@Guekka
Copy link

Guekka commented Jan 14, 2021

Hey, came here trying to make a precompiled preamble with XeLaTeX. The above patch worked for me. Is there a reason for not making it the default?

@davidcarlisle
Copy link
Owner

@Guekka mostly just that I haven't really updated mylatex.ltx ever.

I'm not sure the patch really is useful in many contexts, most xelatex files will load some opentype fonts in the preamble, so it is rather limited what you can really save in a custom format without danger of changing the formatting somewhere. But I left this issue open, perhaps I could be persuaded...

xetex is a bit easier to make work than luatex, as it is "just" the fonts you can not dump. in luatex you can not dump lua state either and that is much harder to avoid.

@Guekka
Copy link

Guekka commented Jan 14, 2021

Well, this patch allowed me to precompile my preamble. It is 500 lines long, and I only had to move two lines to another file.
(\usepackage{fontspec} and \usepackage{fontawesome})
My compile times were reduced from 2.1s to 0.9s, which makes a real difference when taking notes.

So I cannot speak for luatex, but it would be a welcome addition for xetex.

@davidcarlisle
Copy link
Owner

@Guekka I suppose so, I'll see what I can do. I had thought that mylatex wasn't that useful in modern times (I wrote it for friends using old MSDOS implementations (when I had a Sun 3) and this saved about 5 minutes per run at the time:-) But I agree in today's less patient world, saving a couple of seconds is still worth it... (it is likely to be more than that if you use tikz)

@jfbu
Copy link
Author

jfbu commented Jan 15, 2021

@davidcarlisle Hi David, only to mention that mylatex is used in preview.sty and the Emacs/AUCTeX system so changes to it should preferably be tested against these contexts and signaled to their maintainers before roll-out in case something is not backwards compatible. By the way Emacs/AUCTeX goes to certain lengths to make sure mylatex can be used with filenames having spaces or non-ascii characters so in particular if we you address this maintainers overthere should be pinged. Best,

@jfbu
Copy link
Author

jfbu commented Jan 15, 2021

@davidcarlisle my wording is a bit confused (as a rule in this whole thread, again) and I meant to say that Emacs/AUCTeX uses the preview.sty + mylatex.ltx combination, not that preview.sty relied on mylatex, sorry.

edit : damn'it, again I am not precise : Emacs/AUCTeX does that only optionally.

@davidcarlisle
Copy link
Owner

@jfbu noted. The only change (should) be to make cases that don't work at all currently with xetex work, so compatibility shouldn't be an issue but tex being tex it is never that simple:-)

@ikumikeita
Copy link

ikumikeita commented Jun 15, 2022

Hi,
I'm not sure whether this is a right place to ask. If not please forgive me.

Consider the following examples:

--- abc.tex ---

\documentclass{article}
\begin{document}
abc
\end{document}

--- abc.ini ---

\input mylatex.ltx \relax%

Make a subdirectory named "build" by mkdir build.

Then the following command leads to error:
lualatex -ini --output-directory=build/ \&lualatex abc.ini "\nonstopmode" "\input" abc.tex
The output is

This is LuaHBTeX, Version 1.15.0 (TeX Live 2022)  (INITEX)
 restricted system commands enabled.
(./abc.ini
LaTeX2e <2021-11-15> patch level 1
 L3 programming layer <2022-02-24>
(/usr/local/texlive/2022/texmf-dist/tex/latex/mylatex/mylatex.ltx))
! I can't find file `'.
<to be read again> 
\nonstopmode 
<*> &lualatex abc.ini \nonstopmode
                                \input abc.tex
(Press Enter to retry, or Control-D to exit)
Please type another input file name:^D  
! Emergency stop.
<to be read again> 
\nonstopmode 
<*> &lualatex abc.ini \nonstopmode
                                \input abc.tex
 301 words of node memory still in use:
   1 hlist, 1 dir, 3 kern, 1 glyph, 1 attribute, 39 glue_spec, 1 attribute_list
 nodes
   avail lists: 2:10,3:3,4:1
!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on abc.log.

However, any of the following variants runs without error:
lualatex -ini \&lualatex abc.ini "\nonstopmode" "\input" abc.tex (omitting --output-directory option)
lualatex -ini --output-directory=build/ \&lualatex abc.ini abc.tex (without \TeXcode \input syntax)
pdflatex -ini --output-directory=build/ \&pdflatex abc.ini "\nonstopmode" "\input" abc.tex (pdflatex instead of lualatex)
In other words, the error occurs only with the combination of lualatex, \TeXcode \input and --output-directory option. This affects preview-latex, subsystem of AUCTeX: https://lists.gnu.org/archive/html/auctex/2022-06/msg00020.html

Is there any fix or workaround for this error? (Whom (where) should I ask about this issue? Here? Lualatex developers? Luatex developers?)

@davidcarlisle
Copy link
Owner

@ikumikeita this is the right repository, but the wrong issue thread as the issues with luatex and xetex are very different, please keep this one to xetex.

With luatex, it is very hard (with or without mylatex.ltx) to dump package code in a format. You can not save Lua state or OpenType fonts in the format, so typically you need to alter the package code to delay Lua and font loading to \everyjob this may require an arbitrary amount of code change in each package. Some packages you can simply redfine some high level interface commands, and some you need to make low level changes, and some it is hard/impossible to make work at all.....

So feel free to open issues here for LuaTeX if only as a place of recording issues, but the issues may remain unresolved. That is just the way it is. Unless LuaTeX itself changes to support saving Lua state in the format

@ikumikeita
Copy link

@davidcarlisle Thank you for your kind reply. I'll open a new issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants