Re: etex and pdftex have diverging memory constraints?

On Wed, Aug 27, 2025 at 16:07 (+0200), jfbu wrote:



>> Le 27 août 2025 à 15:59, Jim Diamond <[email protected]> a écrit :

>> On Wed, Aug 27, 2025 at 15:15 (+0200), jfbu via tex-live wrote:

>>> Hi,

>>> Sorry if this is very basic, but I always thought the sole difference between etex and pdftex is that the former works to produce a dvi and the latter a pdf.

>>> So, I was surprised to discover (or perhaps I observed that years ago and forgot) that pdftex has slightly more constrained memory constraints on macro expansion.

>>> Here is a test file testxintLength.tex

>>> ----
>>> \input xintkernel.sty
>>> \edef\x{\romannumeral\xintreplicate{%     
>>> 2492679%
>>> }{F}}
>>> \expandafter\xintLength\expandafter{\x}
>>> \bye
>>> ----

>>> (the code creates a macro \x expanding to 2492679 F's then attempts to measure how many F's there are).

>>> Compilation with etex succeeds  (even with 2492694 in place of 2492679).

>>> $ etex testxintLength.tex
>>> This is pdfTeX, Version 3.141592653-2.6-1.40.28 (TeX Live 2025) (preloaded format=etex)
>>> restricted \write18 enabled.
>>> entering extended mode
>>> (./testxintLength.tex (./xintkernel.sty) [1] )
>>> Output written on testxintLength.dvi (1 page, 216 bytes).
>>> Transcript written on testxintLength.log.

>>> But compilation with pdftex fails.

>>> $ pdftex testxintLength
>>> This is pdfTeX, Version 3.141592653-2.6-1.40.28 (TeX Live 2025) (preloaded format=pdftex)
>>> restricted \write18 enabled.
>>> entering extended mode
>>> (./testxintLength.tex (./xintkernel.sty)
>>> ! TeX capacity exceeded, sorry [main memory size=5000000].
>>> <inserted text> 2492679
>>>                        \xintlength ...xint_c_i \xint_c_ \xint_bye \relax                                                     l.5 \expandafter\xintLength\expandafter{\x}
>>>                                            !  ==> Fatal error occurred, no output PDF file produced!
>>> Transcript written on testxintLength.log.

>>> I am a bit surprised because I can not connect the constraints on TeX macro expansion with the output format DVI or PDF.  Needless to say I am completely ignorant of the innards.

>> Pdftex works fine on this example (Slackware64 15.0), at least in the sense
>> that pdftex runs to completion.  The output PDF doesn't show the F's, at
>> least with the viewer I tried.

>> Perhaps if you mention what OS you are using some additional help might be
>> forthcoming.


> Hello I am on macOS (as Herbert who also says the file compiles on his system).

I saw Herbert's reply after I initially replied.

As I recall from the last time I had the "pleasure" of using MacOS, is it
not the case that Apple is getting paranoid about any programs you didn't
get from them?  If so, is there some possibility that different ways of
installing programs give different permissions or capabilities of the
program?

> The output PDF will contain simply the number 2492679 and the page number.

Mea culpa.  I decided to experiment further by adding

\x

to the file before \bye, and was so intrigued (*cough*) about the missing
text that I forgot your original test case.

And for certainty, your original unmodified test case runs to successful
completion on my system.

> On my machine I have confirmed via modifying main_memory in texmf.cnf and
> recompiling the etex and pdftex formats that the limits (a bit lower for
> pdftex) are about half the value of main_memory parameter.

I haven't even looked vaguely in the direction of TeX internals in 25 or 30
years, but presumably "half" has to do with using 16 bits for each
character in that string?  (I'm sure there are people reading this who can
straighten me out if that is utter nonsense.)

> I am intrigued by yours and Herbert report that compilation succeeds.  I
> was working with the dev version of xint, but the used macros have had no
> modification but to be sure I checked out the 1.4m from 2022 which is
> last release and as expected I got same result.

I have an unmolested version of texlive.  I haven't updated in a week or
three, however.

> It is unavoidable that a crash occurs with etex/pdftex if the 2492679 is
> replaced by some large enough number, so if you have time to waste on
> this you can try to set it to a higher value.

2493679 works
2495679 fails

Doing a binary search (data structures and algorithms class wasn't a
waste!) I see 2494144 works but 2494143 fails.  I don't think I can get a
paper out of this earth-shattering research, but maybe a conference
publication. :-)


> I am surprised the upper limit at my locale is lower than what you and
> Herbert experiment... and Herbert confirmed being with TL default memory
> settings like myself.

When mine fails, it says
        ! TeX capacity exceeded, sorry [main memory size=5000000].



"I don't have any solution, but I certainly admire the problem."
        --- Ashleigh Brilliant 


Good luck with the hunt.

                                Jim