174

I'm compiling one pretty simple LaTeX file with few dozen of pgfplots. Now, I can compile part by part, but am unable to compile it all at once. I really need good precision for all those plots (noise plots) and would like to avoid decimating graphs more. When I run compile, I get this error message:

[...] TeX capacity exceeded, sorry [main memory size=3000000].

Is there a way to avoid this message somehow? How am I supposed to use pgfplots if I can't plot couple dozen of figures with it without having TeX overloaded? How to expand TeX memory size, please?

Improve this question
1
  • One simple way of making tikz-figures created by matlab2tikz smaller is to use the command cleanfigure in Matlab, before using matlab2tikz. I got this advice from a colleague recently. Commented Oct 14, 2018 at 17:18

11 Answers 11

Reset to default
115

The pgfplots package can be particularly heavy on TeX's memory, especially if you are creating plots with lots of data points. Indeed, there is a section in the pgfplots manual about expanding TeX's memory. However, that does not mean that expanding TeX's memory is the best solution. Instead, I would recommend using the 'externalization' approach (section 7.1 of the pgfplots manual).

The idea of externalization is to compile each plot as a separate TeX job. This leads to a graphic which can be used in the main job. Thus each plot has its own memory requirement, separate from all of the other plots. This usually avoids needing to make TeX's memory bigger. At the same time, the resulting files can be kept between TeX runs, which will speed up compilation for the second and subsequent runs. The latest version of the externalization system needs you to do two things. First, you put

\usepgfplotslibrary{external} \tikzexternalize

in your preamble, to turn the system on. Secondly, you will need to enable 'shell escape'. This is done at the command line by adding the -shell-escape switch:

pdflatex -shell-escape <filename>

The same can be done in LaTeX editors: there is usually a place in the settings for this type of thing. I'll just add that shell escape does has some security implications: use only with documents that you trust!

Improve this answer
15
  • 7
    I should probably say that I saw it coming. I had the feeling that "expanding the memory size" is not the answer, and externalization works flawlessly - plus, every following pdflatex run is ridiculously faster! Thanks! ...I should only find a way now to forward -shell-escape from rubber --pdf command that I'm using to pdflatex... Commented Dec 30, 2010 at 9:59
  • 28
    Also, try the lualatex compiler which appears to dynamically allocate more memory if it needs it. Commented Apr 20, 2011 at 4:44
  • 10
    For everyone wondering why their images won't be updated after re-running pdflatex using the -shell-escape option: For every tikzpicture an extra .pdf file is created. Either delete the file, so it will be recreated next time you run pdflatex or put a \tikzset{external/force remake} somewhere in your document. Commented Jun 6, 2013 at 15:49
  • 15
    I tried this, but i get an error: ! Package tikz Error: Sorry, the system call 'pdflatex -shell-escape -halt-on-e rror -interaction=batchmode -jobname "chap_naturalite-figure0" "\def\tikzextern alrealjob{chap_naturalite}\input{chap_naturalite}"' did NOT result in a usable output file 'chap_naturalite-figure0' (expected one of .pdf:.jpg:.jpeg:.png:).. Tried to delete the existing files and force remake but it does not change the problem. Commented Sep 30, 2014 at 1:01
  • 5
    I have put the -shell-escape command for the build of pdflatex as you mentionned in your first post. My pdflatex command line is then written pdflatex -shell-escape -synctex=1 -interaction=nonstopmode %.tex. Commented Sep 30, 2014 at 18:14
87

Every now and then I have the same problem when compiling certain files. Of course, I have forgotten every time how I solved it in the past. Then, I google and I am getting to this page. Next, I remember that I landed on this page for several times, but, unfortunately, the answers did'nt help me in the past for my kind of problem. To break that Groundhog Day circle, I write down this answer which may help myself (for the next circle) and hopefully other people. The bad news is that it works only (?) for MikTeX users.

pdflatex --enable-write18 --extra-mem-bot=10000000 --synctex=1 <filename>

You may replace 10000000 by another ridiculous high number.

As was commented by @coatless, you may have to add --extra-mem-top=10000000 for other document types than my test example. The MiKTeX manual contains the following explanations:

  • --extra-mem-bot=n: Set the extra size (in memory words) for large data structures like boxes, glue, breakpoints, et al. Relevant only after the memory dump file has been read.

  • --extra-mem-top=n: Set the extra size (in memory words) for chars, tokens, et al. Relevant only after the memory dump file has been read.

For illustration some example log entries from a test document:

Failing example without the extra memory:

21229 strings out of 493921 360228 string characters out of 3152094 3000000 words of memory out of 3000000 24030 multiletter control sequences out of 15000+200000 49729 words of font info for 45 fonts, out of 3000000 for 9000 841 hyphenation exceptions out of 8191

Working example with --extra-mem-bot=10000000:

23652 strings out of 493921 401787 string characters out of 3152094 10571134 words of memory out of 13000000 24308 multiletter control sequences out of 15000+200000 49729 words of font info for 45 fonts, out of 3000000 for 9000 841 hyphenation exceptions out of 8191

Working example with --extra-mem-bot=100000000: 

23652 strings out of 493921 401787 string characters out of 3152094 100571134 words of memory out of 103000000 24308 multiletter control sequences out of 15000+200000 49729 words of font info for 45 fonts, out of 3000000 for 9000 841 hyphenation exceptions out of 8191

I guess that my example in truth needed 571134 words of memory (and the the rest is eaten up for fun).

Improve this answer
8
  • 10
    I had to use --extra-mem-top=10000000 instead of --extra-mem-bot=100000000 on MikTex x64 (Win 10) as the previous one didn't work well. Commented Feb 16, 2016 at 12:38
  • Many thanks @Coatless, I had the same issue, only worked with --extra-mem-bot Commented Sep 5, 2016 at 6:50
  • @Coatless Thanks. The --extra-mem-top solved it for me. Commented Nov 1, 2016 at 22:03
  • 1
    Working solution for me as a MikTeX user. I wish I had come to it earlier. It would have spared me a lot of time spent at debugging attempts on a long document I'm developing. Commented Jul 21, 2017 at 9:44
  • 2
    Sorry for my stupid question, but where do I type this? I am running Windows 7, Miktex 2.9 and Texmaker 4.5. Commented Nov 24, 2017 at 22:00
61

Due to AlexG's comment, the memory size has been a constant large value for recent versions of TeX Live. It seems you don't need to extend this.

In older TeX Live, you can modify /some/path/to/texlive/some/subpath/web2c/texmf.cnf, which can be found by typing on the terminal: 

kpsewhich -a texmf.cnf

Then modify the memory size by including, e.g. :

main_memory = 3000000

(See comments in texmf.cnf file for more details.)

Then run the following command as root to recreate the format files:

fmtutil-sys --all
Improve this answer
7
  • @Edin: your correction is wrong. The file mentioned by Leo Liu is the one to modify, as it's not touched by any TeX Live update. Commented May 4, 2011 at 8:41
  • 12
    The /some/path/to/texlive/2010/texmf/web2c/texmf.cnf is the official config file which will be overwritten in an update. It should not be modified. Instead the /some/path/to/texlive/2010/texmf.cnf should be used to change all settings which are different from the default. This is pointed out in the comments of the main web2c/texmf.cnf config file, which maybe is what Leo Lio meant in his last sentence. Commented May 4, 2011 at 10:23
  • 2
    You can also set higher values for extra_mem_bot and extra_mem_top, which will give you even more memory in total. Also the description is saying that the maximum value for main_memory is 7999999. Commented Feb 6, 2017 at 8:53
  • 2
    @LeoLiu : The maximum value main_memory = 12435455(to be found by bisection) has been a constant for many TL releases now. I think it is worth noting in the answer. Commented Jun 16, 2017 at 7:10
  • 1
    This worked for me amazingly well, except it took me a while to find a place where to put it: TeXMaker/Options/Configure Texmaker/Commands. Also had the problem to specify <filename> so now every compilation of files that need the extra memory I get !Latex Error: File 'MyTextfile.txt' not found, but the pdf file is generated correctly. When compiling other, smaller, projects, no error occurs. The interaction between Texmaker and MikTeX has me totally confused. I would have assumed Texmaker Commands settings are across the board, not for specific .tex files. Commented Nov 23, 2022 at 5:45
44

after trying all the options (changing texmf.cnf file, etc.) I found on the web, the only one that worked for me was to use lualatex. This is the simplest and more elegant way to do it. Lualatex is available in the Texlive distribution by default.

Improve this answer
2
  • 1
    see macosx-tex.576846.n2.nabble.com/Lua-td6030741.html for instructions on using lualatex on a mac ::: note: in pdfLuaLatex.engine you may need to change pdflualatex -synctex=1 "$1" to lualatex -synctex=1 "$1" Commented Jan 4, 2013 at 13:19
  • I learnt this after 8 years of making compromises. Must be upvoted more Commented Mar 28 at 14:31
14

In MikTex, the memory size can be permanently increased by this way:

1, For Windows, Open a command Window and enter:

initexmf --edit-config-file=latex

For Ubuntu, try this:

gedit .miktex/texmfs/config/miktex/config/latex.ini

  1. In the file opened (it may be empty), add:

    main_memory=12000000

  2. Save the file and quit. Then in the command window, enter:

    initexmf --dump=latex

Then repeat the above steps for pdflatex (replace latex with pdflatex). If the error persists, try increasing the main_memory value. For lualatex, the memory allocation is dynamic, hence the aforementioned steps are unnecessary, but if the compilation fails for lualatex, try for that as well.

Improve this answer
6

Just to add to this if anyone still has this problem, I found a pretty simple way to fix it. I was getting this error:

TeX capacity exceeded, sorry [main memory size=5000000]

I went through a lot of guides to increase the main memory and nothing was working :/

But I read a few articles where they talked about save_size instead of main_memory.

I edited my texmf.cnf but not the one in the web2c folder, I changed the at in the ..../texlive/yyyy/texmf.cnf (since I'm using Windows my path was C:\texlive\2015\texmf.cnf). The file looked as follows:

% (Public domain.) % This texmf.cnf file should contain only your personal changes from the % original texmf.cnf (for example, as chosen in the installer). % % That is, if you need to make changes to texmf.cnf, put your custom % settings in this file, which is .../texlive/YYYY/texmf.cnf, rather than % the distributed file (which is .../texlive/YYYY/texmf-dist/web2c/texmf.cnf). % And include *only* your changed values, not a copy of the whole thing! % OSFONTDIR = $SystemRoot/fonts//

So I added the following two lines:

main_memory = 7999999 save_size = 7999999

I then opened up command prompt and ran:

fmtutil-sys --all

And that was it :) I was able to build my stuff! I know that save_size wasn't mentioned at all in the error message but I found that adding that line fixed the issue :)!

Improve this answer
6

It wasn't working for me either (MikTex on Win10 x64). I tried to change the pdflatex.ini file (see here point 4) but ended with the error after the dump:

Sorry, but "C:\Program Files\MiKTeX 2.9\miktex\bin\x64\miktex-pdftex.exe" did not succeed.

Therefore I did following: MikeTex Settings -> Formats, looked for pdflatex and clicked "Build". Again an error, but then I opened the pdflatex.ini, changed line for line the maximum value and looked up to which it still was able to succesful "Build".

Now it works with these values (8GB RAM):

main_memory=79999999 save_size=79999 extra_mem_bot=79999999 extra_mem_top=79999999 font_mem_size=799999 pool_size=7999999 buf_size=7999999 EnableWrite18=p stack-size=200000 max-strings=79999999
Improve this answer
1
  • So, that is the largest? How to set for XeLaTeX? Commented Sep 26, 2023 at 2:34
2

Try with \clearpage command, it worked for me.

Improve this answer
1
  • 2
    Welcome to tex.sx. It's possible that splitting up the processing of plots could decrease the need for more memory, but it might also result in making it impossible to place a;; plots where they are wanted. More details, please. Commented Sep 28, 2021 at 14:54
1

By writing my document I also got this error. I found a nice workaround without using LuaLaTeX, changing any properties or making somehow a picture of your desired diagram. You can still using pdfLaTeX.

The following steps describe how you have to proceed:

(1)

As usual, create your *.tex-file with matlab2tikz.

(2)

Create a minimal working example (MWE) like the one. The example has some additional packages with the standalone class too remove unneeded margin: 

\documentclass[preview]{standalone} %I don't know, why I have to include the preview as an optional argument. Without I got an error. \usepackage{graphicx} \usepackage{pgfplots} \pgfplotsset{compat=1.16} \usepackage{siunitx} \begin{document} \begin{figure}[h] \input{< your generated *.tex-file >} \end{figure} \end{document}

(3)

With step (2) you can now include the generated *.pdf-file in your "normal" document:

... \begin{figure}[h] \includegraphics{< your generated *.pdf-file from step (2) >} \caption[short text]{longer text} \label{fig:figure1} \end{figure} ...

As you can see the advantages of this methode is that you can also include captions and labels.

Improve this answer
1
  • Can you further explain how this help with capacity issues? Commented Sep 15, 2021 at 16:08
0

If you are using TeXstudio go to Options > Configure TeXstudio, and on the Build tab change the Default Compiler from PdfLaTex to LuaLaTeX.

This solved my problem, I hope it can help somebody out there.

Improve this answer
0

I had trouble to find how to increase the memory in a container of github action like mattnotmitt/doxygen-action, so saving for the future, in Doxyfile put this line:

LATEX_CMD_NAME = "\"pdflatex\" -cnf-line=extra_mem_top=12000000 -cnf-line=extra_mem_bot=12000000 -cnf-line=main_memory=12000000"
Improve this answer

You must log in to answer this question.