Optimizing Your LaTeX Workflow: A Guide to Choosing a Build System

Martin Isaksson

15 min read October 1, 2023

Categories

• academia
• latex

Tags

• latex

Table of contents

1. Introduction
2. Background
   1. Why don’t just use Overleaf?
   2. Installing LaTeX
   3. Building a LaTeX document
3. Method
   1. The document
   2. Timing measurements
4. Experiments
   1. Running bare commands
   2. arara
   3. latexmk
   4. rubber and rubber-info
   5. scons
   6. latexrun
   7. Using a precompiled preamble
   8. Ramdisk
   9. Flags
   10. Dvips
5. Results
   1. Build times
   2. Manageable output
   3. Ease of use
   4. File size
   5. Final score
6. Related work
7. Discussion and limitations
8. Conclusion

Long L^aT_eX build times can be a significant challenge for researchers and developers, hampering productivity and efficiency. This issue arises due to the complexity of L^aT_eX documents and the diversity of build systems available. We present a comprehensive exploration of L^aT_eX build systems, helping authors choose the most suitable one. By identifying the best build system, authors can streamline their workflow, reduce build times, and ultimately enhance their research and development endeavors.

Introduction

I still remember the issues I had, many years ago, trying to build my Master thesis report using latex and dvips. Since then my build process has gone through a few iterations, to simplify, to make it faster, or to accommodate some new thing that I learned. Frustrated by the long build times, I have tried many things which may or may not have helped. The discussion (for example  Speeding up L^aT_eX compilation) seems to be plagued by hearsay. For a typesetting system that is used by scientists, the discussion is unexpectedly remarkably unscientific.

In this blog post we will embark on an exploration of this rabbit hole, unraveling the intricacies of different L^aT_eX build systems. We will dive deep into the world of L^aT_eX compilation methods, comparing and contrasting different local build methods. By the end, you’ll be equipped with the knowledge to choose the right path for your next L^aT_eX adventure.

Background

Why don’t just use Overleaf?

 Overleaf is fantastic, but a local L^aT_eX build environment can offer advantages over Overleaf in certain situations. A key benefit is control. With a local setup, users have complete control over their L^aT_eX distribution, packages, configuration and development environment. For me, the enhanced privacy and security is the most important reason to using a local build environment.

It is important to note that  Overleaf excels in collaborative and cloud-based scenarios when teams working on L^aT_eX documents can work simultaneously in the same document.  Overleaf allows real-time collaboration with version control and seamless sharing and also eliminates the need to install and manage L^aT_eX distributions and packages, making it accessible to users who may not be experienced installing complex software such as L^aT_eX.

Installing L^aT_eX

Your method of installing L^aT_eX varies with your operating system, the level of control you want, and other constraints. We will not go into this in detail here, see Getting L^aT_eX and Free TeX implementations.

Building a L^aT_eX document

L^aT_eX employs a multi-pass typesetting process to enable various features like table of contents, lists of figures, cross-referencing, glossaries, indexing, and bibliographic citations. In this process, the data generated during one pass (compilation) is saved to intermediate files and then serves as input for any subsequent passes.

graph LR; .tex-->|latex|.dvi; .tex-->|pdflatex, xelatex|.pdf; .dvi-->|dvips|.ps; .ps-->|ps2pdf|.pdf; .dvi-->|dvipdfm, dvipdfmx|.pdf;

We have many choices to make here (see  What are the pros and cons pertaining to “latex -> dvipdfm” versus “latex -> dvips -> ps2pdf”? ), but in this case we know we want a Portable Document Format (PDF), .pdf, file. There are some differences depending on which workflow we pick here, but for this document we can pick any of them.

The multi-pass typesetting process involves several steps to create a PDF document with references using BibTeX and glossaries using makeglossaries:

1. Compilation with L^aT_eX. First we run pdflatex on the input .tex file. This initial compilation generates an intermediate .aux file containing information about citations, cross-references, and glossary entries. Since we are using the package glossaries-extra, we also get a makeindex style file .ist and an .acn file.

2. BibTeX for References If we have references in our document, we need to run BibTeX on the .aux file. BibTeX reads our bibliography database (.bib) and generates a .bbl file, which contains the formatted reference information.

3. Glossaries with makeglossaries If we have a glossary in our document, we use makeglossaries on the .aux file. makeglossaries reads the glossary definitions and generates the necessary files to include glossary entries in our document.

4. L^aT_eX Compilation (2nd Pass) Now we need to run pdflatex on the .tex file again. This time, L^aT_eX incorporates the formatted references from the .bbl file into our document. If there are any citations, they will be correctly numbered and formatted in the reference list, but the citations themselves will be rendered as [?].

5. L^aT_eX Compilation (3rd Pass) Running pdflatex on the .tex file once more ensures that glossary entries are properly integrated into your document and that citations are properly numbered.

6. Final compilations We need to repeat the pdflatex compilation step as many times as needed to resolve all cross-references and ensure the document is correctly formatted. L^aT_eX may issue warnings or errors during this process that need to be addressed. In our case, for this document, we only need to run pdflatex three times in total.

7. PDF Generation Once all the necessary information is integrated into your document, a PDF file is generated as the output.

See  File extensions related to L^aT_eX, etc and  latex-auxfiles for more information on different file formats that you might encounter during the build process.

Method

The document

The example document we will use in the remainder of this post is based on the  IEEE conference template (Shell, 2002). We will make some changes, for example adding a reference file using BiBTeX and a glossary using the glossaries-extra package. We will also add three example figures. These figures are available both in .eps and .pdf format, so that we don’t need to run an expensive conversion process for each figure. We will include these figures without the file suffix, for example

\includegraphics[width=.9\linewidth]{example-image-a}.

The first page contains our title and a figure.

The second page contains some references.

The third page contains the reference list and an acronym.

The last and fourth page contains the list of acronyms.

There are many things that effect the compilation times of a L^aT_eX document. See  What affects compilation times, especially in longer documents?. The document we use here is short and doesn’t contain a lot of features, and we do this to keep compilation times low since we will be repeating this build many times.

You can view the  source code for this document on Overleaf.

Timing measurements

To be able to compare the different methods, we will use a short timing script. For each subdirectory it will run make clean test.pdf, 20 times each and measure the time with /usr/bin/time. The different build methods fall into different categories, see the following sections for a quick introduction to each category.

Experiments

If you’d like to follow along at home, please take a look at this Gitlab repository.

Running bare commands

Here’s a short summary of how to build a L^aT_eX document using the pdflatex, bibtex, and makeglossaries commands in the terminal

pdflatex test
bibtex test
makeglossaries test
pdflatex test

This is the classical method, used by hopefully very few people and is included here for completeness.

arara

Arara (Island of TeX, 2023) is a powerful and flexible build automation tool specifically designed for compiling L^aT_eX documents. Developed as a cross-platform solution, Arara simplifies the compilation process by allowing users to define compilation sequences through a user-friendly configuration file. With its intuitive YAML-based syntax, Arara lets you specify various compilation steps, such as running pdflatex, bibtex, and makeglossaries, in a defined order. This eliminates the need to remember and execute complex command sequences manually. By offering a clear and structured approach to compiling L^aT_eX documents, Arara enhances productivity and reduces the likelihood of errors, making it an indispensable tool for L^aT_eX enthusiasts and professionals alike.

latexmk

latexmk is a command-line tool designed to simplify the process of compiling L^aT_eX documents. It automates the compilation workflow by intelligently handling multiple runs of L^aT_eX and associated tools to ensure that all references, citations, cross-references, glossaries, and bibliographies are resolved correctly. latexmk is included in TeX Live. To customize the behavior of latexmk we can use Perl-scripts, and here we give a short example of a document specific configuration file that also includes makeglossaries.

add_cus_dep('glo', 'gls', 0, 'run_makeglossaries');
add_cus_dep('acn', 'acr', 0, 'run_makeglossaries');

sub run_makeglossaries {
  if ( $silent ) {
    system "makeglossaries -q '$_[0]'";
  }
  else {
    system "makeglossaries '$_[0]'";
  };
}

push @generated_exts, 'glo', 'gls', 'glg';
push @generated_exts, 'slo', 'slg', 'sls';
push @generated_exts, 'acn', 'acr', 'alg';
$clean_ext .= ' %R.ist %R.xdy';%    

To compile a document we can run latexmk -pdflatex -bibtex -r latexmkrc main.tex.

See  How does Overleaf compile my project? for a longer discussion on latexmk and customization.

rubber and rubber-info

rubber is a command-line tool designed to simplify the process of compiling L^aT_eX documents. Like latexmk, it automates the compilation workflow, aiming to provide a seamless and efficient way to handle L^aT_eX projects with various dependencies. We run this with the command rubber -d -m glossaries test.tex.

❯ pipenv run rubber-info test.tex
There was no error.
There is no undefined reference.
There is no warning.
There is no bad box.

scons

SCons is a Python-based Open Source build system that simplifies the process of building and managing complex projects.

# Make sure scons finds executables
import os
env = Environment(ENV=os.environ)

# Target and source files
pdf_output = env.PDF(target='test.pdf', source='test.tex')

# The Precious function is a method provided by SCons to mark a target
# as "precious" or "not to be deleted." This means that if SCons decides to
# delete temporary files or intermediate build artifacts after a build, it will
# not delete this particular target file even if it's considered an intermediate
# or temporary file.
# https://www.scons.org/doc/0.96.91/HTML/scons-user/c1924.html
env.Precious(pdf_output)

latexrun

latexrun is a Python-based build system designed to streamline the process of compiling L^aT_eX documents. It aims to provide a comprehensive solution for managing the compilation process and handling various dependencies.

 See L^aT_eX run. Run latexrun.. Here we use a fork of  latexrun that also supports makeglossaries.

Using a precompiled preamble

The first part of your document, the preamble, changes must less often than the body of the document. We can use this to our advantage by pre-compiling the preamble. We need to split our file into a preamble, and a body part and annotate each part properly.

The preamble should end with \endofdump.

% static part
...
\endofdump

We can compile this part with L^aT_eX in the following way latex -ini -jobname="preamble" "&latex" mylatexformat.ltx "preamble.tex". This will result in a file called preamble.fmt which we can reference in the body of the document with

%&preamble
\endofdump
\begin{document}
...

See Faster L^aT_eX part IV: Use a precompiled preamble and Glossaries and mylatexformat incompatible?.

Ramdisk

Mounting a directory as a RAM disk can significantly speed up tasks like L^aT_eX compilation by using RAM for storage instead of the hard drive. Here we create a small RAM disk that just fits our document. How to mount the ramdisk in practice depends on your operating system and the details are therefore left to the interesting reader to figure out.

Flags

In L^aT_eX, flags like batchmode and draftmode can help you optimize the compilation process. Using batchmode suppresses most output, making the build faster and less cluttered in the terminal. This is useful when you’re confident your document is mostly error-free and want a quicker compile. The draftmode flag, on the other hand, speeds up compilation by skipping the inclusion of images and performing only minimal typesetting. This is great for quick previews where fine details aren’t necessary, such as early steps in the compilation process. Both flags can be added when running pdflatex or other L^aT_eX compilers, like pdflatex -interaction=batchmode test.tex or pdflatex -draftmode test.tex.

Dvips

In the early days of L^aT_eX, the standard workflow involved compiling the L^aT_eX source code into a Device Independent (.dvi) file. This format was platform-agnostic but not easily shareable or viewable. To produce a more accessible Portable Document Format (.pdf), users had two primary routes:

• Convert .dvi to PostScript (.ps) using dvips, and then convert the .ps file to .pdf using ps2pdf.
• Use dvipdfm to directly convert the .dvi file to .pdf.

These methods were often cumbersome but necessary prior to the advent of pdfLaTeX, which allowed for direct compilation to .pdf files. We will include these older pipelines in our list of experiments to provide a more comprehensive view of the evolution of L^aT_eX.

Results

Build times

Our most important metric, and the one we set out to measure, is the build time. Faster build times improve the efficiency of the document creation process, allowing you to iterate more quickly through drafts. For those new to L^aT_eX, a slow build process can be discouraging. Faster build times can make the learning curve less steep. For me personally, long build times can interrupt my flow and concentration, impacting overall productivity.

Build times

In this figure we can see that the fastest build system is preamble, where we precompile the preamble using latex and compile the rest of the document with latex+dvips+ps2pdf. This speeds up the build a lot! In fact, in the first half of the list we see a lot of the same type of pipeline.

Manageable output

Human-understandable output, especially during errors, is crucial in L^aT_eX document building. Most of the information printed however, isn’t very useful, so it would make sense to try and reduce this and keep the output to a minimum. In this test we simply count the number of lines on stdout, but it should be said that some of these build systems, such as latexrun have really nice colored output in the case things do go wrong.

Number of lines printed to stdout.

The first part of the list is dominated by build tools such as latexrun, arara and rubber.

Ease of use

A build system must of course be easy to use, and as a proxy for that we measure the number of characters in the Makefile target. In the case that you do use a Makefile, simply running make would of course be easy enough.

Number of lines in the Maketarget

File size

A smaller file size offers advantages in terms of ease of handling, quicker uploads, and efficient storage. Therefore, the file size serves as a crucial parameter in evaluating performance.

PDF file size

It is interesting that there is such a big difference between the smallest and largest file size, even for such a simple document.

Final score

In the spirit of Eurovision and Melodifestivalen, we will adopt a point allocation system that mirrors the excitement and competition of these iconic music events. In Eurovision, each participating country awards points to their favorite songs, with the famous ‘douze points’ (12 points) reserved for the top choice. Similarly, the runner-up receives 10 points, acknowledging outstanding performances, and we will continue this tradition here.

Total scores received.

The build system with the greatest final score is latexrun. We have seen how we can improve the compilation time for a small toy project, but these improvements carry over to real documents as well. For example, compiling a recent IEEE paper using latexrun took 16.87 seconds compared to 25.71 seconds using latexmk.

Related work

There are many general build systems such as Snakemake (Koster & Rahmann, 2012), GNU Make, CMake, etc that can be used also to build L^aT_eX documents. There are also many L^aT_eX specific build systems such as make-latex, AutoL^aT_eX and ltx2any, that didn’t make it into this comparison for one reason or another and I hope to return to them at a later date.

Each of these tools can be use in a more general pipeline as explored in How to annoy your co-authors: a Gitlab CI pipeline for L^aT_eX .

If you have many TikZ (Feuersänger et al., 2014; Tantau, 2007) or pgfplots (Feuersänger, 2014) figures, it might be worthwhile to externalize them (Wenneker, 2012).

Discussion and limitations

In this blog post, we have strived to cover the most common build systems and tool combinations for L^aT_eX document compilation. These systems have been chosen based on their popularity and widespread use within the L^aT_eX community. However, it’s important to recognize that the world of L^aT_eX is vast, and new tools and approaches are continually emerging.

Therefore, one obvious limitation of this blog post is its inevitable inability to encompass every possible L^aT_eX build system and tool combination. The L^aT_eX ecosystem is dynamic, with developers and enthusiasts constantly devising innovative ways to streamline the document creation process. As a result, there may be niche or specialized tools that we have not explored here.

Some characteristics of a build system might be more important to you. For example, the output from latexrun in the case of a fault is much easier to read than for other build systems, and it leaves a build directory that is almost clean. However, build directory cleanness is not measured in this blog post.

Your choice of a L^aT_eX build system should of course be guided by your individual preferences and workflow requirements. For example, when selecting a build system, consider factors such as your operating system and L^aT_eX distribution. Some tools are more compatible with specific platforms, and this compatibility restricts what options are available to you.

Conclusion

In the realm of L^aT_eX document compilation, where efficiency and reliability are paramount, one contender stands out as the victor – latexrun when precompiling the preamble. Through our exploration of various build systems, including latexmk, arara, SCons, and latexrun, it becomes clear that latexrun in any of our variations stand out as a very good build system.

In the ever-evolving landscape of L^aT_eX document compilation, choosing the right build system is a critical decision. While each contender we explored has its strengths, latexrun emerges as the true champion, combining efficiency, reproducibility, and versatility in a single package. As you embark on your document creation journey, consider embracing latexrun as your reliable ally in crafting elegant and polished documents, all while reclaiming precious time and focus for what truly matters – your content.

References

1. Shell, M. (2002). How to use the IEEEtran LATEX class. Journal of LaTeX Class Files, 12.
2. Island of TeX. (2023). arara — The cool TeX automation tool. https://islandoftex.gitlab.io/arara/
3. Koster, J., & Rahmann, S. (2012). Snakemake — a scalable bioinformatics workflow engine. Bioinformatics, 28(19), 2520–2522. https://academic.oup.com/bioinformatics/article-lookup/doi/10.1093/bioinformatics/bts480
4. Feuersänger, C., Menke, H., & Tantau, T. (2014). TikZ & PGF manual. https://www.ctan.org/pkg/pgf
5. Tantau, T. (2007). The TikZ and pgf Packages. 1–405. https://www.ctan.org/pkg/pgf
6. Feuersänger, C. (2014). Manual for Plotting Package pgfplots. 1–500. https://ctan.org/pkg/pgfplots
7. Wenneker, F. (2012). Faster LaTeX part II: External TikZ library. https://web.archive.org/web/20160724213512/http://www.howtotex.com/tips-tricks/faster-latex-part-ii-external-tikz-library

Suggested citation

If you would like to cite this work, here is a suggested citation in BibTeX format.

@misc{isaksson_2023,
  author="Isaksson, Martin",
  title={{Martin's blog --- Optimizing Your LaTeX Workflow: A Guide to Choosing a Build System}},
  year=2023,
  url=https://blog.martisak.se/2023/10/01/compiling/,
  note = "[Online; accessed 2024-10-03]"
}

Share Tweet LinkedIn Reddit

Revisions