Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build daemon for katex #115

Open
SquidDev opened this issue Jul 28, 2022 · 1 comment
Open

Build daemon for katex #115

SquidDev opened this issue Jul 28, 2022 · 1 comment
Labels
web For issues/pull requests relating to the 1lab website itself.

Comments

@SquidDev
Copy link
Collaborator

I've been spending some time looking at what the slow bits are of a clean 1lab build:

Name Count Total Average Max
node 1,931 6m18s 0.20s 0.53s
pdflatex 258 4m16s 0.99s 1.80s
agda 1 4m04s 4m04s 4m04s
agda html 227 48.81s 0.22s 1.12s
git 383 12.58s 0.03s 0.13s
pdftocairo 258 9.54s 0.04s 0.22s

While there's some things which are always going to be slow (Agda), the obvious issue here is KaTeX. During a fresh build, we're spinning up 1.9k KaTeX processes, each of which runs for a fraction of a second!

It would be worth experimenting with creating some sort of build daemon here; just a basic JS program which reads an equation from stdin and writes the resulting SVG to stdout. Our Shake code can then maintain a pool of these, avoiding the cost of spawning a process for each job.

@SquidDev SquidDev added the web For issues/pull requests relating to the 1lab website itself. label Jul 28, 2022
@SquidDev
Copy link
Collaborator Author

One other thing @plt-amy suggested is kicking off the KaTex and diagram builds earlier. We can create a separate job which parses Markdown and compiles the equations/diagrams, and start that at the same time as the main Agda build.

It won't reduce single-core time, but means we can benefit from parallelism earlier on, meaning we don't end up with a build graph like this:

A graph of our Shake build. The X axis represents time and the Y axis represents number of parallel jobs. For the first four minutes of the build, only a single job (Agda) is running. After Agda is finished, all 16 cores are fully utilised for another minute, after which the build finishes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
web For issues/pull requests relating to the 1lab website itself.
Projects
None yet
Development

No branches or pull requests

1 participant