TPA-RFC-29: Use a plugin for compiling SASS files
RFC: https://gitlab.torproject.org/tpo/tpa/team/-/wikis/policy/tpa-rfc-29-lektor-scss-plugin
Lektor has a few plugins that will automatically compile SASS files as part of the build process lektor-scss, lektor-scsscompile. Using one of them would streamline the SASS development process and eliminate the need for the dart SASS compiler (the npm compiler doesn't compile the lego assets).
I think it would also be really useful to write a custom plugin similar to the two above that allows for jinja2 templating. I've run into a few issues where the SCSS files assume the static files are located at /static, which breaks under things like gitlab pages which place them at /project-name/static, and jinja templating should be able to fix it with the url filter.
Edit: Here's an informal proposal for the plugin
What?
This is a proposal to add the lektor-scss plugin to lego that automatically builds SASS/SCSS files as part of the lektor build process and dev server. The intended outcome will be a lower barrier of entry for contributors, and an easier and less complex build process for each site's SCSS.
How?
The plugin wraps the python libsass library. When the lektor project is built, the plugin calls libsass to compile the source directory to the output directory. Our current SCSS build process of sass lego/assets/scss:lego/assets/static/css does the same thing, just with the dart SASS compiler.
When the build server is running, lektor-scss creates a dependency list of SCSS source files, and on rebuilds checks the modification time on source files and only rebuilds when needed.
Why?
Sites using lego (usually) use lego's SCSS bundle. The source for this bundle is in lego/assets/scss, and the build bundles are in lego/assets/static/css. Sites use these by symlinking the bundle directory, and including the custom-built bootstrap.css. When a site wants to update, change, or add to its styles, the SCSS is changed and rebuilt with sass lego/assets/scss:lego/assets/static/css. Both of these directories are in lego, which means changing and rebuilding SCSS both require making an MR to lego.
This greatly increases the barrier to entry for contributing. A new contributor (hypothetically) wants to fix a tiny CSS bug on torproject.org. They have to figure out that the CSS is actually stored in lego, clone lego, make their changes, manually install the sass binary and rebuild, then commit to lego, then update lego and commit in the tpo repo. With this plugin, the process becomes "clone the tpo repo, make changes to SCSS, and commit"
The plugin also gives us the opportunity to rethink how we use SCSS and lego. If SCSS is built automatically with no dependencies, we won't need to symlink the entire SCSS directory; that lets sites have additional SCSS that doesn't need to be added to lego and doesn't pollute the main bundle used by all the other sites. We also wouldn't need track the built CSS bundles in git; that stops the repo from inflating too much, and reduces noise in commits and merge requests.
How does this affect lego and existing sites?
None of the sites will be affected by this plugin being merged. Each site would have to enable the plugin with a build flag (-f scss). Once enabled, the plugin will only update SCSS as needed, using no extra build time unless an SCSS file has changed (which would need to be re-compiled manually anyway).
I ran a few benchmarks; one with the plugin enabled and set to "compact" output, one with the plugin enabled and set to "compressed" output, and one with the plugin installed but disabled. Compressed and disabled were within a second of each other. Compact took an additional 20 seconds, though I'm not sure why.
All of these benchmarks were run in a fresh clone of the tpo repo, with both the repo and lektor build directory in tmpfs. All benchmarks were built twice to deal with translations.
lektor clean --yes
rm -rf public
find . -type f -iname 'contents+*.lr' -delete
time bash -c 'lektor b -O public &> /dev/null && lektor b -O public &> /dev/null'benchmark results:
enabled, compact: real 6m53.257s user 6m18.245s sys 0m31.810s
enabled, compressed: real 6m31.341s user 6m0.905s sys 0m29.421s
disabled: real 6m32.028s user 6m0.510s sys 0m29.469s
A second run of just compact gave similar results as the others, so I think the first run was a fluke: real 6m30.299s user 6m0.094s sys 0m29.328s
What's next?
After this plugin is merged, sites that use lego can take advantage of it by creating a config/scss.ini, and adding the -f scss flag to lektor b or lektor s. Sites can incorporate it into CI by adding scss to the LEKTOR_BUILD_FLAGS CI variable.
# scss/config.ini
output_dir=assets/static/css
output_style=compact