Flores, plugin architecture and dependency hell
written on February 19, 2023
A few months ago I started working on a static site generator called Flores, with the main use case in mind being my
own site. I wanted to try to describe why I felt like making yet another static site
generator (I promise it's not because I thought I could do it better
!) and talk about some of the common
problems I have with frameworks in general.
Why build a new generator anyway?
Before Flores, I was using Jekyll to generate my site. I actually think Jekyll gets a lot of things right and that's probably why it's popular; Flores is also very much inspired by Jekyll. There were, however, two big problems with Jekyll for me: its scope and plugin-based architecture, and the resulting dependency hell.
To me, the whole purpose of a static site generator (SSG) is simplicity: you use one because
you don't want to handcraft every single page, inevitably repeating yourself, and also because
you want to be able to deploy your site with minimal effort, ideally on any platform. As such,
an SSG's job should be to take care of all of the common sense
things that the vast
majority of the users will want, and of course leave space for extra functionality for the
power users
that want to do more complex stuff.
Where frameworks like Jekyll fail—in my humble opinion—is that they actually do
not take care of all of the basic, common sense
stuff, and they instead choose to
delegate pretty much everything to plugins (or otherwise extra config settings). Take code
highlighting for example: when I was using Jekyll for my personal site, you needed to add a
dependency on Rouge, specify it as the
highlighter (although now I think it's the default?) and specify a Markdown flavor. I get the
motivation behind this: you enable power users to potentially use their obscure, little-known
highlighter and Markdown parser. And at the end of the day, sure, why not? Computers are a
social construct anyway.
Well, I think this goes against the whole SSGs strive for simplicity thing. When
designing an interface, engineers tend to want to expose as many buttons and switches as
possible (probably because we ourselves would like more access under the hood
), without
necessarily questioning this choice. But if we think about it... is I wanna use my own
Markdown flavor/code highlighter
a legitimate
use case, to the point where the
framework has to bend over backwards to support it? I disagree with that. You throw a rock out
here, you hit a different shade of bike
shed paint. You can always argue that one flavor of Markdown is better than another, or
that one highlighter plugin is better than another, but at the end of the day—again, in
my humble opinion—any of them will do just fine, so long as they're decent.
To give a concrete example: Flores also depends on a code highlighter, Pygments. However, it does not allow the user to specify a different one, let alone roll their own one. This comes with two advantages:
- You don't have to worry about maintaining compatible dependencies. It's Flores' job to guarantee that it will work on the platforms it claims to support, and it's simpler for Flores maintainers (yours truly) to have to deal with a single code highlighter, both in terms of how it interacts with the rest of the SSG and in terms of compatibility with different platforms.
- You don't have to care. Why specify something in a config file? Why not have code highlighting work out of the box with literally no input from the user as far as the config is concerned? If you're using a SSG, that means you're tech savvy enough to write some HTML, CSS and maybe even some JavaScript, so it's likely that you'll have code snippets in your pages/blogposts.
To expand on (2), that's kind of what I meant before when I was talking about common
sense
features; an SSG will most likely power a site that has snippets, that has blogposts
and so on. Another feature that I didn't find in Jekyll (and that I wrote down immediately when
listing the things that I wanted Flores to support out of the box) is image
optimization: it is extremely common to want to optimize image size, whether that
concerns compressing more or producing multiple different sizes out of an original image to
serve to devices with different screen sizes. With Jekyll I had to write my own bash script
that did that; with Flores I can just specify it in a few lines of config.
Same thing goes for Sass/SCSS support: why not have it out of the box in 2023? Flores supports
it, and if you think it's horrible and don't want to use anything but pure CSS, well... you
just can. You simply use .css
files, and
that's it, Flores doesn't process them in any way whatsoever. The only thing that's not
supported out of the box is some sort of JavaScript transpiler, something like TypeScript or
Babel, but that's because it's much heavier
as a dependency and because static sites
should probably have little or no JavaScript anyway (again, aren't they supposed to be
simple?). I'm still thinking about supporting it though (optionally, just like
Sass/SCSS).
The problem(s) with plugin architecture
More generally (i.e., not just regarding SSGs), I dislike the whole plugin architecture
model, where basically everything's a plugin and the core
of the framework only supports
the absolute bare minimum.
If the only downside of plugins was that you might get overwhelmed by the choices, then maybe that would have been fine; the problem is that these choices do not come for free. Jekyll, for instance, has you managing the Gems (that's Ruby for libraries, I believe) that you use via plugins, along with the dependency hell that comes with that: you might have version conflicts, or some Gems might not work on some platforms etc. Again, I understand the design choice here: you push that responsibility onto the user, and if they construct a site with a gazillion dependencies, too bad, that's on them. I disagree with it because I believe that in most cases the user just wants to have a site, not deal with the internals of the SSG or the nuanced conflicts of different Gems.
Flores is not completely free of this either, by the way: for the moment, it's using
Python-Markdown (or Markdown? folks please decide on a name), which,
uh... leaves a lot to be desired, to say the least. It's also plagued by plugin architecture,
but its documentation (unlike Jekyll's) is sparse and confusing (looking at you, CodeHilite) and way too
many things are considered optional
(like tables, seriously?). At least that's not
exposed to the user, but Flores has to deal with this mess internally.
Even if you still think that plugin architecture is a good idea, you first have to do it right:
the core must still support all of the basic, common sense
features, and leave the
truly optional stuff to plugins. If we again take Python-Markdown as an example, given
that it is a developer-oriented framework, then sure, you can argue that abbreviations are not an
essential feature, but I really don't think you can say the same for tables or code blocks or
code highlighting.
Sure, maybe many of these problems come from Markdown itself and how it's defined (or rather,
how it is not), but I don't agree
that a Markdown parser should adopt the not my problem
attitude here; at the end of the
day, when we get a chance to fix an issue, we should, right?
The last problem I have with plugin architecture is that you're counting a lot on
adoption. We can say that libraries of programming languages are plugins, sort of; the standard
lifecycle of a programming language dictates that, if a library is used extensively by the
majority of the community and/or becomes basically an essential part of the language, then the
language devs usually merge that library into the standard library or the builtins of the
language (mainly to allow devs to use it without having to depend on an external
library
for something essential). This is why I said that plugin architecture depends on
adoption: you're counting on people developing that sort of stuff for you so
that you can even have that functionality available to the users of the framework. But what if
your framework is not very popular, or what if you don't have the time to develop a plugin
that's actually an essential feature yourself? This is more or less the current status with
code highlighting for Python-Markdown: the main
plugin is CodeHilite—which, again,
leaves a lot to be desired—and as far
as I know the only alternative is Highlight,
which has its own set of problems. So basically my choice as Flores' developer is to either
roll my own highlighter, which defeats the whole purpose of delegating this task to the
Markdown parser, or to just live with the problems of the (limited) existing choices. I truly
think that CodeHilite would be better if it was the only choice and part of the core
of Python-Markdown, because then you have to make something solid and deal with bug reports and
feature requests, as there is no alternative. This pushes the burden on the developer, which is
where it should be: the framework is the developer's job, not the user's!
Is Flores actually better?
Of course, I'm way too biased to answer this question in a conclusive way. I can, however,
compare building my own site with Jekyll and with Flores. To be clear, I'm basing the
comparison on the previous version of the site, simply because a lot of simplifications were
done during the latest redesign that have nothing to do with the underlying SSG. By using
Flores' built-in image optimization feature alone, we save almost 70 MiB worth of data on the
source code
of the site (meaning the contents of the Git repo, not the actual site
that's served). In terms of the actual site size, we save ~35 MiB, although some of that is due
to missing page issues that Flores
has at the time of writing; a big part of it is nonetheless due to the fact that, because of
the janky homebrew workaround I used to have in order to optimize images, Jekyll will also dump
the original images on the site, while Flores will only dump the produced
images (i.e.,
optimized images in the specified sizes). Of course that can be fixed in Jekyll, but in Flores
you simply don't have to worry about it.
It's also just much neater. With Jekyll:
$ shell
$ tree . -L 1
.
├── 404.md
├── about-me.md
├── archive.md
├── blog.md
├── compile_cvs.sh
├── compile_images.sh
├── _config.yml
├── css
├── cv_pdfs
├── _data
├── _drafts
├── favicon.ico
├── Gemfile
├── Gemfile.lock
├── images
├── _includes
├── index.md
├── js
├── _layouts
├── _posts
├── _projects
├── README.md
├── _sass
├── _site
├── sketches.md
└── thanks.md
12 directories, 14 files
And with Flores:
$ shell
$ tree . -L 1
.
├── _assets
├── compile_cvs.sh
├── _css
├── cv_pdfs
├── _data
├── _drafts
├── _js
├── _pages
├── _posts
├── _projects
├── README.md
├── _site
└── _templates
11 directories, 2 files
Again, you can argue that this is purely aesthetic
or irrelevant to the actual efficacy
of the SSG or whatever, but I think it really counts. A more organized repo means that you can
spot where things are much more easily; I truly do not understand why Jekyll went with the
just dump all the
option.
.md
page files at the root of the
repo
The last thing I'd like to note here is that Flores' architecture isn't exactly the
opposite
of plugin architecture: you can still totally add extra steps or mechanisms to
the site build. For example, the way I currently deal with my CV is that I have an extra step
during the deployment of the site, where my CV repo is cloned, the CV PDF is built from source
with TeXLive and finally it gets copied into the _assets
directory, allowing Flores to pick it up and treat it
just like any other asset. Similarly, you could choose to call a script to transpile TypeScript
sources to JavaScript, and let Flores handle the resulting JavaScript files as normal
JavaScript files. I think this way of dealing with extra features
allows power users to
do whatever they want and at the same time does not add extra cognitive load to the
average users who just want the bare minimum. And besides all that: I don't claim to have
designed a perfect SSG, far from it! If anyone comes around with a feature request that is
legitimate and falls under the common sense
features, I'm more than happy to add it to
Flores as a core feature, given that it doesn't unnecessarily bloat the SSG or introduce scope
creep.
Conclusion
To be clear, I don't mean for this post to be about bashing Jekyll/Python-Markdown
or
anything of the sort; I'm only using them as examples to show why I personally disagree with
how (and how often) this plugin architecture
is used. It ends up pushing almost every
responsibility onto the user and having them deal with relatively low level
stuff,
digging into the framework or even having to deal with dependency conflicts, when all they
wanted to do was to build their damn static site. I think there's a time and a place to expose
all the bells and whistles of the underlying interface, but we do have to remind ourselves: not
everyone cares about every nook and cranny of a framework we've built, and in fact they most
likely never want to look under the hood
if they don't have to.
More generally, and for QA's sake, software should strive to have as few dependencies as possible. I'm not saying that you should never have a plugin-based architecture, only that you don't always need it, and that we should think twice before implementing a design choice that adds a lot of extra load QA-wise without having a very good argument supporting its use.