My year of Eleventy plugins

One year ago, I published my first Eleventy plugin (and my first npm package of any kind): eleventy-plugin-youtube-embed. I’d recently rebuilt my personal site using the Eleventy static site generator, and wanted a simple way to add embedded YouTube videos in Markdown. I suspected others might want to do the same.

Quite a few did! Today I know that while it’s possible to make this task easy, it’s far from simple. And that first plugin has since grown into a (tiny) plugin ecosystem of its own, called Embed Everything, which now covers not just YouTube but Twitch, Vimeo, TikTok, and more.

I thought I’d mark this milestone by sharing some of the things I’ve learned over the last year.

The numbers

First, some raw statistics. These figures cover the whole family of plugins.

  • 25,000+: Total number of package downloads since launch.
  • 428: Public repos with one or more of my packages as a dependency.
  • 395: Total commits across all repos.
  • 113: Package versions released.
  • 2: Outside code contributors. (Thanks Leslie and Scott! 🎉)
  • 1: Feature request from Mr. Eleventy himself.

Overall, it’s been really rewarding to see people use something I’ve made. It’s also been hard! In fact, the rewarding parts and the hard parts have mostly been the same parts.

Testing: it’s hard

I had never once written a test before starting this project.

I always knew that I should know more about testing, but (as with so many things) it never clicked until I truly needed it to solve a problem. Which was: I needed to prove, programmatically, that these plugins do what they claim to do, and do so in predictable and reliable ways.

My initial tiptoe into writing tests was simple fire-fighting: I got a couple of GitHub issues filed by early adopters who found the YouTube plugin wasn’t working for them. It quickly became clear that my early haphazard efforts at spotting edge cases was not going to cut it, and I needed a better system. That finally spurred me to make a concerted effort to dig into testing.

I’ve found that learning to write tests (and, to be clear, I consider that learning process very much ongoing) has had the following benefits:

  • It improves overall project structure
    Re-writing these codebases for testing has made them more modular and functional, and therefore easier to read and reason about. Breaking functions down into their smallest constituent parts, so that they’re accessible to tests in the first place, makes them more legible and composable — both for me and (I hope) for other contributors.

  • It gives you a framework for thinking more deeply
    I’m self-taught; getting code to work at all is often a triumph for me. Going back to figure out all the ways things could go wrong always just felt like a drag. But I also never had a framework for thinking about errors. Now, I can sit down and map out expected inputs and outputs, consider possible exceptions, and think logically about how to handle them. It’s not that I couldn’t imagine stuff breaking before; it’s that I didn’t know how to articulate my intuitions. It’s like I could hum the tune but didn’t know musical notation. Now, the known unknowns are much easier to spot and write down.

  • It helps you move faster
    Simple trial and error has two drawbacks: First, it’s unreliable because it’s often un-repeatable, with techniques and approaches varying from person to person and machine to machine. Secondly, and as a result, it just takes a long time. Having tests that can run automatically, regularly, and consistently makes it possible to make changes and updates with greater confidence, and that is so much faster and more productive. There are definitely updates and features in the plugins today that I would not have had the confidence to send out in the wild if not for the enforced discipline of a continuous integration setup.

For lots of people, all these points will be old news, and it’s clear I have more to learn. But adopting even the basics has been enlightening. And if you’re in the same boat I was with testing, I highly recommend diving in.

Versioning: it’s hard

I’m making a good-faith effort to stick to the principles of semantic versioning. But it does feel like semver can be overzealous in a way that’s not necessarily helpful to the majority of end users.

Take any of the Embed Everything plugins: their “public APIs” have been quite stable, mostly because what they do is pretty basic: URL in, embed markup out.

If you’re consuming these plugins as packages from npm, in fact, there are many version releases that would have no visible change whatsoever. In general, we .npmignore test files, the code of conduct, CI config files, and so on, because that stuff really only matters to people contributing to the codebase, not consumers of the package. But those files are also where most of the actual commit action is. So I frequently end up releasing new minor or patch versions that represent no actual change for the vast majority of users.

To me, that doesn’t feel like it’s quite living up to the intended benefit of semantic versioning, which is conveying meaning. Meaning to whom? Open source software has multiple user bases, and a patch release that’s meaningful to a contributor might be meaningless to an end user drowning in Dependabot notifications.

I don’t know how to fix that, and obviously it’s a complex problem. But it’s been on my mind (since I’m kind of drowning in Dependabot notifications).

Naming things: it’s hard

In retrospect, I really wish I’d picked the right naming scheme for the first two plugins. Both the YouTube and Vimeo plugins follow the format eleventy-plugin-{service}-embed. With the later plugins, I changed it to eleventy-plugin-embed-{service}, so that they’d cluster together alphabetically in web searches, in the dependency list in package.json, and so on.

At the time I published the YouTube plugin, I wasn’t thinking about extending the model to other services, so the package name structure didn’t seem that important. The mismatched name pattern has basically zero practical effect, but it just...bugs me. At this point, I think changing it isn’t worth the trouble.

Self-doubt: it’s hard

I’m glad these plugins exist! I’m happy to have contributed them to the Eleventy community, and I intend to keep going. But I definitely also harbour doubts. They include:

  • Implementation
    I’m occasionally haunted by the idea that the fundamental nature of these plugins is...wrong. They work by running a regular expression on Eleventy’s HTML output. Parsing HTML with regex is somewhat famously (though not universally) regarded as a bad idea. I guess the only response I have here is that, given my skills and knowledge today, I’ve made the best thing I’m currently capable of. And yet, I wonder if there’s some dramatically better way — and whether I’d recognize it if I saw it.

  • Performance and scale
    Especially with embed-everything, which aggregates all the individual plugins, I worry that regexing every single HTML file multiple times is not sustainable, especially as Eleventy matures, people’s sites get bigger, and the number of files grows.

  • Sustainability
    Everyone knows open source has a burnout problem. That’s not a problem for me at the moment, but I am aware that keeping these packages up to date requires periodic maintenance and I want to strike a workable cadence for the long term. I’m not really sure what that looks like yet.

Roadmap to v2.0

I do have thoughts about the next major release(s). The top-line feature I’m planning is inline options, so you can change the settings of individual embeds with simple keywords on the same line as the URL in your Markdown.

Here’s what I mean:

<!-- Embed this video with a 4:3 aspect ratio: -->
https://www.youtube.com/watch?v=mT0RNrTDHkI 4:3

<!-- Autoplay and loop this video: -->
https://www.youtube.com/watch?v=mT0RNrTDHkI autoplay loop

The idea is to support features that you likely need to control on a per-video basis, like autoplay or aspect ratio. Some videos require slight tweaks and currently there’s no way to break the mold on a case-by-case basis. I think inline options are the way to handle that.

This represents a breaking change to the existing regular expression structure, which is currently very strict about skipping paragraphs that contain anything other than a URL. But I think the additional flexibility it unlocks is worth it.

My intent is to cut a new major version of all the plugins at the same time, even if the inline options available at 2.0 are limited or non-existent. But I think breaking the RegEx pattern should be done only once across the board, even if the new features come later.

Other things on the radar:

  • Drop support for Node.js 8. Definitely a 2.0 breaking change.
  • Support the new aspect-ratio CSS property launched in Chrome v88.
  • Standardize all the plugins on an embedMethod option (name could change!) to make it easier to better support multiple types of HTML output. For example: iframe, script, lite, oembed, and so on.

The latter two ideas here could be non-breaking 1.x releases too, I haven’t totally settled on how to do it. And all of this is up for discussion — if you have ideas or want to help out, I’m all ears!

Tagged: Eleventy