Handling changes to cached static content

Caching static content

I run lighttpd as my HTTP server. It's setup to tell all clients to cache everything inside /static/ for a month, by using mod_expire:

$HTTP["url"] =~ "^/static/" {
    expire.url = ( "" => "access plus 1 months" )
}

I put all static content which rarely changes inside /static/, e.g. CSS and JavaScript files. A client requests such a file only once and caches it, until it expires. Next time, it's not fetched from the server, but simply loaded from the cache. This lowers traffic and CPU cycles, saving bandwidth and power consumption for both server and client.

Principally, this is a good idea and something everybody running a HTTP server should do, in some way.

The problem

The problem is that if something is changed, a client already having cached the file in question won't notice the change: As said, it won't request the file from the server but load it from it's cache. This could lead to a messed up layout or even an unfunctional page if relevant portions of e.g. the CSS style have been changed, in the worst case for a full month.

According to what I found, there's no way to directly work around it. If a browser did cache such a file with a defined time to live, it won't request it again during that time. There's no server-side way to tell a client it should reload such a file. No way to invalidate the cache or the expire date, no way to bypass this mechanism.

The client can possibly force-reload the page (e.g. by pressing CTRL+F5 or such, depending on the browser used), including all already cached files. But some people may not know this, and in some situations this might even not be possible at all: E.g. I'm using the Kiwi Browser on my phone, which does not seem to even have such a force-reload function.

The solution

If a file's name changes, it doesn't matter if the content is the same or almost the same. For the client, it's another file. So I simply introduced a revision number. E.g. instead of /static/css/style.css, it's now /static/css/style-1.css, after the next change, it would be /static/css/style-2.css and so on. Each client will request a file renamed like this after the change (of course again cache it) and display the changes correctly.

But all the HTML has to be changed then! Here, Jekyll, the lovely static web page generator, comes into play.

I simply created a YAML data file, which acts as a map for all the static content I want to handle like this. E.g. one could call it _data/static.yml, with entries like this:

shared-css: "/static/css/shared-1.css"
style-css : "/static/css/style-1.css"

And then, we simply don't reference such files directly anymore, but let Jekyll do the work: E.g. instead of

<link rel="stylesheet" href="/static/css/style.css">

one would write

<link rel="stylesheet" href="{{ site.data.static['style-css'] }}">

This works as well for CSS or JavaScript files, as soon as a "front matter" is added. E.g. one would reference such a file inside a CSS file like that:

---
layout: none
---

...
@import url("{{ site.data.static['shared-css'] }}");
...

The parsed result delivered to _site/ is simply the CSS file like before (as we did not apply any layout), but with the stuff referenced by {{ ... }} replaced with the real thing.

This way, if some "static" file changes, one only has to bump the revision counter and update the _data/static.yml file. Which are two additional steps, but this does not happen too often. And the rest happens automagically :-) Still, the static stuff is cached as it should be, but when changes are done, everybody gets the new version.