Alex Wybraniec

Technical Lead of Digital Products and Services

Before Camden

Free

In around 2015, I started using Cloudflare to improve the performance and security of a range of relatively high traffic sites and instantly saw benefits.

When you're a small business, quick is good, but free is better.

and we were able to experiment with Cloudflare because they offered their core functionality as part of a genuinely usable free tier.

SKIP: I think it is important not to under estimate the impact of this, and that it is increasingly scarce in the tooling and platforms we expect junior level and boot camp graduates to be able to use.

In most cases, Cloudflare "just worked", and while clients were grateful their projects performed better and we slept well knowing some security improvements were in place, there was some skepticism about what we'd actually done if it was so easy to implement this change.

Without the need to spin up new hardware, or configure complex request/response blocks in Perl like syntax, implementing Cloudflare seemed a bit...

Boring

Joining Camden

I joined Camden in 2022 which was a boom year for digital transformation logographics on vendor websites.

As part of the recently formed Digital and Products and Services team, our remit was to set standards and explore solutions that could augment or replace what we were being offered by traditional suppliers.

As part of a cross functional team, I believed I'd be building agile and iterative greenfield apps to create user centered solutions that combined our skills in experience, product and tech.

What I was actually doing was asking suppliers if they could do the very bear minimum we'd asked them to do, and vaguely adhere to the bullet point claims they'd put on their G-Cloud synopsis.

It was shocking to me how much technical interia there was whenever I tried to do anything, and from my niave small agency background the cost and timescales of even the simplest change blew my mind.

I was frustrated but also determined, and started to look at what we could do differently with the solutions we already had in place.

Cloudflare at Camden

Infrastructure	InfoSec	Engineering
DNS	WAF	Compute
Domain Registration	Rate Limiting	Storage
Access	Advanced Bot Management	Caching
SSL + ACM	Page Shield	Turnstile

Before I get started on my main topic, I thought I'd touch on Camden's Cloudflare usage as a whole.

We have been an enterprise customer for about three years, and we use a range of Cloudflare services across different teams.

While not directly in my remit, my existing experience with Cloudflare products meant that I became involved in the roll out of services people traditionally think of when you talk about Cloudflare, but

But was further involved in the run up to the 2024 General Election as we sought to harden our defenses across our core digital estate.

While the event itself went without a hitch, we learned a lot in the process.

A couple of interesting things we observed from a well configured WAF set up included

that for a period of about four months in 2024, Camden Council's biggest cyber attacker was... Camden Council... due to a misconfigured vulnerability scanner that had been left running by a contractor

and

that despite having no WordPress sites in our main estate, a large number of unmitigated requests were an attempt to access or exploit known WordPress issues.

While these requests could not have led to a compromise, mitigating them at the Edge via the Cloudflare WAF, before those requests reached our servers yielded a significant reduction in resource usage and improved response times across the estate.

Pages and Workers

(Static and Static-ish)

Project 1: Opt-in Beta

Frontend Library

Camden's Frontend Library builds on the work done by the GDS team, and utilises the GOV UK Frontend Library as its foundation

It extends and overrides existing components and adds some additional functionality to meet our needs.

Once bundled into the package we publish via the NPM registry, the Frontend Library consists of static assets such as CSS and JS files, and a suite of macros representing each component built in Mozilla's templating language Nunjucks.

When coupled with the guidance provided by our separate Design System documentation, the frontend library allows internal and external developers to quickly build experiences that are consistent, concise and accessible.

While people outside of Digital Products and Services understood the potential value of the work, some of the feedback we received from services that had been wooed by jazzier offerings was it was a bit...

Boring

Frontend + Content

But a frontend library is only part of the tooling needed to develop a project. Ultimately, you need something to combine it with content and data to create a usable output.

With increasing pressure to deliver something public by the end of the year, and after assessing a several different platform options, we faced an issue that (rightly) any new technology we wanted to use publicly would need to go through our InfoSec processes and at that point our processes for approving open source software was still maturing.

Additonally, we'd need to ask our constantly busy infrastructure team to procure and provision environments that would add long lead times and complexity to what was meant to be a high impact, but low effort project.

Finally, as this would be a public facing project on new technology, we'd have to arrange an Out of Hours support rota to handle any downtime, and myself and my small team weren't quite ready for pager duty.

What I wanted was something that was simple to develop with, had low complexity in terms of deployment, and in theory "couldn't go down at 3am on a Sunday morning (or if it did, it wouldn't be something I could resolve)

I needed something...

Boring

Not true

Also Not true

Static

Static site generators are nothing new but one of those concepts that everyone agrees is great, but is massively under used.

For those that don't know, but may have already guessed, a static site generator combines content and data with structural components and outputs the result as boring basic HTML files.

This shifts all the complexity of generating the files away from the public facing internet, and therefore truely static sites are generally super secure, performant and portable.

This works well in almost all content driven contexts, where the majority of pages change infrequently but persist for a long time.

A good example of this would be Cloudflare's own documentation site, which the internet tells me is built using the Hugo static site generator.

Hugo and indeed others like Jeckyl are great and have been around for years, but there's some complexity to setting up their rendering process and out of the box, neither supported Nunjucks natively.

After some further exploring, we found...

11ty

The thing that made 11ty appealing, aside from its weird possum logo, was that it was JS specific, almost instantly usable and could implement our Nunjuck components out of the box.

SKIP: One thing that is important to me when considering technology choices in a Local Government context is are we creating any lock in to the technology or the platform. In the short time I'd been at Camden I'd already encountered several situations where our projects were locked in to a specific vendors technology stack despite in some cases that platforms being open source.

It also has minimal moving parts and is agnostic about where you deploy your output. While this is to be expected for a static site generator, you still need somewhere for the generation to take place, ideally not locally on a developer's machine and then stuffed into the repo.

So we had a tool, but now we needed a pipeline and somewhere to deploy the output. We'd originally considered Azure Static Web Apps, which would have been fine, but we were having issues getting the right resource groups set up and while I was waiting for that to happen, I started to dig around for other things we had access to.

Pages

We were already a Cloudflare customer, and all I need to get started was for someone to give me the appropriate permissions within our account.

At this time, or shortly after, 11ty was a first class option in the Pages build process and integrating the CI/CD build was a simple as creating a new project in the admin UI and linking it to a GitHub repo.

From that point, every time a push event occured on a branch or PR, the build process automatically kicked off, and deployed the project either to the Production URL or a per PR preview URL.

These preview URLs where especially helpful when managing Jira tickets linked to specific PRs, and allowed us to QA and UAT additional functionality in isolation.

"It just worked" and we were able to build our beta site simply using our components and directory and file based paths within our repository.

Result

While the manual management of content was slow, we actually finished the project before we got access to Azure.

We met our deadline and our hard and soft requirements.

Essentially our new site had no moving parts and a nearly zero attack surface, which meant our InfoSec team were happy

Pages has seemingly a flat response time for all requests, and therefore infrastructure had no concern about performance or availability

And the output of the site was as close to our Frontend Library code as possible, which meant it was accessible and consistent.

People were happy, but when I got asked about the technology and I had to explain that it was

Boring

More real world

Project 2: Families

CMS

A cms or content management system as you probably know is a tool for creating and editing content for a product without needing to directly amend code.

Aside from the obvious ones like WordPress or Drupal (both widely successful projects written in the same unsexy language), there's a range of cooler managed service available such as Sanity or Contentful.

Interestingly there was an attempt by Cloudflare to develop their own edge based CMS project in 2021 but that repository was archived literally a couple of weeks ago.

The closest I could find to a Cloudflare specific solution was Microfeed, but at the time we were looking at solutions this was very much in its infancy.

We needed a more fully fledged and flexible option, and based on our experience, the Django based CMS Wagtail was a good all round fit.

However, this is where complexity started to creep in, as we needed more traditional server/container setup for hosting Wagtail, and Cloudflare did not offer any container support at the time (again, they've literally just announced some form of container support, but I haven't had time to read the docs yet).

We picked Heroku as the platform for deploying our Wagtail applications, and began considering how to integrate our 11ty build process.

In almost all cases, traditional CMSs combine code and content dynamically at request time in the same process as they return it to a visitor. This creates a potentially huge attack surface, and well crafted malicious requests are the most common vector for compromising websites.

In order to maintain our static desires, we needed to integrate our frontend build process in an headless manner.

Headless

A Headless CMS is one that returns only structured content and avoids most of the presentational mark up the makes up a web page

The gained some popularity in the 2010s, but again, they are another solution that everyone likes but don't implement enough.

Wagtail has good out of the box headless capabilities, and we choose the return the stored content as GraphQL when the 11ty build process requested it.

Essentially, when a change event is detected in Wagtail, it calls a Pages webhook which tells Pages to start to rebuild the project.

As part of the build process, the entire content of the site is requested in one massive GraphQL query, and 11ty breaks this down into specific pages and combines the content with our front end library components to create each page as a separate HTML file.

Rebuilding every page of a website whenever there is even the smallest change might seem inefficent, and it is, but we balanced this ineffiency with the complexity of writing a method for detecting the cascading impact of a single change and the numerous edge cases which would dramatically increase our code base.

In real world conditions, it takes about 4 minutes to build and deploy 200 pages of content plus additional assets like PDFs and images.

If you dig into these timings, a minute of that process is starting up the CI environment and an additional minute is waiting for our extremely low powered CMS instances to generate the GraphQL response.

However...

AX

My experience working with editorial teams taught me that Author Experience AX is as critical to a projects success as it's U and D siblings.

Despite our CMS instances being responsive enough for content designers to easily work with them, in a static context the practise of publishing your changes and then quickly checking what these changes actually look like doesn't work.

The delay between a change being made in the cms and it being visible to the public becomes frustrating.

Luckily, Wagtail has a great Live Preview function, which displays real time content changes within a rendered version of the page

This is great, except that to enable this functionality, we needed to run 11ty in a special "serverless" mode and this requires full NodeJS compatibility which was missing from Cloudflare's offering at the time.

We had to deploy this functionality to Azure's serverless Functions platform, which meant our "simple" architecture was now running on multiple different infrastructure and required semi complex deployment processes.

While we have this down to a fine art now, and most of it can be terraformed (meaning the set up is done via code not clicking around), this was an early lesson that using Cloudflare simplicity came with some limitations.

Result

But again, the result was good. From a technical point of view we got this project live quicker than we could have done with any other platform or supplier,

and we now have an almost fully fledged reusable and extendable platform

Since then we've launched three more production sites using this set up, and two more are being filled with content.

These projects have all the previously listed benefits of a static site, and in the last 18 months

we had 2 mins of downtime in total (that's 99.999746% uptime) with no engineering effort on our side

That might not sound super impressive, but because we utilise very little server resources for our CMSes, and Pages is essentially free up until some quite extremem usage (mainly around number of rebuilds per day)

we're deploying production ready, scalable and resilient websites for as little as £5 per month.

We'd planned to release this as a public repos using the GOV UK frontend library, but we're going to wait till we've reworked this to be more compliant with Cloudflare's new direction

But when we do re-work this, we'll be able to swap out the entire build process

with zero downtime and users on both sides will know nothing about it.

And we're not really planning to tell anyone about this change, because it's...

Boring

Not everything is static

The problem is not everything is static

Search for example is considered core functionality of most websites, but requires dynamic functionality to retrieve and render results

You could do this using a frontend framework like React to query an API endpoint and return the results, and that React application could still be considered static and be deployed to Pages.

But now you have to maintain a React frontend, deal with the accessibility overheads of dynamic interfaces and likely have a visible route back to your API which opens up your attack surface.

Server Side Rendering of dynamic functionality gives you a lot of benefits and allows you to hide complexity (or mess) from the public internet.

Obviously not suitable for all experiences, but 90% of what we're building day in day out in the public sector fits nicely.

This is where we get to the -ish of static-ish, and our first foray into... Cloudflare wwwwoo

Workers
Pages Functions

Our first attempt to add static-ish functionality to a project was using Pages Functions, and while we managed to develop a couple of working examples,

the functionality felt a little bit crowbar-ed in and there were a couple of weird hoops you had to jump through to achieve seamless experiences.

There was also some confusion as to whether Functions replaced Workers, or complememted it, and ultimately

it felt like some confusing middle ground.

Blog posts from this era talked about Functions being the platform for full stack applications, but almost as soon as we started to do more in it, it became clear that Workers was probably were the future lay.

Workers

From April of this year onwards, Workers became Cloudflare's suggested route for static, static-ish and full blown dynamic projects

Workers has been around since 2017 in a couple of different configurations, but has been evolving regularly into what we have today

What started as some convenient glue between Cloudflare products is now a full stack platform capable of hosting complex applications

It's easy to get started, and capable of complexity, especially when you start to "bind" other Cloudflare services such as a datastores, queues and even WAF functionality like rate limiting.

Pretty much all of this is available on the free tier, with lots of useful example applications in Cloudflare generally excellent and AI friendly documentation.

So all of this is great, but...

(and this might not be true anymore)

The workers JS runtime is only compatible with a subset of Node functionality which as you may remember is the reason we had to use Azure for our live previews.

It also means that you cannot use something like the popular Express framework with it. This starts to make Worker specific applications feel a bit locked in to the Cloudflare platform.

Luckily just as we were starting to get a bit wary, we found...

Hono

Hono is an "express-like" framework which works brilliantly within the cloudflare platform, but doesn't lock us into it because it complies with Web Standard APIs

You can deploy Hono applications to AWS Lambda, Vercel, Bun and Deno or even your own hardware using the Node adapter

It's got 25k stars on GitHub, growth is steady and hanging around different internet watering holes, you definitely hear people talking about it with some sense of refreshment

It's opinionated, but not obnoxious, feature rich but not overloaded, and like we saw with Pages originally, it just works out of the box.

Cloudflare themselves have started to use it in their stack, and now employ the creator Yusuke Wada as a developer advocate.

It's not the only Cloudflare tailored framework out there, and if I was less concerned about platform lock-in I think Redwood SDK is an interesting project

So what are we building with this well thought out framework deployed to serverless edge functions on a planet scale network?

Of course... it's something

Boring

Case Management

And it's true. Aside from content, which as you've seen, we've already fixed, transactions and case management are the core interface for most people with their local authority.

Enabling citizens (who are able) to self serve frees up contact centre teams to deal with the digital excluded or digitally intimidated. It also means you can interact with your local authority when and where is best for you.

The problem is almost every service within a local authority has its own case management platform, and most of these actually do look like they were last updated via FTP in the 90s.

A recent internal case study found that to pay a penalty charge notice in Camden required a user to cross between four different platforms each using different design language spanning the last 15 years.

Additionally, each of these different platforms requires their own planned or unplanned maintainance, and downtime is frequent and impactful.

It is not unreasonable to think a digital service should work outside of office hours, but increasingly this is a problem for our citizens.

So...

Project 3: Citizen Facing Transactions

Result

What next

Well, while there is no official end of life, it has been announced that Workers should be used instead of Pages going forward

And at the same time, 11ty is dropping some core functionality we rely on, so while our hand is being forced, we'll be looking to move all of our static site generation to Hono in the next year.

And then there's a lot of citizen facing experiences for us to work on

And we'd like to open source our work, and see if anyone else wants to use it

And maybe, once we've achieved all of this, we'll come back and tell you about our "not" boring work, like our RAG search engine that we built in a couple of hours using the AutoRag functionality Cloudflare released last week.

Why Cloudflare?

Why not Cloudflare?

Acknowledgements