nicholas.cloud

Pets have names, livestock is tagged

Nicholas Whittaker — Sat, 12 Jul 2025 16:48:01 +1000

There’s an oft-quoted phrase in the cloud/DevOps space.

Treat infrastructure as cattle, not pets.

To me, the advice is broad and emblematic for a lot of modern practices. Prefer disposable containers over long-lived hosts. Build infrastructure that scales horizontally to accomodate demand. Design architectures where components can be swapped out on the fly.

I thought I’d recap some recent changes to how I provision and manage this blog - addressing a few places where hostnames were hardcoded for convenience.

This blog runs on a DigitalOcean droplet managed with Ansible. The inventory of hosts for Ansible to manage is generated from querying my Tailscale tailnet.

Previously, that inventory was a 1:1 mapping of machine names to hostnames. Hosts in an inventory can be grouped as well though. What if each Tailscale machine is tagged with the projects it hosts?

Mapping each of these tags to a group in the Ansible needs only a small bit of wrangling with the inventory script.

{
 "project_buildkite_uploader": {
 "hosts": ["gyro.tailbf155.ts.net."]
 },
 "project_opentelemetry_collector": {
 "hosts": ["gyro.tailbf155.ts.net."]
 },
 "project_vault": {
 "hosts": ["gyro.tailbf155.ts.net."]
 },
 "project_blog": {
 "hosts": ["blog-bfab9eeb.tailbf155.ts.net."]
 },
 "project_writefreely": {
 "hosts": ["boyd.tailbf155.ts.net."]
 },
 "_meta": {}
}

Now I can swap out the hardcoded blog hostname, instead matching against all hosts in the project_blog group.

 - name: Deploy Blog (https://nicholas.cloud/)
- hosts: blog
+ hosts: project_blog

There’s a few more benefits in addressing this reliance on static hostnames:

It’s easier to support multiple hosts for a project
Adding/removing hosts is done by tagging, rather than code changes

There’s a similar change to the build pipeline for my blog’s content, which now gets deployed to all tagged targets.

All in all, a nice little win for idle tinkering.

Addendum to my blogroll escapade

Nicholas Whittaker — Sun, 18 May 2025 20:24:04 +1000

Following on from setting up my blogroll last week, I’ve realised it doesn’t render as a pretty webpage in most web browsers. Serves me right for only testing in Firefox!

To style the document, I wrote the XLST in an xsl:stylesheet element with an id="blogroll" attribute. A processing instruction applies the stylesheet to the document.

<?xml-stylesheet type="text/xsl" href="#blogroll"?>

XML processors will look for the stylesheet by its ID, blogroll in this case. But when looking at an element, how do know if one of its attributes is an ID? The XML spec goes into detail about this.

In order for such an attribute value to be used as a fragment identifier in a URI, the XDM attribute node must generally have the is-id property […]. This property will typically be set if the attribute is defined in a DTD as being of type ID, or if is defined in a schema as being of type xs:ID.

Turns out, most processors need the ID attribute to be explicitly labelled. So I added the necessary document type declaration (DTD) explicitly specifying the ID attribute for xsl:stylesheet elements - in this case called id.

<!DOCTYPE OPML [<!ATTLIST xsl:stylesheet id ID #REQUIRED>]>

In Firefox’s case, the browser seems to assume id will be the ID attribute. As other browsers show though, that isn’t a safe assumption.

Anyway with a fix now in place, I hope you enjoy perusing my blogroll!

Committing XML horrors to style my blogroll

Nicholas Whittaker — Sat, 10 May 2025 20:25:13 +1000

For a while, I’ve provided a list of suggested blog/feeds on my website in a blogroll.

This blogroll takes the form of an OPML file that RSS readers can import. It’s pretty neat, but making it browser-friendly has always been on my wishlist.

So it was pretty cool to stumble across XSLT - a means for transforming XML documents.

In this case, XSLT offers a means to transform the OPML blogroll into HTML that a browser can render and style. I got the inspiration from Ruben Schade and his guide on styling OPML with XSLT, which he used on his own blogroll.

Applying this to my blogroll gave me the best of both worlds:

The blogroll resembles my website’s regular HTML pages in a web browser
The file itself is still valid OPML which RSS readers can import

As an additional challenge, I wanted to continue generating the blogroll dynamically in Hugo rather than hand-curating the XML. I’ve been doing this with a custom output format, listing feeds in the page’s metadata.

---
title: "Blogroll"
outputs:
 - opml
params:
 feeds:
 - name: Lachlan Jacob
 xml: https://blog.etopiei.com/feed.php
 html: https://blog.etopiei.com/

Generating these OPML/XSLT documents dynamically came with another benefit, as I could avoid duplicating the base HTML template used for the rest of my website. There were a few roadbumps, but they were pretty negligible:

XML is stricter with self-closing elements, like <hr>
Common HTML entities like & need to be defined explicitly or removed

Aside from that, the main challenge was bundling the XSLT into the existing document. Linking it from a static file is an option, but I wanted to include templating logic in the XSLT as well. Fortunately, XLST can be embedded and referenced from within an existing OPML document.

<?xml version="1.0" encoding="utf-8"?>
<?xml-stylesheet type="text/xsl" href="#blogroll"?>

<opml version="2.0">

<head>
 <!-- OPML head -->
</head>
<body>
 <!-- OPML body -->
</body>

<xsl:stylesheet id="blogroll" version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
 <!-- XSLT transformation -->
</xsl:stylesheet>

</opml>

With that addition my blogroll now renders in-browser, and looks like most pages of my website!

As a last step, I did also need to add an Nginx directive for browsers to treat the OPML file as a XML document.

# Serve OPML files as XML to force in-browser viewing
location /blogroll.opml {
 default_type application/xml;
}

So if you’re interested in reading or following other interesting feeds now, have a read of my blogroll!

Using Buildkite OIDC with Hashicorp Vault

Nicholas Whittaker — Tue, 19 Sep 2023 17:54:23 +1000

Earlier this year, Buildkite announced support for OpenID Connect tokens. Briefly, a Buildkite agent can request a signed JWT (JSON Web Token) from Buildkite representing details (claims) about its current job. This JWT can then be used to authenticate with systems that accept it.

For Hashicorp Vault, services typically authenticate using the AppRole method with a senstive set of credentials. It’s fine to use this flow on a Buildkite agent to access Vault secrets, but the credentials for this are long-lived.

The new OIDC flow removes to need to manage these long-lived credentials, and also makes it possible to craft fine-grained policies for a Buildkite agent without requiring multiple sets of login credentials!

I currently run my general-purpose CI workloads on a single autoscaling cluster of Buildkite agents. This lean approach is great for me since I don’t have the need to scale or segment my CI setup like a larger organisation might.

There are shortcomings though to all my pipelines sharing these agents. Each pipeline may have its own secrets, but agents must have access to all of them because they could run jobs from any pipeline. Even if secrets follow least privilege practices, an agent’s overly broad access grows with each new secret.

As an example, here’s a policy that allows secrets in a key-value store matching the path buildkite/* to be read.

resource "vault_policy" "buildkite_agent" {
 name = "buildkite-agent"
 policy = <<-POLICY
 path "kv/data/buildkite/*" {
 capabilities = ["read"]
 }
 POLICY
}

An authenticated agent running a job for the chocolate pipeline can read the buildkite/chocolate secret, but there’s nothing stopping it from also reading the buildkite/strawberry secret.

Without dedicated agents for each pipeline, jobs have unchecked access to all buildkite/* secrets. Setting up OIDC authentication for my agents instead provides an alternative method to enforce stricter policies and limit access.

First off, we need to create the Vault backend that will accept the JWTs obtained from Buildkite.

resource "vault_jwt_auth_backend" "buildkite" {
 path = "buildkite"
 oidc_discovery_url = "https://agent.buildkite.com"
}

A new role for this backend dictates the requirements to log in with a JWT. Setting the bound_audiences and bound_claims is important to ensure the JWTs are intended for my Vault instance and that they belong to my Buildkite organisation.

resource "vault_jwt_auth_backend_role" "buildkite_agent" {
 backend = vault_jwt_auth_backend.buildkite.path
 role_name = "buildkite-agent"
 token_policies = ["default", vault_policy.buildkite_agent.name]
 role_type = "jwt"
 user_claim = "sub"

 bound_audiences = ["vault.nicholas.cloud"]
 bound_claims = {
 organization_slug = "nchlswhttkr"
 }

 claim_mappings = {
 pipeline_slug = "pipeline_slug"
 }
}

The claim_mappings block above specifies metadata to be copied from the JWT’s claims, in this case the pipeline_slug denoting the pipeline the job belongs to.

The metadata can be referenced in a role’s policies using template syntax, which we can use to limit the buildkite-agent policy from before.

 resource "vault_policy" "buildkite_agent" {
 name = "buildkite-agent"
 policy = <<-POLICY
- path "kv/data/buildkite/*" {
+ path "kv/data/buildkite/{{identity.entity.aliases.${vault_jwt_auth_backend.buildkite.accessor}.metadata.pipeline_slug}}" {
 capabilities = ["read"]
 }
 POLICY
 }

The template itself is a little cumbersome, but the resulting policy works like a charm. For a job to read the buildkite/chocolate secret now, it must originate from the chocolate pipeline.

path "kv/data/buildkite/{{identity.entity.aliases.auth_jwt_e9c0606b.metadata.pipeline_slug}}" {
 capabilities = ["read"]
}

Logging into Vault now requires only the short-lived token from Buildkite!

vault write auth/buildkite/login role=buildkite-agent \
 jwt="$(buildkite-agent oidc request-token --audience vault.nicholas.cloud)"

Signing Terraform provider releases with a local Buildkite agent

Nicholas Whittaker — Mon, 17 Jul 2023 10:54:19 +1000

For a while now, I’ve built and published my own Terraform provider for retrieving secrets from a pass store. One of the requirements to publish a Terraform provider is that every release must be signed with a GPG key.

I have a Buildkite pipeline to build and publish these releases to GitHub. A step in this pipeline has access to a private key for signing, but it’s a different key from the one I use on my own machine. I consider the latter too sensitive to expose freely to my Buildkite agents.

With that said, managing a second key just to publish my Terraform provider is quite irksome when it has no other use for me. However, it’s unfortunately necessary if I don’t want to expose my regular key to my CI environment.

But if the worry is around exposing a secret to Buildkite agents running outside my machine, why not introduce an agent that runs specifically on my machine?

Buildkite agents are commonly run on easy-to-reproduce hosts, such as virtual machines or containers in a cluster. They can be run on any host with a supported OS/architecture though, which includes my daily driver MacBook. Running an agent locally is doable, but there are also good reasons why it usually isn’t suitable.

Developer environments tend be volatile, with tools and dependencies that might be missing or on an incompatible version
They tend to have poor availability, only being online so long as the agent is running and the machine is not shut down

Given I’ll only be needing the agent infrequently when I’m making a release, and it only needs to sign the release on GitHub, these concerns are minimal. Time to get cracking!

With the old build script cleaned up and a new step to handle the release signing, the only key change I need to make is to target the Buildkite agent running locally rather than using the default queue.

 - label: ":github: Sign and publish release"
 key: sign-release
 depends_on: create-release
 command: .buildkite/sign-release.sh
 if: build.env("BUILDKITE_TAG") =~ /^v\d/
+ agents:
+ queue: nchlswhttkr

When a build runs the brunt of the work is done on default agents, but the signing step targets the agent with matching tags that I’m running locally.

Locally, I can see the Buildkite agent running the job.

I’m prompted to unlock my GPG key by the pinentry program I wrote for myself.

Finally, the signed checksum is uploaded to the GitHub release. The new provider version can now be downloaded and used by Terraform!

With signing on the local agent working, I can do away with the second key I needed before. All of my releases can now be signed with my regular GPG key, and better yet without compromising on how I limit access to it!

My Apple Watch, six months in

Nicholas Whittaker — Sun, 16 Jul 2023 22:24:24 +1000

Since late September, I’ve been working to establish an exercise routine as I improve my fitness. One of my decisions in that journey has been to buy a smart watch, and having just ticked over six months since purchase I thought it would be an opportune time to reflect on some of my highlights.

Seeing as I predominantly use Apple devices already and wanted to avoid any great deal of setup, I opted for an Apple Watch. They’re still pricier than a nice Fitbit or Garmin watch, but with the recent release of the more afforable Apple Watch SE, I was happy taking the plunge. While it doesn’t have all the features of the flagship watches, it has essentials for me like fall detection and a battery life that can easily last a full day out.

With the watch in hand and the sunk cost as a motivation to be active, I’ve been using it to record my workouts and my daily activity. All the data is synced across to my phone, visible in the Health app alongside a host of other derived and self-reported measures like resting heart rate and my weight. I try not to focus on the day-to-day numbers too much, but seeing where I’m trending is excellent motivation to continue my work.

Recording activities has also meant I can use Strava to share my activities with my friends, which in turns drives me to keep up my habits. I’ve paid for an annual membership, which has been useful for setting myself weekly goals and identifying how hard I’ve been pushing myself.

Riding

Late last year I started riding my bike again, looking for more intense activity than just a brisk walk. I’m fortunate to live close to the bay trail along Melbourne’s east coast, so I worked myself up travelling further and further along the coast until I could reach the CBD or go down to Mordialloc. Weekends have also been a good opportunity to go on longer rides!

With these more intense rides, it’s been useful to see how much I’m exerting myself on my watch. I’ve actually switched over the default display I use for riding to one that shows my heart rate and Apple’s approximation of my zones.

I’m working towards completing the 100km ride for Around The Bay in October as well, and the group training rides I’ve done so far have been good both for the exercise and the social coffee afterwards.

Riding at higher speeds and as part of a group is still quite daunting, but the folks at Bicycle Network organising the rides have been welcoming to newcomers like myself. It feels pretty awesome to go zooming along Beach Road bright and early on a Saturday!

Running

Back in May, I went on a holiday to Phillip Island to compete in a lawn bowls tournament. I had hoped to borrow a bike in Cowes, but with the winter weather and no gear I was content wandering the town. The day after arriving though I had some slack time before dinner, so I decided to go for a run along the beach to close my rings for the day.

That first run was pretty unpleasant, and after a very overeager starting pace I was glad that I’d opted to walk after the first kilometre. The second leg of the run was also grueling, particularly with a couple of hills on the way back to my hotel room. I was glad to be done and collapsed into my bed when I got back. The parma for dinner that night was particularly satisfying.

Luckily my legs were feeling alright for the tournament the next day, although I couldn’t handle the wind and was eliminated in the group stage.

On my last day in Cowes, I went for another run before checking out. Although it was also pretty tough, extending the distance to run a little further and see more along the coast made it more enjoyable.

Since then, I’ve tried to run at least twice a week. My route varies, but I’ve found a flat-ish 2.5km circuit around my local park where I can focus on balancing my pace against my exertion. My exercise group will sometimes do weekend runs, mixing distance with exercises like stair climbing.

I’m still yet to hit 5km, but most recently a double lap of Caulfield Park was a good progress check. Running with my friend helped me push myself, and being able to run twice the distance I did two months ago (with a 5 minute rest!) is a proud accomplishment for me.

As with cycling, it’s been insightful to see the analysis my watch and Strava provide. Running is much more intense for me, so being able to see the changes measured and visualised is in itself rewarding. My recovery has improved, my heart rate climbs at a slower rate than before, and my pace over the same course each week has improved dramatically.

Seeing the slow trend up as I run more each week feels great too.

Closing my rings

Lastly, being able to able to see a general measure of my acitivty visualised with the three rings Apple uses has itself been a good driver to keep myself active. Even if I’m having a day off, it feels satisfying to close my rings with a walk so I know I’m not spending the entire day inside sitting idle at my desk.

It’s also pretty cool that I can scroll back through my activity and see six months of progress. I might have been sick from time to time, but at the end of the day I’ve built a strong habit since I got my watch.

Fixing a slow Tailscale SSH connection

Nicholas Whittaker — Sun, 25 Sep 2022 22:00:00 +1000

I’ve been trying out Tailscale recently to simplify networking between my devices. With the beta launch of Tailscale SSH offering the ability to connect to my DigitalOcean droplet without SSH keypairs, I was eager to incorporate it into my setup.

Said setup is a matter for another time, but with Tailscale SSH enabled for my droplet I was able to remote in with a plain ssh nicholas@gandra-dee!

However, there was a visible half-second delay when entering commands. While not a dealbreaker, it made each session a frustrating experience!

I started off by getting a gauge of the problem. How responsive was the connection? It wasn’t too slow, but it wasn’t fast either.

$ ping -qc 10 gandra-dee
PING gandra-dee.nchlswhttkr.github.beta.tailscale.net (100.79.138.83): 56 data bytes

--- gandra-dee.nchlswhttkr.github.beta.tailscale.net ping statistics ---
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 353.175/535.999/1690.879/398.688 ms

Fortunately in this debugging adventure, I did have a suspect: I’d recently added a firewall on the DigitalOcean side, so there was a good chance I’d missed a rule along the way. It was strange that I could still connect to the droplet despite the firewall likely blocking some of my requests.

Glancing through the Tailscale documentation on firewall rules revealed a likely cause for the slower than expected connection.

However, when both devices are on difficult networks Tailscale may not be able to connect devices peer-to-peer. You’ll still be able to send and receive traffic, thanks to our secure relays (DERP), but the relayed connection won’t be as fast as a peer-to-peer one.

Bingo. A quick tailscale status confirmed the connection was over a relay.

$ tailscale status
100.79.138.83 gandra-dee nchlswhttkr@ linux -
100.82.154.68 flugelhorn nchlswhttkr@ iOS offline
100.88.173.96 phoenix nchlswhttkr@ macOS active; relay "syd", tx 1764024 rx 22460936

While I’d remembered to add a rule to ufw on the droplet to allow Tailscale traffic, the firewall rules on the DigitalOcean side were blocking traffic beyond that. Even though Tailscale couldn’t connect over its typical port it was able to fall back to the HTTPS relay, which the firewall allowed.

The fix is thankfully quick, adding a couple of firewall exceptions with Terraform.

# Allow inbound Tailscale requests
inbound_rule {
 protocol = "udp"
 port_range = "41641"
 source_addresses = ["0.0.0.0/0", "::/0"]
}

# Allow outbound Tailscale requests
outbound_rule {
 protocol = "udp"
 port_range = "41641"
 destination_addresses = ["0.0.0.0/0", "::/0"]
}

Reconnecting to the instance now after applying the change, the remote session starts noticeably faster! The response time to my typing is immediate, and my calm is restored. Running tailscale status again confirms the problem is fixed. (Omitting my IP, of course)

$ tailscale status
100.79.138.83 gandra-dee nchlswhttkr@ linux -
100.82.154.68 flugelhorn nchlswhttkr@ iOS offline
100.88.173.96 phoenix nchlswhttkr@ macOS active; direct 192.0.2.0:22697, tx 1794276 rx 22486396

What’s the connection like now?

$ ping -qc 10 gandra-dee
PING gandra-dee.nchlswhttkr.github.beta.tailscale.net (100.79.138.83): 56 data bytes

--- gandra-dee.nchlswhttkr.github.beta.tailscale.net ping statistics ---
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 89.109/121.486/405.295/94.605 ms

Bliss.

Happy anniversary to Journey!

Nicholas Whittaker — Mon, 14 Mar 2022 16:00:00 +1100

As I’ve just found learned, today is the anniversary of Journey’s release ten years ago. It’s a video game from thatgamecompany where you guide a mysterious wanderer in robes on a pilgrimage to climb a distant mountain.

Journey is one of the games that’s stuck with me a lot since I’ve played it.

Largely a single player experience, it’s possible to encounter another lone human-controlled character on your travel. Communication is limited to a single button for chirping, so you and your partner must figure out your own language as you adventure together. Short repeated chittering might indicate excitement or urgency, whereas longer humming leans closer to a conversation.

It’s also the first game I went for completion in, earning the white robe for my character by collecting all the hidden sigils. Even then, I’ve still replayed it from time to time in the years since.

JOURNEYQUEST by NightMargin

Thinking back to when I first saw Journey on the video game review show Good Game, I’m reminded of how I was a decade ago. I used to avidly tune into the ABC on Tuesday evenings to see the new game reviews. These days, Hex has moved on and Bajo streams on Twitch on those same Tuesday evenings.

In the end, I’m left with memories of Journey that I hope will stay with me for a long time. I’ve got the soundtrack from Austin Wintory on vinyl and digital, and I look forwrad to playing it again one day.

Listen to Journey, by Austin Wintory on Bandcamp

A good start for October

Nicholas Whittaker — Fri, 01 Oct 2021 16:00:00 +1000

Somehow, we’re now into the final quarter of the year. Here’s a few things that interest me, and may interest you!

Hacktoberfest is back for another year, hopefully the controls they introduced after the terrible spam-fest of 2020 will avoid creating too much work for maintainers. It’s nice to see that they’re making merge requests on GitLab eligible too, and highlighting financial contributions as a way to support open source software.

A friend of mine in the UK is running a Kickstarter campaign to sell enamel Pikachu pins, if pins are your thing.

Today is also Bandcamp Friday, where Bandcamp waives their service fee for sales. It’s an excellent opportunity to support smaller/local artists and independent labels.

Here’s five albums I’ve been listening to recently.

Listen to Indigo, by Nadje Noordhuis + James Shipp on Bandcamp

I saw Nadje perform with Luke Howard when we weren’t in lockdown earlier this year. The world needs more gentle trumpet and flugel soloists, her sound is honey for my ears.

Listen to THE SAMBA AND CRUNK EP, by brad bellard on Bandcamp

Brad’s a Melbourne-based funk keys player, he livestreams a few times each week on Twitch. In fact, he’s live as I write this!

Listen to There's a Field, by Hello Satellites on Bandcamp

Hello Satellites was one of the performances I attended at Tempo Rubato in Brunswick before the most recent bout of lockdowns. First time I’ve seen a harp live too!

Listen to What We Call Life, by Jordan Rakei on Bandcamp

Jordan Rakei’s voice is magical, and his new album sounds incredible. I’ve been thoroughly enjoying it this last week.

Listen to My Heart is Live in Berlin, by Theo Katzman on Bandcamp

I love him on Vulfpeck and I love him as a solo artist, Theo is a powerhouse with his voice and guitar. He’s got three great albums under his belt now, and this live album is great introduction in my opinion.

Happy listening!

A suspicious email from Cloudflare

Nicholas Whittaker — Thu, 23 Sep 2021 20:00:00 +1000

I received an email from Cloudflare today about an account that had been created with my email address. Going by the details and the headers, it certainly seemed to have originated from Cloudflare. It looked like someone else had signed up with my details.

Now, it’s worth noting that I do already use Cloudflare to manage this website. The email address I use for that account is hosted separately though, otherwise I’m one DNS mistake away from making my inbox unreachable!

A bit of investigating online later, and it looks like this certainly is malicious behaviour.

Going by this Twitter thread, it seems unverified accounts are able to provision API credentials. Unlike an authenticated browser session, these credentials persist even when you reset an account’s password.

If a target went through the reset flow to gain control of the account created in their name, the attacker would still have near-complete account access with existing API credentials. These would continue to persist until deleted from the account.

I’m not sure why an attacker would target me, but I assume they just got my email as part of a list. Fortunately, it’s straightforward to reset the account’s password (without verifying the email address) and request the account be deleted.

Backing up and restoring a self-hosted Plausible instance

Nicholas Whittaker — Sat, 14 Aug 2021 11:00:00 +1000

I’ve been using Plausible Analytics on this website for a few months now and I’m a fan for three key reasons.

They’re open source
They explicitly detail how they track web traffic in a privacy-preserving manner
Customers can self-host their own Plausible instances

Being able to self-host Plausible gives me ownership of the data it collects, but it also makes me responsible for storing this data and backing it up. I manage backups for my instance with several Ansible playbooks, but the same can be done with plain shell commands.

A self-hosted Plausible instance is run as a collection of Docker containers and volumes managed by Docker Compose. As a result, a full backup of each volume gives you a copy of all of the data you’ll need for a restore.

So let’s dive in! To start, bring down your running Plausible instance.

cd hosting # your clone of github.com/plausible/hosting
docker-compose down

With Plausible stopped, you can take a copy of each Docker volume. I do this by mounting each volume to a plain Ubuntu container and running tar, writing to a mounted directory on the host.

mkdir $HOME/backups/

docker run --rm \
 --mount "source=hosting_db-data,destination=/var/lib/postgresql/data,readonly" \
 --mount "type=bind,source=$HOME/backups,destination=/backups" \
 ubuntu \
 tar --gzip --create --file /backups/plausible-user-data.tar.gz --directory /var/lib/postgresql/data/ .

docker run --rm \
 --mount "source=hosting_event-data,destination=/var/lib/clickhouse,readonly" \
 --mount "type=bind,source=$HOME/backups,destination=/backups" \
 ubuntu \
 tar --gzip --create --file /backups/plausible-event-data.tar.gz --directory /var/lib/clickhouse/ .

To restore a backup, you can remove the old volumes and extract your tarballs into new volumes.

docker volume rm hosting_db-data hosting_event-data

docker run --rm \
 --mount "source=hosting_db-data,destination=/var/lib/postgresql/data" \
 --mount "type=bind,source=$HOME/backups,destination=/backups,readonly" \
 ubuntu \
 tar --extract --file /backups/plausible-user-data.tar.gz --directory /var/lib/postgresql/data/

docker run --rm \
 --mount "source=hosting_event-data,destination=/var/lib/clickhouse" \
 --mount "type=bind,source=$HOME/backups,destination=/backups,readonly" \
 ubuntu \
 tar --extract --file /backups/plausible-event-data.tar.gz --directory /var/lib/clickhouse/

All that’s left after this is to restart the Plausible containers. You may also want to pull changes from Plausible’s hosting repo and the latest Docker images.

# Optionally, update your clone and pull latest images
git pull
docker-compose pull

docker-compose up --detach

From here, it’s up to you what you do with your backups. I’d suggest moving them to an external store, whether that’s your machine via rsync or a storage service with your provider of choice.

Happy coding!

Exporting iCloud reminders to Fastmail

Nicholas Whittaker — Sat, 24 Jul 2021 15:00:00 +1000

If you’ve snooped the MX records for this site recently, you might have noticed that I’ve moved to Fastmail. In addition to email hosting, Fastmail also offers CalDAV accounts for users, so I’m trying it out for my calendar and reminders.

While the Apple ecosystem supports CalDAV accounts, they don’t make it easy for you to export your reminders from iCloud into your account of choice. A Reddit post points out that it is possible to copy these reminders across with the Shortcuts app though, so I decided to give that a try. Here’s a test run with a simple list of reminders.

There’s a few catches with the approach to be aware of, since iCloud reminders have some exclusive (CalDAV-incompatible) features.

If the reminder has notes attached, they need to be included in the Notes of the “Create reminder” widget or they won’t be copied
Some details can’t be transferred - namely attached photos, due dates and URLs

When selecting the reminders to migrate, you’ll want to filter out completed reminders. When the reminders are created in your new list, they’ll be marked as incomplete.

Interestingly, a confirmation prompt shows up when deleting reminders. If you’re deleting many at once (I had one list of about ~70 reminders), you’ll have to OK it several times before the shortcut proceeds.

Afterwards the shortcut finishes running, all of your reminders will have moved across and you’ll be good to go!

Here’s the shortcut I wrote, if you’d like to use it.

A close call with Nginx and the alias directive

Nicholas Whittaker — Thu, 15 Jul 2021 23:00:00 +1000

When Nginx is serving this website, it’s usually serving static files from the local machine. One method to accomplish this is with the alias directive, which substitutes the request location for a filepath. I use it to map requests to nicholas.cloud/files/ to a directory for public file-sharing.

One catch with this setup is that you’ll get a 404 if you visit nicholas.cloud/files, as it lacks a trailing slash. Users often overlook and forget this slash, so many websites these days choose to deal with it internally and show the right page.

If this trailing slash is such an encumbrance, why not drop it from my own config? This way, if someone goes to nicholas.cloud/files, they’ll end up on the right path.

I figured it would be nice to have, so I made a small change to my Nginx config.

 # FILES
- location /files/ {
+ location /files {
 alias /home/nicholas/public-files/;
 add_header "Cache-Control" "public, max-age=0, s-maxage=60";
 }

One reload later, and requests missing the trailing slash were successfully redirected! But while this change was convenient, I felt concerned at the time that it was a tad unsafe. I decided to experiment.

Poor path matching logic is a common (and lucrative) vulnerability for webservers like this. If an attacker can access parent directories above the target folder, all manner of sensitive files on the host can be exposed.

As it turns out, my change created exactly this opening.

Lo and behold, Nginx was now trying to serve the TLS private key that sits in the root of my user directory. There’s two things to note here.

It is not wise for these senstive files to be sitting in a such an exposed location.
I’m really glad I set the file’s permissions to be private now.

At that moment, any path starting with /files would follow the alias directive. Nginx was accessing /home/nicholas/public-files/../nicholas.cloud.key, and finding my key.

Thankfully, the fix this time was only a quick revert away. If only it was always that easy. 😓

Next time, I think I’ll stick to writing a workaround rule rather than making such a reckless change. 😅

As a point of reflection, it’s worth noting that the Nginx documentation for alias specifically uses the term “replacement”.

Defines a replacement for the specified location.

If that isn’t a big caution sign, I don’t know what is!

Managing secrets in a Rush monorepo with Pass

Nicholas Whittaker — Mon, 14 Jun 2021 14:00:00 +1000

For a while now, I’ve used Rush to manage and deploy a number of Cloudflare Workers from a monorepo. It’s been great for me so far, offering incremental builds and support for alternative package managers like PNPM.

One thing Rush leaves to the discretion of maintainers is secrets management. Given tooling and infrastructure can vary drastically between organisations and even individual projects, there’s nothing wrong with this decision. However, it has lead to me implementing my own less-than-desirable setup.

Every project follows its own build.sh script to load secrets, build and deploy
Cloudflare-related credentials are read from a shared script
Workers that need third-party API tokens read them from their own .env file

This works, but it has a number of shortcomings. What if a worker needs to be deployed to a different Cloudflare zone (website) from every other worker? How do I manage/keep track of all these .env files?

I ended up looking to the pass password manager. I’ve found it convenient for my personal projects, as it leverages my existing GPG setup and makes it easy to store/retrieve secrets from the command line.

A few changes later, and now the build scripts for each project are explicit about what secrets they need! Here’s an abridged example.

- source ../../set-cloudflare-secrets.sh
+ export CF_ACCOUNT_ID=$(pass show workers/cloudflare-account-id)
+ export CF_API_TOKEN=$(pass show workers/cloudflare-api-token)
+ export CF_ZONE_ID=$(pass show workers/cloudflare-zone-id-nicholas.cloud)
- source .env
+ export MAILGUN_API_KEY=$(pass show workers/newsletter-subscription-form/mailgun-api-key)
+ export EMAIL_SIGNING_SECRET=$(pass show workers/newsletter-subscription-form/email-signing-secret)

I did find an interesting interaction between Rush and the GPG agent. Rush attempts to build projects in parallel where possible, and if too many processes are decrypting secrets at once the GPG agent will return a Cannot allocate memory error.

Thankfully this can be fixed by adding the --auto-expand-secmem option to the agent’s config. This allows gcrypt (used by GPG) to allocate secure memory as needed.

# ~/.gnupg/gpg-agent.conf
auto-expand-secmem

With the GPG agent restarted, I can now build many projects with secrets in parallel! It’s also good to have my secrets sitting safely outside source control, stored in a place where I can easily back them up.

Using pass to fetch and decrypt secrets does admittedly add a few seconds to each build. Thankfully, Rush’s parallelism keeps the overall build comparatively fast. In my eyes, the tradeoff is worth it.

I'm excited for The Great Ace Attorney Chronicles and the Story Mode feature

Nicholas Whittaker — Sat, 29 May 2021 23:00:00 +1000

Late last year, I grabbed the original Ace Attorney trilogy on discount after stumbling across a trending fan video. I’ve now played through all six mainline games, and I’m optimistic for a seventh to coincide with the series’ 20th anniversary later this year.

While a new game isn’t confirmed, Capcom will be releasing a localised bundle of The Great Ace Attorney 1 & 2 in a few months. This duo was previously exclusive to Japan, and it’s exiting to see them getting an international release.

A feature being introduced in the game is Story Mode, where the game will automatically play itself on the player’s behalf. Some internet shadows are criticising the optional feature, saying it removes the logic puzzles and problem solving the series is founded upon.

A reply to the critics captures the motivation behind this feature simply and succinctly. Every player should have the opportunity to experience these games, without the needless trial-and-error that the games can sometimes be guilty (ha!) of.

You won’t [need story mode], but some might. This isn’t for you, it’s for them.

Anyway, I’m looking forward to contradiction hunting on July 27th!

HERLOCK SHOLMES REAL by Fudgenuggets

How do you navigate copyright issues around England’s greatest detective? Just call them Herlock Sholmes instead!

Pretty commits

Nicholas Whittaker — Tue, 25 May 2021 22:00:00 +1000

In a chance glance through the Git history of one of my projects last week, I found commit 53331294. A hash with eight leading numbers! A sequence like this isn’t particularly rare, but it’s also not an everyday occurence.

Today at work, I notice the palindrome 39eee93 while using git reset¹. SHA-1 works in mysterious ways. :thinking_face:

Of course, there are plenty of other pretty commits out in the wild. You can always try brute-forcing a commit with more leading zeroes than 00000000000000, or cracking a joke with deadbeef.

If you’re musically inclined, what melodies use the natural notes bar G? The lick is unfortunately not SHA-friendly.

When you commit next, consider a pause to admire the hash. Who knows what pattern you might spot?

¹ If you find yourself navigating the fog that is git rebase, remember to throw out a tag here and there. It’s a great catch if (like me) you occasionally need to reset. 😅

Live previewing Hugo sites with Cloudflare Tunnel

Nicholas Whittaker — Sun, 09 May 2021 15:00:00 +1000

Recently, Cloudflare made their Tunnel service (formerly Argo Tunnel) free and available for all Cloudflare users. For an existing free-tier user like myself, there’s no better time to try it out!

Tunnels are designed for securely connecting an origin server to a web-facing endpoint. They’re particularly great if

You’re a service operating from a network/zone that doesn’t allow inbound connections
You’re a developer looking to expose your local server to the web

There are generic tools out there that fill this niche (ngrok is great for quick demos, and there’s inlets if you prefer self-hosting), but Cloudflare’s offering works well for me because I already use them to manage my website.

Down to business - I’ll be using my website for this example because it’s built with Hugo.

After installing the cloudflared tunnel client, there’s a few commands to authenticate and create a tunnel. I’m calling my tunnel preview, and exposing it at tunnel.nicholas.cloud.

cloudflared tunnel login
cloudflared tunnel create preview
cloudflared tunnel route dns preview tunnel.nicholas.cloud

With a tunnel created, we can start tunnelling traffic from port 1313, the default port for Hugo’s dev server.

cloudflared tunnel run --url localhost:1313 preview

Visiting https://tunnel.nicholas.cloud/ now will show a 502 Bad gateway error, since our Hugo server isn’t running yet. Let’s start it up!

hugo server \
 --baseURL=https://tunnel.nicholas.cloud/ \
 --liveReloadPort=443 \
 --appendPort=false

There’s a few options we specify with this development server, seeing as we’re no longer serving traffic from localhost.

Overriding the default URL to your chosen URL with baseURL
Disabling appendPort, since the tunnel’s exit responds to traffic on the default HTTPS port 443
Likewise, the liveReloadPort should be 443 so pages will be reloaded as we make changes

There you have it, a live preview of your Hugo site on your own domain thanks to Cloudflare Tunnel!

Thanks to Hugo (not the site generator!) for his bit on previewing Hugo sites in GitHub Codespaces, which inspired me to try this out!

There's a lot of automation going on here

Nicholas Whittaker — Sat, 01 May 2021 21:00:00 +1000

With a little bit of invested time, I’ve figured out a nice little process for previewing changes to my site. The technical side of it is a writeup for another day, but it’s too cool to not to talk about here and now.

I start with my local changes on a new branch, pushing it to GitHub. The response includes a link to create a new pull request.

git push -u origin HEAD
# remote:
# remote: Create a pull request for 'disable-analytics-in-preview' on GitHub by visiting:
# remote: https://github.com/nchlswhttkr/website/pull/new/disable-analytics-in-preview
# remote:
# To github.com:nchlswhttkr/website.git
# * [new branch] HEAD -> disable-analytics-in-preview
# Branch 'disable-analytics-in-preview' set up to track remote branch 'disable-analytics-in-preview' from 'origin'.

The commit message determines the pull request’s title. Referencing an open issue in the PR’s description makes the two pages reference each other.

PR opened. A workflow I’ve made with GitHub Actions kicks off, and my changes are deployed to a preview domain. A link to the deployment is included with the pull request.

PR merged. The same workflow kicks off and cleans up the deployed preview, marking it as inactive. The merged branch is deleted. Since the description a reference to “close #41”, the corresponding issue is also closed. ¹

It’s kinda amazing how much of a breeze previewing is for me now, only ever a pull request away. GitHub’s developer experience really shines here too.

¹ Fun fact: GitHub doesn’t like it when you close 100 issues at once with a single pull request.

No FLoC for me please Google

Nicholas Whittaker — Fri, 23 Apr 2021 22:00:00 +1000

A new approach to online tracking being trialed by Google has been under scrutiny this week, from developers and privacy advocates alike.

Along with others, I’m explicitly opting-out my website out of this program. Thankfully it’s a one-liner in my Nginx config.

+ add_header "Permissions-Policy" "interest-cohort=()";

According to Google, this tracking method is activated (for the time being) when sites do either of the following.

Call document.interestCohort()
Load ads or ad-related resources (whatever Google determines this to be)

With that said, there’s no saying when or how this will change. Taking the hardline approach to avoid this behaviour seems sensible to me.

I’ve got more to say on the stances companies like Google take on privacy, but that’s a longer piece of writing for another day.

Thanks to Ruben Schade for his bit on FLoC too!

Traversing the sources of a song

Nicholas Whittaker — Fri, 16 Apr 2021 23:00:00 +1000

I stumbled across an old animation from a few years ago yesterday, set to a rather funky beat. Not the most fascinating thing on its own. Interesting though, how my perspective has changed a bit after looking into the music a little bit more.

By itself, the song is great. There’s more lurking beneath the surface though.

It uses the vocals from Fire Fly by Childish Gambino.

The vocals are mashed up with the instrumental from MF DOOM’s Coffin Nails.

DOOM uses a sample from Dave Matthews’ guitar solo in Space Oddity.

Travere the tree, and you’re listening to a mashup with a sample from a cover of a David Bowie song!

Isn’t it funny how deep the roots of a song can reach?

bingo sandbelt for senator by an0nymooose