Local TLS the easy way, with Let's Encrypt and Certbot

One huge and welcome change in the last decade or so for Web development is that secure connections are ubiquitous. Free certificates are available and most hosting providers make it very easy to obtain, use and renew the certificates automatically. This site, for example, runs on Amazon Web Services and enabling https took not much more effort than ticking a few checkboxes.

Browsers, like Chrome, Firefox and Safari, have also played their part, by gently steering users toward secure connections, warning about sending sensitive information over plain text and switching to https automatically, when available.

The last bastion for plain old http connections are web apps running on your local network, like a home wiki. But, so what? This is my local network, right? Yes, it is, but using SSL to encrypt the traffic between your browser and the application is a good, belt-and-braces, defence-in-depth security step. It can protect you from a dodgy device or compromised application, inside of your network, from listening in on your traffic.

Enabling local certs

There are a few ways to enable SSL connections to local servers. The easiest way is just use self-signed certificates. The disadvantage is that it requires you to accept these certificates on each browser you want to use.

Another option, which many larger companies use, is to run your own Certificate Authority. That isn't easy, and you'll still need to configure all of your browsers to accept your local root certificate. (This can be done with configuration scripting like Puppet or something. But see the isn't easy part...)

Either of those methods would be fine if I were the only user, but that isn't always the case. We have a local, household wiki, which acts as this house's README. I want to make that available to any guests, where they can find info like where the cat food is, when the rubbish is picked up and where the water shut off valve is. Once connected to my WiFi, guests can open the wiki using QR codes posted around the house. I don't want them to have to click through any warnings about insecure connections or have to accept any self-signed certificates.

Local domain

The option I went with was to use a real, Let's Encrypt certificate for a domain I have registered but not used publicly. In my case, I had sullivanhome.org.uk available.

The first step was to tell my router that this domain is the "local" one, which is easy enough to do on most routers, as long as you can get to the admin pages.

Once this is set up, local computers resolve on that domain.

Certbot

The next step is to obtain a wildcard certificate (e.g., one for "*.sullivanhome.org.uk"). My applications run across two Raspberry Pis, one called Pineapple and the other called Blueberry. I want one cert that I can apply on both, so that either pineapple.sullivanhome.org.uk or blueberry.sullivanhome.org.uk present as valid (with a lock icon and no errors) in the browser.

Fortunately, this is made extremely easy with certbot. This little tool from the Electronic Frontier Foundation automates the process of obtaining and periodically renewing the certificate almost entirely.

On Pineapple (which runs Ubuntu), I installed certbot from the snap store. I also had to change the snap store permissions and install a plugin for the registration provider I'm using (Amazon's Route53). There are plugins for a wide variety of registrars.

sudo snap install --classic certbot
sudo snap set certbot trust-plugin-with-root=ok
sudo snap install certbot-dns-route53

To get our certificate, I just ran the following.

certbot certonly --dns-route53 -d sullivanhome.org.uk -d *.sullivanhome.org.uk

That command needs a bit of explanation: The certonly argument tells certbot to just download the certs. It can install the certs directly in many applications and web servers (e.g., nginx or apache), but I have several different applications using the same certs, so I'll do that later. The -dns-route53 option tells certbot to use the plugin to temporarily add entries in the DNS to prove that we own the domain. Finally, the -d option tells certbot what certificates to register.

Certbot then prompted me for various bits it needs (like my email address), registered me with Let's Encrypt and downloaded the certificates to /etc/letencrypt. The /etc/letsencrypt/live directory looks like:

lrwxrwxrwx 1 root root  43 Jan  4 15:54 cert.pem -> ../../archive/sullivanhome.org.uk/cert1.pem
lrwxrwxrwx 1 root root  44 Jan  4 15:54 chain.pem -> ../../archive/sullivanhome.org.uk/chain1.pem
lrwxrwxrwx 1 root root  48 Jan  4 15:54 fullchain.pem -> ../../archive/sullivanhome.org.uk/fullchain1.pem
lrwxrwxrwx 1 root root  46 Jan  4 15:54 privkey.pem -> ../../archive/sullivanhome.org.uk/privkey1.pem

Automatic renewals

Notice that the live certs (the currently active certs) are symlinks. The software also sets up a systemctl timer service that will renew the certificate before it expires and update those symlinks. The renewal feature allows me to provide a script that gets called on a successful renewal (in /etc/letsencrypt/renewal-hooks/post/post-renew-certs.sh). I use this script to copy the file (via scp) from pineapple to blueberry. All of my local web apps and services run in docker, so I also restart the containers on both Raspberry Pis.

#!/bin/bash

# Make a backup copy on blueberry
scp /etc/letsencrypt/archive/sullivanhome.org.uk/* blueberry.local:/etc/letsencrypt/archive/sullivanhome.org.uk
# This will follow the symlinks and create non-symlinked files
ssh blueberry.local rm /etc/letsencrypt/live/sullivanhome.org.uk/*
scp /etc/letsencrypt/live/sullivanhome.org.uk/* blueberry.local:/etc/letsencrypt/live/sullivanhome.org.uk
# restart the services on blueberry -- this script just navigates to the appropriate directory
# runs docker compose restart (like below)
ssh blueberry.local /etc/letsencrypt/renewal-hooks/post/post-renew-certs.sh

cd /media/homeassistant/home_assistant/
docker compose restart

Using the certs

Changing the applications so that they present themselves using these certificates varies a lot by application. Node RED, for example, provides a function in its configuration to retrieve the certs, force https and periodically pick up any changes to the certs. I make the certs available in the docker container in a volume and then:

https: function() {
   // This function should return the options object, or a Promise
   // that resolves to the options object
   return {
       key: require("fs").readFileSync('/certs/live/sullivanhome.org.uk/privkey.pem'),
       cert: require("fs").readFileSync('/certs/live/sullivanhome.org.uk/fullchain.pem')
   }
},

/** If the `https` setting is a function, the following setting can be used
 * to set how often, in hours, the function will be called. That can be used
 * to refresh any certificates.
 */
httpsRefreshInterval : 12,

/** The following property can be used to cause insecure HTTP connections to
 * be redirected to HTTPS.
 */
requireHttps: true,

Wiki

For the home wiki, it's a bit more complicated. It has a database (PostGres) behind it and a front end that does support Let's Encrypt certificates, but in a way that doesn't work well with my setup. So, instead, I installed nginx (sudo apt install nginx) and used what's called a reverse proxy. This sits in front of the non-SSL enabled site and just proxies it, serving it over SSL.

Install and configure nginx

On blueberry, I created a file in /etc/nginx/sites-available called blueberry.sullivanhome.org.uk. The contents of that file is as follows:

server {
listen 80;
listen [::]:80;
listen 443 ssl http2;
listen [::]:443 ssl http2;

ssl_certificate /etc/letsencrypt/live/sullivanhome.org.uk/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/sullivanhome.org.uk/privkey.pem;

   server_name blueberry.sullivanhome.org.uk;
    location / {
        proxy_pass     http://localhost:6830;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Then make that site "available" with a symlink and start it:

sudo ln -s /etc/nginx/sites-available/blueberry.sullivanhome.org.uk /etc/nginx/sites-enabled/blueberry.sullivanhome.org.uk
sudo systemctl reload nginx
# or restart
sudo systemctl start nginx

That's it. When you go to https://blueberry.sullivanhome.org.uk in your browser, nginx forwards your request to localhost:6830, which is where I set the wiki to serve from.

And, that's it

This set up has been working well for me now for three auto-renewals of the 90-day cert and happens without any intervention on my part. But if something does go wrong, I have all I need (backups of the certs, etc) to fix things.

The caveat is that I'm not a security expert and this is unlikely to be the best approach for a public app, or for use in a public place (a restaurant's WiFi, maybe) but it works for my purposes.