March 12, 2020

Well, that was almost too easy.

As described in my previous post, I wanted to automate publishing to this site via Github Actions. As a Jenkins user at work, I expected this to take some time, requiring reading reams of documentation and writing fiddly scripts that don't work the first 56 times you try them.

In the end it took me about 30 minutes, and hardly any RTFMing.

Publishing workflow

The process I wanted to automate is basically two steps, but it requires a bit of setup. I have to have a Java environment and a Clojure build tool called Leiningen. I also need the Amazon Web Service command-line interface and some environmental variables (the S3 bucket to use, the AWS access key and secret, etc.).

Setup secrets

The first step I did was create a new AWS user, with limited access, to do the uploading. I create a user that is for script-use only (it can't log into the console). It also needed access to my S3 bucket, but nothing else.

I then generated a key and secret for this user, and stored that on my repository using the Secrets part of the repository's settings.

Investigate existing workflows

Github has a marketplace of Actions. These are actions that people have put together and shared publicly. In addition to the standard Github ones (for checking out master and setting up Java) needed one for installing and configuring Leiningen and another for syncing my generated pages with the S3 bucket.

For syncing with S3, S3 Sync comes highly rated. For setting up my Clojure tools, I used Setup Clojure.

Combine everything in a workflow

The final step is to create a new directory and file in my repository: .github/workflows/publish.yml.

The first couple of attempts had mistakes in it, but fortunately, I did the work on a branch and then created a pull request. Github will automatically syntax-check any workflows in the pull request. The first time, I mixed the uses and runs keys in the same steps, which isn't allowed. The second time I tried it, it caught a simple syntax error (a missing full colon).

Once the check passed, I merged and then fixed a typo to check if it worked. And it did. The first time. I'm still catching my breath!

Here's the final script:

name: Upload Website

    - master

    runs-on: ubuntu-latest
    - name: Checkout Master
      uses: actions/checkout@master
    - name: Prepare Java
      uses: actions/setup-java@v1
        java-version: 1.8
    - name: Prepare Clojure
      uses: DeLaGuardo/setup-clojure@master
        lein: 2.9.2
    - name: Build
      run: lein run
    - name: publish
      uses: jakejarvis/s3-sync-action@master
        args: --follow-symlinks --delete
        AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
        AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
        AWS_REGION: 'eu-west-1'
        SOURCE_DIR: 'public'

Notice that the only part that needed to be explicitly run was the lein build command (lein run), everything else just happens. And the steps are nice and clear. The labels I gave to the steps (name:...) show up in the actions, and I can drill down into each step and see the logs.

Overall, this was very easy and impressive. The site-generation and the continuous integration pipeline is beginning to look like a usable MVP (minimum viable product) for the charity site.

Tags: github continuous integration programming