radu's blog

Cross platform GitHub Action for downloading, extracting, and adding tools to path

· Radu Matei

Ever since I started to use GitHub Actions, one of the tasks I copy and pasted the most contained the following steps:

  • download a file or an archive containing a statically compiled tool
  • extract if it is an archive
  • copy the target tool to a directory in the path

And after an embarrassing number of tries, my jobs would contain a step that would resemble the following (taken from an actual GitHub Action):

jobs:
  configurator:
    runs-on: ubuntu-latest
    steps:
    - name: configure Helm 3
      run: |
        export HELM_PLATFORM=linux-amd64 && export HELM_VERSION=helm-v3.0.0-alpha.2
        wget https://get.helm.sh/$HELM_VERSION-$HELM_PLATFORM.tar.gz
        tar -xvzf $HELM_VERSION-$HELM_PLATFORM.tar.gz
        rm -rf $HELM_VERSION-$HELM_PLATFORM.tar.gz
        mv $HELM_PLATFORM/helm <some-directory-in-path>/helm3
        chmod +x $GOBIN/helm3

Now, the above works just fine you only ever need to to this task once - but this has a couple of implications:

  • every time I need to configure another tool I would need to copy and paste the configuration.
  • if I want to configure the same tool on Windows, I would have to find out the PowerShell syntax for this, which would most likely mean I just wouldn’t set up a Windows job.

So how could all this be avoided? By using a custom GitHub Action - engineerd/configurator.

Using the custom GitHub Action

So let’s see how to use the custom GitHub Action to achieve the same thing on a Linux worker:

jobs:
  configurator:
    runs-on: ubuntu-latest
    steps:
      - uses: engineerd/[email protected]
        with:
          name: "hb3"
          url: "https://get.helm.sh/helm-v3.0.0-beta.3-linux-amd64.tar.gz"
          pathInArchive: "linux-amd64/helm"
      - name: Testing
        run: |
          hb3 --help

The action has three inputs:

  • the URL of the file to download
  • the name used to invoke the tool
  • if the file is an archive, the path in the archive (relative to the root of the archive) to the target file

The action will automatically determine if the file at the URL is an archive (based on the file extension), and will copy the target file (the tool you wish to configure) to a directory in the path. Just make sure the URL is appropriate for the operating system of the worker.

Here are the logs generated by the action above, on an Ubuntu worker:

Run engineerd/[email protected]
  with:
    name: hb3
    url: https://get.helm.sh/helm-v3.0.0-beta.3-linux-amd64.tar.gz
    pathInArchive: linux-amd64/helm
Downloading tool from https://get.helm.sh/helm-v3.0.0-beta.3-linux-amd64.tar.gz...
/bin/tar xz -C /tmp/tmp/runner/temp -f /home/runner/work/_temp/757998fb-23b7-4e1e-9cf1-dbcb13e6f94e
chmod +x /home/runner/configurator/bin/hb3
##[add-path]/home/runner/configurator/bin

Run hb3 --help
  hb3 --help
  shell: /bin/bash -e {0}
The Kubernetes package manager

Common actions for Helm:

- helm search:    search for charts
- helm fetch:     download a chart to your local directory to view
- helm install:   upload the chart to Kubernetes
- helm list:      list releases of charts

Specifically, the action:

  • downloads the archive from the url provided
  • extracts the archive
  • moves the file from pathInArchive into a directory in path, and renames it to name.

If the target file is not an archive, the action skips the extraction step.

The same action on Windows

Configuring the same tool on Windows is similar - the differences are in how Windows manages executable extensions, and the URL of the archive:

jobs:
  kind:
    runs-on: windows-latest
    steps:
      - uses: engineerd/[email protected]
        with:
          name: "hb3.exe"
          url: "https://get.helm.sh/helm-v3.0.0-beta.3-windows-amd64.zip"
          pathInArchive: "windows-amd64/helm.exe"
      - name: Testing
        run: |
          hb3 --help

By changing the URL and file extensions, we now have a cross platform way of configuring the tool on Windows:

Run engineerd/[email protected]
  with:
    name: hb3.exe
    url: https://get.helm.sh/helm-v3.0.0-beta.3-windows-amd64.zip
    pathInArchive: windows-amd64/helm.exe
Downloading tool from https://get.helm.sh/helm-v3.0.0-beta.3-windows-amd64.zip...
C:\windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoLogo -Sta -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command "$ErrorActionPreference = 'Stop' ; try { Add-Type -AssemblyName System.IO.Compression.FileSystem } catch { } ; [System.IO.Compression.ZipFile]::ExtractToDirectory('D:\a\_temp\566115c6-fc8c-47ec-97bf-26fdd1ab853a', 'C:\Users\RUNNER~1\AppData\Local\Temp\tmp\runner\temp')"
##[add-path]C:\Users\runneradmin\runneradmin\configurator\bin

Run hb3 --help
  hb3 --help
  shell: C:\windows\system32\cmd.exe /D /E:ON /V:OFF /S /C "CALL "{0}""
The Kubernetes package manager

Common actions for Helm:

- helm search:    search for charts
- helm fetch:     download a chart to your local directory to view
- helm install:   upload the chart to Kubernetes
- helm list:      list releases of charts

A complete cross-platform job

Instead of configuring two separate jobs, we can use a build matrix to configure the URL, name, and path in archive according to the operating system the job is running on:

jobs:
  configurator:
    runs-on: ${{ matrix.config.os }}
    strategy:
      matrix:
        config:
        - {os: "ubuntu-latest", url: "https://get.helm.sh/helm-v3.0.0-beta.3-linux-amd64.tar.gz", name: "hb3", pathInArchive: "linux-amd64/helm" }
        - {os: "windows-latest", url: "https://get.helm.sh/helm-v3.0.0-beta.3-windows-amd64.zip", name: "hb3.exe", pathInArchive: "windows-amd64/helm.exe" }
    steps:
      - uses: engineerd/[email protected]
        with:
          name: ${{ matrix.config.name }}
          url: ${{ matrix.config.url }}
          pathInArchive: ${{ matrix.config.pathInArchive }}
      - name: Testing
        run: |
          hb3 --help

Next steps

This is a generic action for downloading tools and adding them to the path. However, keep in mind that:

  • it currently doesn’t work for macOS, and it has been tested for ubuntu-latest and windows-latest
  • complex flags for the unarchive process have not been tested yet
  • it currently only works for statically compiled tools - however, it might also work for adding entire archive directories to the path.

If you are interested in any of the above, feel free to comment on issues or directly contribute to the project, and let me know if you have any issues.