Prerelease thoughts Part II: CI

Continued from Prerelease thoughts Part I: llvm

Topics

• llvm
• Github and CI (this)
• Packages
• Project

Github and CI

As mulle-objc in full flight is a swarm of dozens of small projects, doing CI outside of my development setup has been useful for increasing the overall quality of the releases. Complaints are rare, that something doesn’t build out of the box. I attribute that largely to the CI.

On the negative side, the CI environment is often lagging the actual deployment platforms, which in theory could slow down the release as well (if you want to stay “green”). The maintenance of a consistent CI over dozens of project can be quite time consuming though. Especially, if each project has to be configured individually.

Migration from TravisCI to Github

I put a lot of effort into my TravisCI scripts, which I now had to abandon as TravisCI became pay-to-play just recently. I probably spent a week designing a CI for Github Actions, which allows me, just like on TravisCI, to build the prerelease branch differently than master/release branches using pre-release versions of mulle-sde and mulle-clang.

The solution I have come up with is nicely modular. Each repository gets this uniform single workflow file added (via a mulle-sde extension):

name: CI

on: [push]

env:
  BUILD_TYPE: release
  OTHER_PROJECTS: "mulle-objc/mulle-objc-developer;
mulle-c/mulle-c-developer;"

jobs:
  build:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ ubuntu-latest]

    steps:
    - name: Set MULLE_HOSTNAME
      run: echo "MULLE_HOSTNAME=ci-${GITHUB_REF##*/}" >> $GITHUB_ENV

    - uses: actions/checkout@v2

    - uses: mulle-sde/github-ci@v1

    - uses: mulle-objc/github-ci@v1

    - name: Add to path
      run:  echo "$HOME/bin" >> $GITHUB_PATH

    - name: Dump Environment
      run:  env | sort

    - name: Mulle-SDE Fetch
      run: mulle-sde fetch

    - name: Mulle-SDE Craft
      run: mulle-sde craft --${BUILD_TYPE:-release}

    - name: Mulle-SDE Test
      run: |
        [ ! -d test ] || mulle-sde test

I hope that I won’t have to touch this in the future, as all the tricky aspects are imported. There is a hidden maintenance cost per library though as each library has its own environment-host-ci-prerelease.sh file, which is used to pull pre-release repositories instead of release archives. Whenever a new library is added to the chain of dependencies, this file needs to be updated.

As this file is very similar across all repositories and superflous entries don’t hurt, it would be useful to think up a way to maintain it outside of each repository. Actually it’s maintained twice, once for the tests and once for the project itself.

act doesn’t work yet but it could be an interesting way to run Github Actions locally.

A Monorepo ?

How could a Monorepo help ? Here is the list of tasks, which a Monorepo would significantly reduce the amount of work that has to be expended, over a collection of n repositories:

• maintainance of version numbers and release notes
• maintainance of mulle-sde dependency configurations
• maintainance of github pages
• maintainance of environment-host-ci-prerelease.sh contents

Here’s what the Monorepo would hinder

• discoverablility on GitHub for individual projects such as mulle-allocator
• having dependencies only of parts of MulleFoundation
• clashes with the modular approach of mulle-objc

I think a Monorepo can be useful for a collection of tools/apps/executables that share a common set of dependencies. But for a collection of libraries that is well structured, it doesn’t fit.

Consequences

Wait and see.

I assume the migration to Github Actions is only a one time cost. If it turns out that I have to do a significant amount of work on CI again next release, I very well might abandon Github Actions as well and do the final integration tests within docker containers locally.

Continue to Prerelease thoughts Part III: Packages