ci-kernels

A collection of kernels used for CI builds, focusing on testing user space tools for networking and eBPF.

The output of this project are vimto-compatible OCI images pushed to ghcr.io/cilium/ci-kernels.

Supported Versions

Here's an up-to-date list of supported Linux kernel images available in the registry. Older images may be cleaned up at any time.

The latest images in each channel receive a floating tag for those not particular about targeting specific versions.

Version	Channel	Image	Tag
7.3-rc4	mainline	ghcr.io/cilium/ci-kernels:7.3-rc4	mainline
7.2.7	stable	ghcr.io/cilium/ci-kernels:7.2.7	stable
6.18.53	longterm	ghcr.io/cilium/ci-kernels:6.18.53	longterm
6.12.111	longterm	ghcr.io/cilium/ci-kernels:6.12.111
6.6.157	longterm	ghcr.io/cilium/ci-kernels:6.6.157
6.1.188	longterm	ghcr.io/cilium/ci-kernels:6.1.188
5.15.221	longterm	ghcr.io/cilium/ci-kernels:5.15.221
5.10.270	longterm	ghcr.io/cilium/ci-kernels:5.10.270

Images are built for linux/amd64 and linux/arm64 targets.

Image Variants

Each version is published in several variants, distinguished by tag suffix:

Suffix	Contents
-debug	unstripped vmlinux, gdb scripts and DWARF-referenced sources
-selftests	the kernel's bpf selftests, only for floating tags (e.g. stable-selftests)
-selftests-debug	debug build of the bpf selftests

Selftests builds break upstream regularly and are best-effort: when they fail to build, the version bump proceeds anyway and the -selftests(-debug) tags keep pointing at the last version that built successfully. They may trail the other tags of their channel by one or more releases.

Running with vimto

vimto is a tool for running interactive programs in microVMs. Running Go tests in a sandboxed kernel using a ci-kernels image is as simple as:

vimto -kernel ghcr.io/cilium/ci-kernels:7.2.7 -- go test . -v

See vimto --help for more information.

Driving a Test Matrix

The matrix action emits the supported versions as a JSON array, for use as a test matrix in other repositories:

jobs:
  matrix:
    runs-on: ubuntu-latest
    outputs:
      matrix: ${{ steps.matrix.outputs.matrix }}
    steps:
    - id: matrix
      uses: cilium/ci-kernels/matrix@v7.3
      with:
        channel: longterm    # optional: mainline, stable, longterm
        latest-only: "true"  # optional: one version per channel

  test:
    needs: matrix
    strategy:
      matrix:
        include: ${{ fromJSON(needs.matrix.outputs.matrix) }}
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v7
    - uses: actions/setup-go@v7
      with:
        go-version: stable
    - run: go run lmb.io/vimto@latest -kernel ghcr.io/cilium/ci-kernels:${{ matrix.version }} -- go test ./...

Every merge to main that changes matrix tags this repository as v<mainline major.minor>.<timestamp>, with floating v<major> and v<major>.<minor> tags following the latest release (see scripts/tag.sh). Pin the tier you're comfortable with: @v7.3 is a good default, and Dependabot picks up rollovers to newer major/minor versions as they appear.

Bumping Kernel Versions

1. ./scripts/update-versions.py
2. Commit and make a PR.

Building Locally

You can approximate CI by running scripts/buildx.sh:

$ ./scripts/buildx.sh 6.1 amd64 vmlinux --tag foo:vmlinux

To inspect the result of a build:

$ ./scripts/buildx.sh 6.1 amd64 build-vmlinux-debug
...
 => => writing image sha256:18d00182c5495376d87dfef5a4363a1b2cbd936af4f893ab437ce006b0f893d4                             0.0s
$ docker run -it sha256:18d00182c5495376d87dfef5a4363a1b2cbd936af4f893ab437ce006b0f893d4
root@1a64a0ade637:/usr/src/linux#

Publishing images manually

CI builds candidate images on pull requests, named after the kernel version plus a hash of the build directory. Merging to main normally just retags the already-tested candidates. If that fails (e.g. for fork PRs, which cannot push candidates), promotion falls back to building on main. To break-glass the process by hand from the merged commit:

$ docker login ghcr.io
$ ./scripts/buildx.sh 6.12.111 amd64,arm64 vmlinux --tag "ghcr.io/cilium/ci-kernels:$(./scripts/candidate-tag.sh 6.12.111)" --push
$ ./scripts/promote.sh 6.12.111

Updating the configuration

The configuration consists of common options in config and platform specific options in config-arm64 and config-x64_64.

To add a new config option:

1. Try adding it to build/config (keep sorted alphabetically)
2. In a checkout of the Linux source code:

   TARGETPLATFORM=linux/arm64 /path/to/build/configure-vmlinux.sh

3. If any symbols are missing you can now run make menuconfig and search for the missing symbols. Figure out which dependencies are missing and add them to the config as well.

Add the config to the arch specific files if it isn't available in general.

Updating the builder

The builder image is still built manually.

1. make builder
2. Test a build via instruction above.
3. make push
4. Add files, commit and make a PR.