[ notes to myself ]

Build and sign container images with Gitlab CI

2 min read

Gitlab has built-in support for keyless signing with Sigstore - if you’re using a hosted version of Gitlab. If you’re self-hosting, this requires a bit more effort to setup.

Here’s how I set up a Gitlab CI workflow to build an image with kaniko, push it to a self-hosted Harbor registry, and sign the digest with Cosign and Vault.

Vault setup

Enable transit secrets engine, create the cosign key, and create a policy for access:

vault secrets enable transit
vault write -f transit/keys/cosign type=ecdsa-p256
vault policy write cosign - '
  path "transit/keys/cosign" {
    capabilities = ["read"]
  }
  path "transit/hmac/cosign/*" {
    capabilities = ["update"]
  }
  path "transit/sign/cosign/*" {
    capabilities = ["update"]
  }
  path "transit/verify/cosign" {
    capabilities = ["create"]
  }
  path "transit/verify/cosign/*" {
    capabilities = ["update"]
  }
'

Set up JWT auth:

vault auth enable jwt
vault write auth/jwt/role/cosign - '
  {
    "bound_audiences": [
      "https://gitlab.example.com"
    ],
    "bound_claims": {
      "iss": [
        "gitlab.example.com",
        "https:/gitlab.example.com"
      ],
      "project_path": [
        "container-images/*"
      ]
    },
    "bound_claims_type": "glob",
    "policies": [
      "cosign"
    ],
    "role_type": "jwt",
    "user_claim": "user_email"
  }
'
vault write auth/jwt/config \
  jwks_url="https://gitlab.example.com/oauth/discovery/keys"

kaniko-cosign image

To be able to handle interacting with Vault, building the image, and signing the digest in a single step, I’m using a customized image which contains all the needed tools. I’ll manually build, sign, and push it to harbor.example.com/library/kaniko-cosign:latest.

Note: I stash the cosign and vault binaries inside of /kaniko/ to make sure they don’t lost during multistage builds.

# Dockerfile
ARG ALPINE_VERSION=3.20
ARG COSIGN_VERSION=2.4.1
ARG VAULT_VERSION=1.18.3
ARG KANIKO_VERSION=1.23.2
 
FROM bitnami/kaniko:${KANIKO_VERSION} AS kaniko
FROM bitnami/cosign:${COSIGN_VERSION} AS cosign
FROM hashicorp/vault:${VAULT_VERSION} AS vault
FROM alpine:${ALPINE_VERSION}
COPY --from=kaniko /kaniko /kaniko
COPY --from=cosign /opt/bitnami/cosign/bin/cosign /kaniko/cosign
COPY --from=vault /bin/vault /kaniko/vault
 
ENV PATH=$PATH:/kaniko \
  DOCKER_CONFIG=/kaniko/.docker
 
WORKDIR /workspace
CMD ["/bin/sh"]

Gitlab pipeline

Note: I use --ignore-path to tell kaniko not to delete /bin/busybox so I can continue to use those tools.

# .gitlab-ci.yml
 
stages:
- build
 
container-build:
  stage: build
  image: harbor.example.com/library/kaniko-cosign:latest
  id_tokens:
    VAULT_ID_TOKEN:
      aud: https://gitlab.example.com
  variables:
    VAULT_ADDR: https://vault.example.com
    VAULT_JWT_ROLE: cosign
    COSIGN_KEY: hashivault://cosign
  before_script:
  - export VAULT_TOKEN="$(vault write --field=token auth/jwt/login role=$VAULT_JWT_ROLE jwt=$VAULT_ID_TOKEN)"
  - echo "{\"auths\":{\"${HARBOR_HOST}\":{\"auth\":\"$(echo -n ${HARBOR_USERNAME}:${HARBOR_PASSWORD} | base64 -w 0)\"}}}" > /kaniko/.docker/config.json
  script:
  - /kaniko/executor
    --context "${CI_PROJECT_DIR}"
    --dockerfile "${CI_PROJECT_DIR}/Dockerfile"
    --destination "${HARBOR_HOST}/${HARBOR_PROJECT}/${CI_PROJECT_NAME}:${CI_COMMIT_TAG}"
    --digest-file "./digest"
    --ignore-path=/bin/busybox
  - /kaniko/cosign sign "${HARBOR_HOST}/${HARBOR_PROJECT}/${CI_PROJECT_NAME}@$(busybox cat digest)" --tlog-upload=false

See also:

Refs:

Backlinks