Back

Build multi arch docker image using Tekton

multi arch builds in tekton using docker buildx

There are three different strategies to build multi-platform images that are supported by Buildx and Dockerfiles:

  • Using the QEMU emulation support in the kernel
  • Building on multiple native nodes using the same builder instance
  • Using a stage in Dockerfile to cross-compile to different architectures

I’ll focus on the first option, cross building with emulation.

docker buildx build --platform linux/amd64,linux/arm64 .

Setup

This differs from platform to platform but one thing we all have in common is pipelines, so I’ve constructed a basic buildx setup for TektonCD task.

Pipeline

Below is a modified task of docker-build

What’s modified

  • Enable DOCKER_BUILDKIT and DOCKER_CLI_EXPERIMENTAL
  • Install docker-buildx plugin
  • Run the latest docker/binfmt tag to use its qemu parts
  • Docker login (using secret for values)
  • Docker build with docker buildx build --platform

Create a secret holding docker username and token

kubectl create secret generic docker-token \
    --from-literal=username="${CONTAINER_REGISTRY_USER}" \
    --from-literal=password="${CONTAINER_REGISTRY_PASSWORD}"
apiVersion: tekton.dev/v1beta1
kind: ClusterTask
metadata:
  name: docker-buildx
  labels:
    app.kubernetes.io/version: "0.1"
  annotations:
    tekton.dev/pipelines.minVersion: "0.12.1"
    tekton.dev/tags: docker, build-image, push-image, dind, buildx, multi-arch
    tekton.dev/displayName: docker-buildx
spec:
  description: >-
    This task will build and push an image using docker buildx.
    The task will build an out image out of a Dockerfile.
    This image will be pushed to an image registry.
    The image will be built and pushed using a dind sidecar over TCP+TLS.    
  params:
  - name: image
    description: Reference of the image docker will produce.
  - name: builder_image
    description: The location of the docker builder image.
    default: docker.io/library/docker:19.03.14
  - name: dockerfile
    description: Path to the Dockerfile to build.
    default: ./Dockerfile
  - name: context
    description: Path to the directory to use as context.
    default: .
  - name: build_extra_args
    description: Extra parameters passed for the build command when building images.
    default: "--platform linux/amd64,linux/arm/v7 --no-cache"
  - name: push_extra_args
    description: Extra parameters passed for the push command when pushing images.
    default: "--push"
  - name: insecure_registry
    description: Allows the user to push to an insecure registry that has been specified
    default: ""
  - name: docker-token-secret
    type: string
    description: name of the secret holding the docker-token
    default: docker-token
  workspaces:
  - name: source
  results:
  - name: IMAGE_DIGEST
    description: Digest of the image just built.
  steps:
  - name: buildx-build-push
    image: $(params.builder_image)
    env:
    # Connect to the sidecar over TCP, with TLS.
    - name: DOCKER_HOST
      value: tcp://localhost:2376
    # Verify TLS.
    - name: DOCKER_TLS_VERIFY
      value: '1'
    # Use the certs generated by the sidecar daemon.
    - name: DOCKER_CERT_PATH
      value: /certs/client
    - name: DOCKERHUB_USER
      valueFrom:
        secretKeyRef:
          name: $(params.docker-token-secret)
          key: username
    - name: DOCKERHUB_PASS
      valueFrom:
        secretKeyRef:
          name: $(params.docker-token-secret)
          key: password
    workingDir: $(workspaces.source.path)
    script: |
      # install depends
      apk add curl jq

      # enable experimental buildx features
      export DOCKER_BUILDKIT=1
      export DOCKER_CLI_EXPERIMENTAL=enabled

      # Download latest buildx bin from github
      mkdir -p ~/.docker/cli-plugins/
      BUILDX_LATEST_BIN_URI=$(curl -s -L https://github.com/docker/buildx/releases/latest | grep 'linux-amd64' | grep 'href' | sed 's/.*href="/https:\/\/github.com/g; s/amd64".*/amd64/g')
      curl -s -L ${BUILDX_LATEST_BIN_URI} -o ~/.docker/cli-plugins/docker-buildx
      chmod a+x ~/.docker/cli-plugins/docker-buildx

      # Get and run the latest docker/binfmt tag to use its qemu parts
      BINFMT_IMAGE_TAG=$(curl -s https://registry.hub.docker.com/v2/repositories/docker/binfmt/tags | jq '.results | sort_by(.last_updated)[-1].name' -r)
      docker run --rm --privileged docker/binfmt:${BINFMT_IMAGE_TAG}

      docker context create tls-environment
      # create the multibuilder
      docker buildx create --name multibuilder --use tls-environment
      docker buildx use multibuilder

      # login to a registry
      echo ${DOCKERHUB_PASS} | docker login -u ${DOCKERHUB_USER} --password-stdin

      # build the containers and push them to the registry then display the images
      docker buildx build $(params.build_extra_args) \
          -f $(params.dockerfile) -t $(params.image) $(params.context) $(params.push_extra_args)      

    volumeMounts:
      - mountPath: /certs/client
        name: dind-certs
  sidecars:
  - image: docker:19.03.14-dind
    name: server
    args:
      - --storage-driver=vfs
      - --userland-proxy=false
      - --debug
    securityContext:
      privileged: true
    env:
    # Write generated certs to the path shared with the client.
    - name: DOCKER_TLS_CERTDIR
      value: /certs
    volumeMounts:
    - mountPath: /certs/client
      name: dind-certs
    # Wait for the dind daemon to generate the certs it will share with the
    # client.
    readinessProbe:
      periodSeconds: 1
      exec:
        command: ['ls', '/certs/client/ca.pem']
  volumes:
  - name: dind-certs
    emptyDir: {}

Make sure you have a base image that supports multiarch like alpine or one of these base images

Built with Hugo
Theme Stack designed by Jimmy