We write a lot of Go at pebl, and most of our deployments are done using containers. In the beginning our simplistic Dockerbuild setup was causing super slow builds. Over time we've been able to get super fast docker builds with just a few changes!
TLDR
Be mindful of layer ordering for unnecessary invalidation
Utilize Go's built-in build cache
Instead of manually cleaning up the image, utilize multistage docker builds
Layers
Pay attention to layers! In general, you can speed up docker builds by placing operations on files that don't change as often (such as go.mod) at the beginning of the Dockerfile.
FROM golang
WORKDIR /build
COPY go.mod go.sum .
RUN go mod download
# ... more build steps ...
Since your dependencies don't change as frequently as code, this will allow docker to reuse this layer for subsequent docker builds. The same idea applies to other languages with package management, so you want to handle your package.json, requirements.txt, Gemfile in a similar fashion.
Use Go's Build Cache
Unlike the old days when compilers were primitive and developers had to manage caching their builds manually, newer languages come with compilers that handle most of this for you.
While this simplifies things when building on your host machine, it can be tricky to utilize during docker builds. Each build is isolated by design, meaning that the compiler can't access the files it generated during the last build.
To fix this, you can specify a special cache mount in the RUN step that contains the docker build:
FROM golang
WORKDIR /build
COPY go.mod go.sum .
RUN go mod download
COPY . .
RUN --mount=type=cache,target=/root/.cache/go-build \
go build -o foo .
# ... more build steps ...
Be mindful of your own setup here as the cache location can be different. You can also configure Go to use your desired cache location by setting the GOCACHE variable.
Minimal Docker Images
At this point your instinct might be to add cleanup steps after the go build. But the downside is that this adds extra time to your build stage, and it can be quite tricky to ensure the clean up is kept up to date as the build changes over time.
Instead we prefer to use multistage docker builds. The other benefit is that you can choose a super slim base image, keeping the final docker image minimal.
FROM golang as build
WORKDIR /build
COPY go.mod go.sum .
RUN go mod download
COPY . .
RUN --mount=type=cache,target=/root/.cache/go-build \
go build -o foo .
FROM alpine
COPY --from=build /build/foo /foo
CMD ["/foo"]
You can even look into the scratch image, which is a specialized docker image with literally nothing in its file system, not even /bin/sh! But be warned that debugging such an image can be quite tricky, as you won't be able to get even a shell to poke around.