Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(gha): add docker push github action #1116

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions .github/workflows/push-docker-on-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
name: Build and Push Docker Image

on:
release:
types: [published]
Comment on lines +4 to +5
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Building of binaries is kicked off from a tag, rather than release. Should we do that here as well?

name: post-release
on:
push:
tags:
- "v*.*.*"
- "!varcon*"

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can be done, for sure!

workflow_dispatch:
inputs:
release_tag:
description: "Release tag to build and push"
required: true

jobs:
build-and-push:
runs-on: ubuntu-latest
permissions:
contents: read # Allows read access to the repository code
packages: write # Allows pushing images to GitHub Container Registry

steps:
- name: Checkout code
uses: actions/checkout@v4
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe only fetch 1 depth is sufficient.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did not find a way to change it to the way you want. I saw the default fetch-depth is 1


- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Log in to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Determine the release tag
id: get_tag
run: |
if [ "${{ github.event_name }}" == "release" ]; then
echo "tag=${GITHUB_REF#refs/tags/}" >> $GITHUB_OUTPUT
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we do this off of a tag, could we simplify this to

run: echo "TAG=${{ github.ref_name }}" >> $GITHUB_ENV

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'll do that too

else
echo "tag=${{ github.event.inputs.release_tag }}" >> $GITHUB_OUTPUT
fi
- name: Build and push Docker image
uses: docker/build-push-action@v6
with:
context: .
push: true
tags: ghcr.io/${{ github.repository_owner }}/${{ github.event.repository.name }}:${{ steps.get_tag.outputs.tag }}
Comment on lines +42 to +47
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also need to build multi-architecture images alongside the CLI release

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ill do it along the weekend 🍾

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my weekend was not very useful, so found time to push this now

# platforms: linux/arm64/v8,darwin/amd64,linux/amd64,windows/amd64,linux/amd64 # was what I was aiming for
# but locally I only got to these 3 (specifically linux/arm64/v8 but yeah)
platforms: linux/arm64,linux/amd64

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH, I didn't work with on Github action to build multi-arch. But in some pipeline tool that I have used, a question is needn't we to install qemu before building image?’

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense. I can add it, but I am adding the required tools inside the docker, so I hope it "just works".

Comment on lines +48 to +50
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels like a during-development note and I'd like it resolved one way or the other

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My PC is MacOS, so I don't have much wiggle room for testing.

I would give it a go if I had more computers with different platforms.

85 changes: 78 additions & 7 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,11 +1,82 @@
ARG DEBIAN_DIST=bullseye
# syntax=docker/dockerfile:1.10
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a complete rewrite of the dockerfile

Can any of this be broken out into smaller (complete) steps (ie commits) to make this easier to follow along?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I can do that, sorry for the mess. This process is more complex for me so I opted to give one big commit and break it into tiny bits if required (mostly because my work was based inside.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@epage you are correct, but the reason I did this is to allow compiling the code to multiple architectures based on TARGETPLATFORM arg (filled by the --platform flag in docker build)

# go to https://hub.docker.com/r/docker/dockerfile to see the latest version of the syntax

FROM rust:${DEBIAN_DIST} as builder
# Stage 1: Build the typos binary
FROM rust:1.81.0-slim-bookworm AS builder

# Install musl-tools for static linking
RUN apt-get update && \
apt-get install -y --no-install-recommends \
liblz4-tool \
musl-tools \
xz-utils \
&& \
rm -rf /var/lib/apt/lists/*

# some targets were not used in the end because rust package is not working with them
# x86_64-pc-windows-msvc \

RUN rustup target add \
aarch64-apple-darwin \
aarch64-unknown-linux-musl \
aarch64-unknown-linux-musl \
x86_64-apple-darwin \
x86_64-unknown-linux-musl \
&& :

# Set the working directory
WORKDIR /usr/src/typos

# Copy the source code into the container
COPY . .
RUN cargo install --path ./crates/typos-cli

FROM debian:${DEBIAN_DIST}-slim
COPY --from=builder /usr/local/cargo/bin/typos /usr/local/bin/typos
ENTRYPOINT ["typos"]
CMD ["--help"]
# Set build arguments
ARG TARGETPLATFORM
ARG BIN_NAME=typos

# Determine the Rust target based on the platform
# fingers crossed this build will just work
# in case I need more platforms - https://github.com/containerd/containerd/blob/90cd777a6c8c92c105625ba086e2e67a0c32d7ed/platforms/platforms.go#L88-L94
# elif [ "${TARGETPLATFORM}" = "windows/amd64" ]; then \
# RUST_TARGET="x86_64-pc-windows-msvc"; \
RUN --mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=/usr/src/typos/target \
set -xeEu \
&& \
ARM_PLATFORMS='linux/arm/v6 linux/arm/v7 linux/arm64/v8 linux/arm64' \
&& \
if [ "${TARGETPLATFORM}" = "darwin/arm64" ]; then \
RUST_TARGET="aarch64-apple-darwin"; \
elif printf '%s\n' ${ARM_PLATFORMS} | grep -Fxq "${TARGETPLATFORM}" ; then \
RUST_TARGET="aarch64-unknown-linux-musl"; \
elif [ "${TARGETPLATFORM}" = "darwin/amd64" ]; then \
RUST_TARGET="x86_64-apple-darwin"; \
elif [ "${TARGETPLATFORM}" = "linux/amd64" ]; then \
RUST_TARGET="x86_64-unknown-linux-musl"; \
else \
echo "Unsupported TARGETPLATFORM: ${TARGETPLATFORM}"; \
exit 1; \
fi \
&& \
echo "Building for ${RUST_TARGET}" \
&& \
cargo build \
--release \
--verbose \
--target ${RUST_TARGET} \
&& \
cp target/${RUST_TARGET}/release/${BIN_NAME} /usr/src/${BIN_NAME}/${BIN_NAME}

# Stage 2: Create the final image
FROM scratch

# Set build arguments
ARG BIN_NAME=typos

# Copy the statically linked binary from the builder stage
COPY --from=builder /usr/src/typos/${BIN_NAME} /${BIN_NAME}

# Set the entrypoint to the typos binary
# This was done to make the default run not scan the whole container for typos
WORKDIR /workdir
georgettica marked this conversation as resolved.
Show resolved Hide resolved
ENTRYPOINT ["/typos"]
219 changes: 219 additions & 0 deletions docs/build_and_push_docker_image_workflow_documentation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,219 @@
# Build and Push Docker Image Workflow Documentation
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I said that I wanted it documented, I meant a write up in the issue or PR so I could understand the expectations behind this. Unsure how much of this will be useful in a living document. If there is, it likely more belongs in CONTRIBUTING.md

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, I will move it there, as moving documentation is easier than rewriting.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the understanding from #1116 (comment), I don't think we need to keep this documentation.


This document provides an overview and detailed explanation of the GitHub Actions workflow that automates the building and pushing of Docker images to the GitHub Container Registry (GHCR).

## Overview

The **Build and Push Docker Image** workflow is designed to:

- **Automate Docker Image Builds**: Whenever a new release is published or the workflow is manually triggered, the workflow builds a Docker image from your repository's code.
- **Push to GitHub Container Registry**: The built Docker image is then pushed to GHCR, tagged appropriately for easy versioning and retrieval.
- **Support Multiple Platforms**: The workflow is set up to build images for multiple platforms, ensuring broad compatibility.

## Workflow Triggers

The workflow is triggered in two scenarios:

1. **On Release Published**: Automatically runs when a new release is published in the repository.
2. **Manual Trigger**: Can be manually triggered via the GitHub Actions tab, allowing you to specify a custom release tag.

## Workflow Details

### Name

- **Workflow Name**: `Build and Push Docker Image`

### Trigger Events

```yaml
on:
release:
types: [published]
workflow_dispatch:
inputs:
release_tag:
description: "Release tag to build and push"
required: true
```
- **Release Published**: Listens for the `published` event on releases.
- **Workflow Dispatch**: Allows manual triggering with an input `release_tag`.

### Permissions

```yaml
permissions:
contents: read # Allows read access to the repository code
packages: write # Allows pushing images to GitHub Container Registry
```

- **Contents**: Read access to fetch the repository code.
- **Packages**: Write access to push Docker images to GHCR.

## Job Breakdown

### Job: `build-and-push`

#### Runs On

- **Environment**: `ubuntu-latest`

#### Steps

1. **Checkout Code**

```yaml
- name: Checkout code
uses: actions/checkout@v4
```

- **Description**: Checks out the repository code so that the workflow can access it.

2. **Set Up Docker Buildx**

```yaml
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
```

- **Description**: Sets up Docker Buildx for building multi-platform images.

3. **Log in to GitHub Container Registry**

```yaml
- name: Log in to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
```

- **Description**: Authenticates with GHCR using your GitHub credentials.

4. **Determine the Release Tag**

```yaml
- name: Determine the release tag
id: get_tag
run: |
if [ "${{ github.event_name }}" == "release" ]; then
echo "tag=${GITHUB_REF#refs/tags/}" >> $GITHUB_OUTPUT
else
echo "tag=${{ github.event.inputs.release_tag }}" >> $GITHUB_OUTPUT
fi
```

- **Description**: Determines the Docker image tag:
- If triggered by a release, uses the release tag.
- If manually triggered, uses the provided `release_tag` input.

5. **Build and Push Docker Image**

```yaml
- name: Build and push Docker image
uses: docker/build-push-action@v6
with:
context: .
push: true
tags: ghcr.io/${{ github.repository_owner }}/${{ github.event.repository.name }}:${{ steps.get_tag.outputs.tag }}
platforms: linux/arm64,linux/amd64
```

- **Description**: Builds and pushes the Docker image to GHCR for the specified platforms.

## Platforms

The workflow builds Docker images for the following platforms:

- **linux/arm64**
- **linux/amd64**

**Note**: Initially, the aim was to build for additional platforms such as `darwin/amd64` and `windows/amd64`, but due to local limitations, the workflow currently builds for the platforms listed above.

## Requirements and Prerequisites

To effectively use this workflow, ensure the following:

- **Dockerfile**: A valid `Dockerfile` must be present at the root of your repository.
- **GitHub Permissions**: The `GITHUB_TOKEN` provided by GitHub Actions must have the necessary permissions (default settings typically suffice).
- **Understanding of Docker and GHCR**: Basic knowledge of Docker image building and pushing to GHCR will be beneficial.
Comment on lines +139 to +140
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was the core of what I was wanting to understand.

If I get the gist of it, instead of us pushing to docker hub, we are pushing to github's container registry which allows us to just use GH_TOKEN without having to setup any count, register secrets, and/or do any manual per-release action.

Is that right?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, that is the gist.

I think GH_TOKEN is if you run it manually, so most times the action won't need even permissions to run.


## How to Use the Workflow

### Triggering the Workflow on Release

1. **Create a New Release**:

- Navigate to the "Releases" section of your repository.
- Click on "Draft a new release".
- Fill in the release details and publish it.

2. **Workflow Execution**:

- Upon publishing, the workflow automatically triggers.
- It will build the Docker image using the release tag and push it to GHCR.

### Manually Triggering the Workflow

1. **Navigate to GitHub Actions**:

- Go to the "Actions" tab in your repository.

2. **Select the Workflow**:

- Find the **Build and Push Docker Image** workflow from the list.

3. **Run the Workflow**:

- Click on the "Run workflow" button.
- Provide the required `release_tag` input when prompted.
- Confirm to start the workflow.

4. **Workflow Execution**:

- The workflow uses the provided `release_tag` to build and push the Docker image.

## Accessing the Pushed Docker Image

The Docker image will be available at:

```
ghcr.io/<repository_owner>/<repository_name>:<tag>
```

Replace:

- `<repository_owner>`: Your GitHub username or organization name.
- `<repository_name>`: The name of your repository.
- `<tag>`: The release tag used during the build.

## Additional Notes

- **Customizing Platforms**: If you need to build for additional platforms, you can modify the `platforms` parameter in the workflow. Be cautious of compatibility and build environment limitations.
- **Troubleshooting**:

- **Build Failures**: Check the workflow logs for detailed error messages.
- **Authentication Issues**: Ensure that `GITHUB_TOKEN` has the necessary permissions.
- **Dockerfile Errors**: Verify that your `Dockerfile` is correctly set up for multi-platform builds.

- **Security Considerations**:

- The `GITHUB_TOKEN` is scoped to the repository and should be kept secure.
- Avoid echoing sensitive information in the workflow logs.

## Understanding the Workflow's Impact

By integrating this workflow:

- **Consistency**: Ensures that Docker images are consistently built and tagged with each release.
- **Automation**: Reduces manual efforts in building and pushing Docker images.
- **Version Control**: Leveraging release tags helps in maintaining versions of your Docker images aligned with your codebase.

## Conclusion

This workflow enhances your CI/CD pipeline by automating the Docker image build and push process. It is essential to familiarize yourself with its steps and requirements to leverage its full potential effectively. Should you have any questions or need further assistance, feel free to reach out or consult the GitHub Actions and Docker documentation.

---

*This documentation aims to provide a comprehensive understanding of the new workflow and the requirements it places on you. Ensure to review and customize any parts of the workflow to suit your specific needs.*