@rootulp , can I take this one?

go for it @tskoyo

Hi team, here's the investigation:

• Docker image ghcr.io/celestiaorg/celestia-app:v3.0.0-mocha only supports the amd64 platform.
• When running this command: docker buildx imagetools inspect ghcr.io/celestiaorg/celestia-app:v3.0.0-mocha, I saw that it doesn't have the manifests for other platforms, but only for amd64 because it was likely built initially from an amd64 machine.

Name:      ghcr.io/celestiaorg/celestia-app:v3.0.0-mocha
MediaType: application/vnd.docker.distribution.manifest.v2+json
Digest:    sha256:75edb998c45094161a80f79a97b1662cb9708ff6ef5878dc8891970d6204be1b

When supporting multiple platforms, the output should be similar to the output from this example (see this part -> docker buildx imagetools inspect adamparco/demo:latest)

This limits users on ARM-based systems and results in either poor performance (via QEMU emulation) or inability to use the image.

Reason

The image was likely built using the default single-platform docker build process, which only targets the architecture of the machine that built this image (e.g., amd64).

To support both amd64 and arm64, the following steps should be implemented:

1. Enable multi-platform build with docker buildx (Buildx allows the creation of images for multiple platforms)

2. Modify the Build Command - Update the Docker build process to target both amd64 and arm64 platforms:

docker buildx build \
  --platform linux/amd64,linux/arm64 \
  -t ghcr.io/celestiaorg/celestia-app:v3.0.0-mocha \
  --push .

3. Verify multi-architecture support:

docker manifest inspect ghcr.io/celestiaorg/celestia-app:v3.0.0-mocha

Risks

• There is no risk of breaking the current amd64 image implementation, as:
  • The amd64 build process remains unchanged.
  • Docker Buildx creates independent images for each platform.
• However, testing should ensure there are no architecture-specific bugs or library issues for ARM64.

Read more about buildx and its purpose here

Thanks for the detailed write-up @tskoyo ! Our Docker images is published via this workflow: https://github.com/celestiaorg/celestia-app/actions/workflows/docker-build-publish.yml which seems defined via

Based on this line I think it should build a Docker image that supports arm64 and amd64. Same with this log. Do you have any idea why it's not building an image that supports both of those platforms?

@rootulp i believe the issue is here in these last 2 steps starting at line 283 https://github.com/celestiaorg/.github/blob/45533fc35847a8568deb0d4cae1851a2872f3651/.github/workflows/reusable_dockerfile_pipeline.yml#L283

The main issue is we build in two separate steps, but we re-use tags, so the manifest is overwritten by the amd64 build. Options:

• a quick fix would be to NOT try and separate these and over complicate trying to ship an amd64 image fast, and just allow the platforms arg to actually be read in as an input to the workflow by the caller, then you'll have a single manifest with all platform builds. This isn't as fun as all the work that was done

• add platform suffix to the tags (ewwwww) but this would ruin the experience of JUST pulling a docker image and it knowing your platform 👎

• continue the complexity and disable the push: true argment, thus building each image then constructing a manifest after build, but then, you lose the "speed" gain for which this was separated for originally, so....easier to just collapse to 1 step