--output type=oci,tar=false produces inconsistent (and invalid 😬) annotations

[tianon]

In short, --output type=oci and --output type=oci,tar=false produce inconsistent results with respect to annotations, and the org.opencontainers.image.ref.name value in the latter case is invalid ("SHOULD", yeah, fair) according to the OCI spec (comma-separated multiple values):

Reproducer:

$ echo 'FROM scratch' > Dockerfile

$ docker buildx create --name master --driver-opt image=moby/buildkit:master --bootstrap
#1 [internal] booting buildkit
#1 pulling image moby/buildkit:master
#1 pulling image moby/buildkit:master 0.4s done
#1 creating container buildx_buildkit_master0
#1 creating container buildx_buildkit_master0 0.4s done
#1 DONE 0.9s
master

$ docker buildx inspect master | head -n14
Name:          master
Driver:        docker-container
Last Activity: 2024-12-06 00:57:48 +0000 UTC

Nodes:
Name:                  master0
Endpoint:              unix:///var/run/docker.sock
Driver Options:        image="moby/buildkit:master"
Status:                running
BuildKit daemon flags: --allow-insecure-entitlement=network.host
BuildKit version:      38a47db
Platforms:             linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/arm64/v8, linux/riscv64, linux/ppc64, linux/ppc64le, linux/s390x, linux/386, linux/loong64, linux/arm/v7, linux/arm/v6
Labels:
 org.mobyproject.buildkit.worker.executor:         oci

$ docker buildx build --builder master --output type=oci,tar=false,dest=dir --tag example.com/foo --tag example.com/bar .
#0 building with "master" instance using docker-container driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 50B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s

#3 exporting to oci image format
#3 exporting layers done
#3 exporting manifest sha256:6cddc7c291e3d05059404a670e53a4e87166bd3f22ea50b9ae60f9d0c4423775 0.0s done
#3 exporting config sha256:471a1b8817eefb6569017c1a76f288e0d4e5c8476eb199485c469d0b033168bf done
#3 DONE 0.0s

$ docker buildx build --builder master --output type=oci,dest=tar.tar --tag example.com/foo --tag example.com/bar .
#0 building with "master" instance using docker-container driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 50B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s

#3 exporting to oci image format
#3 exporting layers done
#3 exporting manifest sha256:6cddc7c291e3d05059404a670e53a4e87166bd3f22ea50b9ae60f9d0c4423775 done
#3 exporting config sha256:471a1b8817eefb6569017c1a76f288e0d4e5c8476eb199485c469d0b033168bf done
#3 sending tarball done
#3 DONE 0.0s

$ diff -u \
	<(tar -xOf tar.tar index.json | jq .) \
	<(jq . dir/index.json)
--- /dev/fd/63	2024-12-05 16:57:49.771641944 -0800
+++ /dev/fd/62	2024-12-05 16:57:49.771641944 -0800
@@ -7,23 +7,8 @@
       "digest": "sha256:6cddc7c291e3d05059404a670e53a4e87166bd3f22ea50b9ae60f9d0c4423775",
       "size": 288,
       "annotations": {
-        "io.containerd.image.name": "example.com/foo:latest",
         "org.opencontainers.image.created": "2024-12-06T00:57:49Z",
-        "org.opencontainers.image.ref.name": "latest"
-      },
-      "platform": {
-        "architecture": "amd64",
-        "os": "linux"
-      }
-    },
-    {
-      "mediaType": "application/vnd.oci.image.manifest.v1+json",
-      "digest": "sha256:6cddc7c291e3d05059404a670e53a4e87166bd3f22ea50b9ae60f9d0c4423775",
-      "size": 288,
-      "annotations": {
-        "io.containerd.image.name": "example.com/bar:latest",
-        "org.opencontainers.image.created": "2024-12-06T00:57:49Z",
-        "org.opencontainers.image.ref.name": "latest"
+        "org.opencontainers.image.ref.name": "example.com/foo:latest,example.com/bar:latest"
       },
       "platform": {
         "architecture": "amd64",

Related issues:

• --output type=docker,tar=true and tar=false create different files #5559
• In the OCI tarball, the RepoTags list is null, is that expected? #1949
• OCI export should only put the tag into org.opencontainers.image.ref.name in index.json  #4615 (comment)

[thaJeztah]

("SHOULD", yeah, fair) according to the OCI spec (comma-separated multiple values):

I was trying to read that part, but I didn't see "comma-separated" there; do you mean it's current doing comma-separated (but shouldn't) or does the OCI spec define that it should use comma-separated values here?

(if it's the latter; I'm wondering if there's some upper-bound, as I could imaging that list could be .... really long (multiplied by long references)

[tianon]

Sorry, it should not use commas (ie, the tar output is correct and valid with duplicated "manifests" entries for multiple tags):

• org.opencontainers.image.ref.name Name of the reference for a target (string).

  • SHOULD only be considered valid when on descriptors on index.json within image layout.

  • Character set of the value SHOULD conform to alphanum of A-Za-z0-9 and separator set of -._:@/+

  • A valid reference matches the following grammar:

    ref       ::= component ("/" component)*
    component ::= alphanum (separator alphanum)*
    alphanum  ::= [A-Za-z0-9]+
    separator ::= [-._:@+] | "--"

[thaJeztah]

Ah, thanks! Sorry I was perhaps a bit slow on grasping on this one ❤