[v0.14] dockerfile: mkdir xxxx not a directory - with COPY --link #5070

tetienne-zenchef · 2024-06-21T10:53:51Z

Hi,

we face this issue with v0.14.1 (and 0.14.0), but was OK with v0.13.2.

0.038 runc run failed: unable to start container process: error during container init: mkdir /var/www: not a directory

To reproduce

FROM public.ecr.aws/debian/debian:11-slim@sha256:1fec6f6a4f281dc401bf40553e9ddc5524714020be06c865d42870c549662dcd AS base-production


WORKDIR /var/www
COPY --link readme.md . # whatever the file
RUN echo "Hello world"

❯ docker buildx create --name ko --driver=docker-container --driver-opt image=moby/buildkit:v0.14.1
ko
❯ docker buildx build  --builder=ko .                                                                                                                 
[+] Building 1.2s (10/10) FINISHED                                                                                                                          docker-container:ko
 => [internal] load build definition from Dockerfile                                                                                                                       0.0s
 => => transferring dockerfile: 240B                                                                                                                                       0.0s
 => [internal] load metadata for public.ecr.aws/debian/debian:11-slim@sha256:1fec6f6a4f281dc401bf40553e9ddc5524714020be06c865d42870c549662dcd                              1.2s
 => [internal] load .dockerignore                                                                                                                                          0.0s
 => => transferring context: 201B                                                                                                                                          0.0s
 => [internal] load build context                                                                                                                                          0.0s
 => => transferring context: 30B                                                                                                                                           0.0s
 => [1/4] FROM public.ecr.aws/debian/debian:11-slim@sha256:1fec6f6a4f281dc401bf40553e9ddc5524714020be06c865d42870c549662dcd                                                0.0s
 => => resolve public.ecr.aws/debian/debian:11-slim@sha256:1fec6f6a4f281dc401bf40553e9ddc5524714020be06c865d42870c549662dcd                                                0.0s
 => CACHED [2/4] WORKDIR /var/www                                                                                                                                          0.0s
 => ERROR [4/4] RUN echo "Hello world"                                                                                                                                     0.0s
 => CACHED [3/4] COPY --link readme.md .                                                                                                                                   0.0s
------
 > [4/4] RUN echo "Hello world":
0.030 runc run failed: unable to start container process: error during container init: mkdir /var/www: not a directory
------
WARNING: No output specified with docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
Dockerfile:8
--------------------
   6 |     
   7 |     COPY --link readme.md .
   8 | >>> RUN echo "Hello world"
   9 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c echo \"Hello world\"" did not complete successfully: exit code: 1

View build details: docker-desktop://dashboard/build/ko/ko0/t1r68fqzqgnzp9y4v5xcpmyis

❯ docker buildx create --name ok --driver=docker-container --driver-opt image=moby/buildkit:v0.13.2
ok
❯ docker buildx build --builder=ok .                                                               
[+] Building 6.3s (11/11) FINISHED                                                                                                                          docker-container:ok
 => [internal] booting buildkit                                                                                                                                            2.3s
 => => pulling image moby/buildkit:v0.13.2                                                                                                                                 1.6s
 => => creating container buildx_buildkit_ok0                                                                                                                              0.7s
 => [internal] load build definition from Dockerfile                                                                                                                       0.0s
 => => transferring dockerfile: 240B                                                                                                                                       0.0s
 => [internal] load metadata for public.ecr.aws/debian/debian:11-slim@sha256:1fec6f6a4f281dc401bf40553e9ddc5524714020be06c865d42870c549662dcd                              1.5s
 => [internal] load .dockerignore                                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                                            0.0s
 => [internal] load build context                                                                                                                                          0.0s
 => => transferring context: 139B                                                                                                                                          0.0s
 => [1/4] FROM public.ecr.aws/debian/debian:11-slim@sha256:1fec6f6a4f281dc401bf40553e9ddc5524714020be06c865d42870c549662dcd                                                1.9s
 => => resolve public.ecr.aws/debian/debian:11-slim@sha256:1fec6f6a4f281dc401bf40553e9ddc5524714020be06c865d42870c549662dcd                                                0.0s
 => => sha256:c982e8b303792a129014d0d049810d7251866836c1f20cd11d8dc5d6690f035a 28.74MB / 28.74MB                                                                           1.5s
 => => extracting sha256:c982e8b303792a129014d0d049810d7251866836c1f20cd11d8dc5d6690f035a                                                                                  0.4s
 => [2/4] WORKDIR /var/www                                                                                                                                                 0.2s
 => [3/4] COPY --link readme.md .                                                                                                                                          0.0s
 => => merging                                                                                                                                                             0.0s
 => [4/4] RUN echo "Hello world"                                                                                                                                           0.0s
WARNING: No output specified with docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load

View build details: docker-desktop://dashboard/build/ok/ok0/k9dy6ewmfufpkwwbqx0otnozv

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview

When doing the copy without --link, build is OK.

❯ docker buildx build --builder=ko --no-cache . 
[+] Building 0.1s (8/8) FINISHED                                                                                                                            docker-container:ko
 => [internal] load build definition from Dockerfile                                                                                                                       0.0s
 => => transferring dockerfile: 233B                                                                                                                                       0.0s
 => [internal] load metadata for public.ecr.aws/debian/debian:11-slim@sha256:1fec6f6a4f281dc401bf40553e9ddc5524714020be06c865d42870c549662dcd                              0.0s
 => [internal] load .dockerignore                                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                                            0.0s
 => [1/4] FROM public.ecr.aws/debian/debian:11-slim@sha256:1fec6f6a4f281dc401bf40553e9ddc5524714020be06c865d42870c549662dcd                                                0.0s
 => => resolve public.ecr.aws/debian/debian:11-slim@sha256:1fec6f6a4f281dc401bf40553e9ddc5524714020be06c865d42870c549662dcd                                                0.0s
 => [internal] load build context                                                                                                                                          0.0s
 => => transferring context: 30B                                                                                                                                           0.0s
 => CACHED [2/4] WORKDIR /var/www                                                                                                                                          0.0s
 => [3/4] COPY readme.md .                                                                                                                                                 0.0s
 => [4/4] RUN echo "Hello world"                                                                                                                                           0.0s
WARNING: No output specified with docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load

View build details: docker-desktop://dashboard/build/ko/ko0/mcs9oact4tuj55bq7i22cp1oj

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview

The text was updated successfully, but these errors were encountered:

ufou · 2024-06-21T12:16:13Z

Yeah, we ran into this same issue, I wonder if this new buildkit image is more strict on COPY command syntax? It seems a potential fix is to ensure that the COPY command ends in a trailing slash

tonistiigi · 2024-06-21T15:26:09Z

Indeed, changing to COPY --link readme ./ seems to be the workaround.

018155fa68e3ab267588f4f1735532bcfb7f02aa is the first bad commit
commit 018155fa68e3ab267588f4f1735532bcfb7f02aa
Author: Anthony Nandaa <[email protected]>
Date:   Fri Apr 5 14:32:58 2024 +0300

    fix: use unix path separator since path already normalized

    In the case for Windows, this line at
    frontend/dockerfile/dockerfile2llb/convert.go#L1142
    ```go
    dest += string(filepath.Separator)
    ```
    was adding the `\\` to a path that is already normalized
    to unix-format, hence ending up with dest paths like
    `/\\` for `C:\\` and `/test\\` for `C:\\test\\`.

    the src paths are well normalized too at ~L1290.

    This change removes the block of code and instead
    does the "/" appending using the keepSlash logic
    that is in system.NormalizePath called in
    pathRelativeToWorkingDir() function before.

    fixes #4696

    Signed-off-by: Anthony Nandaa <[email protected]>

 frontend/dockerfile/dockerfile2llb/convert.go | 8 ++------
 solver/llbsolver/file/backend.go              | 2 +-
 2 files changed, 3 insertions(+), 7 deletions(-)

#4825

@profnandaa @gabriel-samfira

profnandaa · 2024-06-22T06:56:36Z

@tonistiigi -- will take a look. it should be more forgiving for such a case.

In my previous fix on moby#4825, I had removed this line knowing that all that had been addressed in `pahtRelativeToWorkingDir`: ```go if cfg.params.DestPath == "." // <-- this one || cfg.params.DestPath == "" || cfg.params.DestPath[len(cfg.params.DestPath)-1] == filepath.Separator { dest += string(filepath.Separator) } ``` However, I had overlooked the `"."` scenario. `""`, `"/"` are all handled correctly in `system.NormalizePath()`. This change therefore undoes this, to make sure "." is transformed correctly to "./" during COPY operation. fixes moby#5070 Signed-off-by: Anthony Nandaa <[email protected]>

profnandaa · 2024-06-24T15:51:50Z

Fix raised, but I've also opened an issue to add a few test cases on our integration tests to cover these path scenarios.

In my previous fix on moby#4825, I had removed this line knowing that all that had been addressed in `pahtRelativeToWorkingDir`: ```go if cfg.params.DestPath == "." // <-- this one || cfg.params.DestPath == "" || cfg.params.DestPath[len(cfg.params.DestPath)-1] == filepath.Separator { dest += string(filepath.Separator) } ``` However, I had overlooked the `"."` scenario. `""`, `"/"` are all handled correctly in `system.NormalizePath()`. This change therefore undoes this, to make sure "." is transformed correctly to "./" during COPY operation. fixes moby#5070 Signed-off-by: Anthony Nandaa <[email protected]>

In my previous fix on moby#4825, I had removed this line knowing that all that had been addressed in `pahtRelativeToWorkingDir`: ```go if cfg.params.DestPath == "." // <-- this one || cfg.params.DestPath == "" || cfg.params.DestPath[len(cfg.params.DestPath)-1] == filepath.Separator { dest += string(filepath.Separator) } ``` However, I had overlooked the `"."` and `""` scenarios. `""`. The `"/"`case is handled correctly in `system.NormalizePath()`. This change therefore undoes this, to make sure "." is transformed correctly to "./" during COPY operation. fixes moby#5070 Signed-off-by: Anthony Nandaa <[email protected]>

In my previous fix on moby#4825, I had removed this line knowing that all that had been addressed in `pahtRelativeToWorkingDir`: ```go if cfg.params.DestPath == "." // <-- this one || cfg.params.DestPath == "" || cfg.params.DestPath[len(cfg.params.DestPath)-1] == filepath.Separator { dest += string(filepath.Separator) } ``` However, I had overlooked the `"."` and `""` scenarios. `""`. The `"/"`case is handled correctly in `system.NormalizePath()`. This change therefore undoes this, to make sure "." is transformed correctly to "./" during COPY operation, same to "" -> "/". fixes moby#5070 Signed-off-by: Anthony Nandaa <[email protected]>

In my previous fix on moby#4825, I had removed this line knowing that all that had been addressed in `pahtRelativeToWorkingDir`: ```go if cfg.params.DestPath == "." // <-- this one || cfg.params.DestPath == "" || cfg.params.DestPath[len(cfg.params.DestPath)-1] == filepath.Separator { dest += string(filepath.Separator) } ``` However, I had overlooked the `"."` and `""` scenarios. `""`. The `"/"`case is handled correctly in `system.NormalizePath()`. This change therefore undoes this, to make sure "." is transformed correctly to "./" during COPY operation, same to "" -> "./". This is important especially for WORKDIR that are not `/`, so that `COPY --link` operations are handled properly. fixes moby#5070 Signed-off-by: Anthony Nandaa <[email protected]>

moby/buildkit#5070

tetienne-zenchef changed the title ~~Error during container init with v0.14.1 - mkdir xxxx not a directory~~ Error during container init with v0.14.1 - mkdir xxxx not a directory - with COPY --link Jun 21, 2024

tonistiigi added kind/bug area/dockerfile labels Jun 21, 2024

tonistiigi added this to the v0.14.2 milestone Jun 21, 2024

tonistiigi changed the title ~~Error during container init with v0.14.1 - mkdir xxxx not a directory - with COPY --link~~ [v0.14] dockerfile: mkdir xxxx not a directory - with COPY --link Jun 21, 2024

profnandaa self-assigned this Jun 22, 2024

This was referenced Jun 24, 2024

fix: dot and empty paths normalized correctly for COPY to WORKDIR #5080

Merged

tests: add frontend/dockerfile integration tests to cover different path scenarios for COPY, etc. #5081

Open

tonistiigi closed this as completed in #5080 Jun 28, 2024

tonistiigi closed this as completed in c605289 Jun 28, 2024

fhoeben added a commit to fhoeben/hsac-fitnesse-fixtures that referenced this issue Jul 12, 2024

fix?

b6bce81

moby/buildkit#5070

fhoeben added a commit to fhoeben/hsac-fitnesse-fixtures that referenced this issue Jul 12, 2024

fix docker build

74d8ec1

moby/buildkit#5070

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[v0.14] dockerfile: mkdir xxxx not a directory - with COPY --link #5070

[v0.14] dockerfile: mkdir xxxx not a directory - with COPY --link #5070

tetienne-zenchef commented Jun 21, 2024 •

edited

Loading

ufou commented Jun 21, 2024

tonistiigi commented Jun 21, 2024 •

edited

Loading

profnandaa commented Jun 22, 2024 •

edited

Loading

profnandaa commented Jun 24, 2024

[v0.14] dockerfile: mkdir xxxx not a directory - with COPY --link #5070

[v0.14] dockerfile: mkdir xxxx not a directory - with COPY --link #5070

Comments

tetienne-zenchef commented Jun 21, 2024 • edited Loading

ufou commented Jun 21, 2024

tonistiigi commented Jun 21, 2024 • edited Loading

profnandaa commented Jun 22, 2024 • edited Loading

profnandaa commented Jun 24, 2024

tetienne-zenchef commented Jun 21, 2024 •

edited

Loading

tonistiigi commented Jun 21, 2024 •

edited

Loading

profnandaa commented Jun 22, 2024 •

edited

Loading