Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize Dockerfile #1679

Merged
merged 2 commits into from
Jul 5, 2024
Merged

Optimize Dockerfile #1679

merged 2 commits into from
Jul 5, 2024

Conversation

amayer5125
Copy link
Contributor

Use a virtual environment to make copying packages from build stage to the final stage easier. It also reduces the image size by 94Mb.

I used python:3-slim as the final image. Previously we were using python:3.9-slim. I think using the newer python version is a good idea, but I can change it back if we don't want to use the newer version.

Size comparison:

REPOSITORY        TAG       IMAGE ID       CREATED         SIZE
errbotio/errbot   master    416f2383b24a   7 seconds ago   325MB
errbotio/errbot   new       0226c1a39cd1   2 minutes ago   231MB

@sijis
Copy link
Contributor

sijis commented Jan 25, 2024

Thank you for you suggestion on improvement.

I try and not to modify the default PATH variable, as it changes the expected location of files.
Do you think there could be another approach in reducing the files? Could the wheel files be related (even though directory is removed)?

@amayer5125
Copy link
Contributor Author

The PATH changes I make are the same thing a python virtual environment does when you run the activate script. I do append the existing PATH after the virtual environment location so any commands outside of the virtual environment should still work as expected.

The size difference almost certainly has to do with the wheel files from the COPY --from=build /wheel /wheel line. That line copies the files to the image and creates a new layer. The subsequent line starts a new layer and runs rm -rf /wheel which only deletes the files in the new layer. They still exist in the previous layer though, so they contribute to the overall size of the image. I cannot think of a way to copy the files into the new image for the duration of the install and then delete them.

@sijis
Copy link
Contributor

sijis commented Jan 27, 2024

I see what you mean.

Quickly looking, there may be something with RUN --mount that is suggested https://stackoverflow.com/a/76670359, which sounds promising. I have yet to try it myself though.

@amayer5125
Copy link
Contributor Author

I did some reading on --mount. If you do not want to use virtual environments for the image I think it is a great alternative to reduce the size. Switching from COPY to --mount only saved about 21MB. I did some of the other refactors I did in this pull request and got the image size down way farther.

Here are my test results:

REPOSITORY         TAG              IMAGE ID       CREATED          SIZE
errbotio/errbot    venv             e2ceb54767df   34 seconds ago   216MB
errbotio/errbot    mount-refactor   1f06e5451f5d   12 minutes ago   211MB
errbotio/errbot    mount            cb5f34d5318e   25 minutes ago   305MB
errbotio/errbot    master           6d3c52d7d1b6   34 minutes ago   326MB

master

Built off of master with no modifications.

mount

Remove COPY --from=build /wheel /wheel line and add --mount=from=build,source=/wheel,target=/wheel to the pip install line.

mount-refactor

Same as mount tag, plus removed apt install git, use python:3.9 image for build, and remove unneeded FROM base.

venv

This current pull request built with python:3.9 build stage and python:3.9-slim final image.

Use --mount instead of COPY to reduce image size
@sijis sijis force-pushed the optimize-dockerfile branch from a872868 to d4a52fe Compare July 5, 2024 04:37
@sijis
Copy link
Contributor

sijis commented Jul 5, 2024

@amayer5125 Thank you for all the work with this. merging.

@sijis sijis merged commit cbfc29f into errbotio:master Jul 5, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants