Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adapt Docker setup to latest changes #5045

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

tpetillon
Copy link
Contributor

Fixes #5031, #5005.

Requires #4893, or any other solution for #5043.

The Docker setup is a quick and easy way to get an Openstreetmap-carto dev environment running, so it’s probably worth keeping it in a working state. This PR updates the startup script to take recent changes into account, such as #4978. It also updates the base images to more recent versions.

With this PR alone, setting up the containers fails because fonts cannot be downloaded. Applying #4893 allows testing the whole setup.

@dch0ph
Copy link
Contributor

dch0ph commented Nov 23, 2024

Many thanks for working on this. Some minor comments from giving it a test run:

docker-compose up import to import the data (only necessary the first time or when you change the data file).

It would be useful to add that that downloading and importing the water polygons table can take substantial time, and will be the rate limiting step when importing a small subset of OSM data (which will be the normal use case). It doesn't help that the message shown by get-external-data.py is "Checking table X" during download, which gives the impression that things have got stuck (why is "checking" taking so long?).

On the first run, import failed with "Max retries exceeded with url: /download/antarctica-icesheet-outlines-3857.zip (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f345affb7d0>: Failed to resolve 'osmdata.openstreetmap.de", which is odd because the website was functioning. It worked on repeating the docker-compose up import (again this seems to be more an issue/fragility of get-external-data.py rather than the docker set up per se).

[Haven't tested beyond the font import step. This fails with "scripts/get-fonts.sh: 11: curl: not found".]

Dockerfile.import Outdated Show resolved Hide resolved
@tpetillon
Copy link
Contributor Author

It doesn't help that the message shown by get-external-data.py is "Checking table X" during download, which gives the impression that things have got stuck (why is "checking" taking so long?).

I have added a few more log message to help with tracking the import process, and mentioned the external data download in DOCKER.md.

The latest version can now go through the import process without having to apply more patches. Noto Emoji is downloaded from the Internet Archive using the file @pjduplooygis has uploaded earlier this year. And because fonts are downloaded separately during the import process, they have been removed from the package list in the Dockerfile.

I don’t really want to go much further regarding fonts, as it isn’t the main topic of this PR.

@dch0ph
Copy link
Contributor

dch0ph commented Dec 21, 2024

Have now confirmed this works when merged with #4893.

The (existing) instructions about setting environment variables e.g. to adjust the behaviour of get-external-data.py are a bit confusing and don't seem to apply to running Docker containers from Windows/WSL. But that is a separate topic to this PR, which is focussed on getting the Docker set-up working again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Docker issue with the new osm2pgsql style
3 participants