Docker hub rate-limit / authentication of list-tags/repository calls

[sizowie]

Hi there,

we're constantly hitting the Docker Hub rate limit with an paid account/subscription. As for now our usage is around 50% of the allowed pulls/day (5.000) with hundreds of replications distributed over the whole day.

Therefore I'm wondering if pre-pull actions of Harbor like checking repository and tags for existence on remote side are also made authenticated with supplied Registry credentials or not?

2024-12-02T00:00:22Z [ERROR] [/pkg/reg/adapter/dockerhub/adapter.go:452]: list tags error: 429 -- {"detail": "Rate limit exceeded", "error": false}
...
2024-12-02T00:00:22Z [ERROR] [/pkg/reg/adapter/dockerhub/adapter.go:425]: list repos error: 429 -- {"detail": "Rate limit exceeded", "error": false}
...

Thank you!
Cheers, Janek

Versions:

• harbor version: v2.10.2
• docker engine version: 26.1.4
• docker-compose version: v2.24.1

[Vad1mo]

Docker Hub as changed the rate limits in Nov. There are quite a few issues regarding proxy-cache rate limit.

We should investigate the PR #19984

[stonezdj]

Yes, it uses the credentials provided.
You should follow this guide to check the rate limit: https://docs.docker.com/docker-hub/download-rate-limit/#rate-limit
The list tag will consume the quota.
Retry doesn't make sense for current issue.

[wy65701436]

According to the official Docker documentation, listing tags and repositories should not consume the pull count quota.

@sizowie, have you confirmed this with the upstream registry? I believe we can't take further action until we have a clear understanding of the situation.