AWS ECR login fails in windows-2022 20241211.1.0 #11172
Comments
|
Hi @PatTheSilent , Thank you for raising the issue , we will look into the issue and update Asap |
|
We have a similar issue moving from |
vidyasagarnimmagaddi
added
the
awaiting-deployment
|
Hi folks, just adding that my organization has also run into this issue. Our logs indicate we first experienced the failure at the end of last week (went unnoticed going into the weekend), and we continue to see it this morning. As with @PatTheSilent, we are encountering this in a workflow that uses a GitHub-hosted large runner pool, and as with @mavimo, we're seeing it during the first One difference from @mavimo is that we are seeing this failure in @kishorekumar-anchala please see below for some lightly sanitized logs, and please let me know if I can provide any additional information to help your investigation. This is a major blocker for building Windows-based Docker images in CI, so hoping our logs may help things along: |
|
@austindrenski this is related on how the secret is stored in the windows, see here, I suspect that during the upgrade we switch to a new version of docker that switch where we store secrets. As workaround we add: where |
|
Thanks @mavimo, that does indeed get us unblocked in the meantime! Note In the spirit of contributing copy/pasta for future readers, here's what we added just before the first login action step of each affected workflow: - name: Temporary workaround for actions/runner-images#11172
run: |
mkdir --parents ~/.docker
echo '{"auths":{"ghcr.io":{}}}' > ~/.docker/config.json
shell: bash |
|
Hello everyone, Thank you for reporting the problem. We have identified root cause and working on fix. |
|
Fix is deployed and I confirmed that issue is resolved. Could you please double check that issue is resolved for you too? |
|
@maxim-lobanov can confirm my builds pass now without a problem |
|
@maxim-lobanov Any idea what the propagation time is internally, e.g. to 1ES Pools? (still seeing failures here, not sure when we can expect recovery to unblock CIs - we see image version 156.0.0 right now) |
|
@NickCraver sorry, I am not familiar with 1ES pools and didn't expect them to be impacted 🤔. This issue was related to configuration of Larger Runners in GitHub Actions and even Standard Runners were not impacted by this problem (Standard Runners and Larger Runners re-use the same image). If you feel that it is the same issue on 1ES pools and timeline of issue matches, could you please share some additional logs from begin of job? Like image version, agent, etc.
I am bit confused because our image versions have different format like |
|
@maxim-lobanov Definitely can - they're internal links though, Teams? It may be unrelated, we're seeing issue with git credential storage on Windows that happened at the same time (starting Saturday) and it only applies to this image, not an older one running in another pool running the same tests in another region, so we have a data point there to compare to at least. |
|
@maxim-lobanov Heads up we are seeing the same behavior on 20241215.2 internally. |
Description
We've started experiencing failures to login to AWS ECR starting from windows-2022 20241211.1.0
We're using "big" runners, which have received the update to 20241211.1.0 on which we're experiencing this failure.
We're not experiencing this on windows-2022 runners, which still have 20241201.2.0
Platforms affected
Runner images affected
Image version and build link
Works as expected in 20241201.2.0
fails in 20241211.1.0
Is it regression?
20241201.2.0
Expected behavior
I'm able to authenticate with AWS ECR using aws-actions/amazon-ecr-login@v1
Actual behavior
Repro steps
The text was updated successfully, but these errors were encountered: