Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: parameterise s3 build cache setup #465

Merged
merged 6 commits into from
Sep 11, 2024
Merged

Conversation

galargh
Copy link
Contributor

@galargh galargh commented Sep 3, 2024

As we're setting up a new cache bucket, we'd like to be able to control its' configuration via GitHub vars/secrets fully.

@AlejandroCabeza
Copy link

Are you planning to make the bucket publicly accessible? As Protocol Labs' was previously?

@diegomrsantos
Copy link
Contributor

@achingbrain complementing @AlejandroCabeza question, is there any cache that can be used at the moment, or from now on projects using this repo will need to set up their own individually?

@achingbrain
Copy link
Member

I don't know, sorry - you may need to stand your own s3 bucket up.

If that adds too much overhead, you could apply for a grant from the libp2p foundation to contribute to a shared infrastructure?

@AlejandroCabeza
Copy link

I don't know, sorry - you may need to stand your own s3 bucket up.

If that adds too much overhead, you could apply for a grant from the libp2p foundation to contribute to a shared infrastructure?

Alright, thanks for the answer!

@galargh galargh force-pushed the galargh/s3-build-cache branch 4 times, most recently from 572d20f to 31176a4 Compare September 5, 2024 15:56
@galargh galargh force-pushed the galargh/s3-build-cache branch from 31176a4 to 69f4fcc Compare September 5, 2024 16:01
@galargh galargh requested a review from achingbrain September 5, 2024 16:09
const cachePath = fs.mkdtempSync(path.join(os.tmpdir(), 'cache'))
const archivePath = path.join(cachePath, 'archive.tar.gz')
const dockerLoadedMsg = child_process.execSync(`aws s3 cp s3://${AWS_BUCKET}/imageCache/${cacheKey}-${arch}.tar.gz ${archivePath} && docker image load -i ${archivePath}`).toString();
const loadedImageId = dockerLoadedMsg.match(/Loaded image( ID)?: (.*)/)[2];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const loadedImageId = dockerLoadedMsg.match(/Loaded image( ID)?: (.*)/)[2];
const loadedImageId = dockerLoadedMsg.match(/Loaded image( ID)?: (.*)/)[2];

Copy link
Member

@achingbrain achingbrain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but CI is failing

@galargh galargh merged commit e5e32c7 into master Sep 11, 2024
3 checks passed
@galargh galargh deleted the galargh/s3-build-cache branch September 11, 2024 08:01
AlejandroCabeza added a commit to vacp2p/libp2p-test-plans that referenced this pull request Sep 12, 2024
* upstream/master:
  chore: update the link to the interop dashboard [skip ci]
  chore: parameterise s3 build cache setup (libp2p#465)
  chore: Create funding.json
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants