Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove subset.txt #97

Merged
merged 1 commit into from
Nov 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 5 additions & 19 deletions Jenkinsfile.meta
Original file line number Diff line number Diff line change
Expand Up @@ -45,17 +45,6 @@ node {
sh '''
git submodule update --remote --merge .doi
git submodule update --remote --merge .scripts

# TODO once "repos_anti_subset" in "doi.jq" is empty, we can remove this (and all associated usages of "subset.txt" can just be "--all" or go away completely)
# in all the places we need to interact with our "subset" it's a lot easier to have an explicit list of what's included, so we'll continue to generate "subset.txt" until it contains the full set
bashbrew list --all --repos | jq -L.scripts -rsR '
include "doi";
rtrimstr("\n")
| split("\n")
| . - repos_anti_subset
| join("\n")
' > subset.txt
git add subset.txt
'''
}

Expand All @@ -65,11 +54,11 @@ node {
string(credentialsId: 'dockerhub-public-proxy-host', variable: 'DOCKERHUB_PUBLIC_PROXY_HOST'),
]) {
stage('Fetch') {
sh 'bashbrew --library .doi/library fetch $(cat subset.txt)' // TODO --all
sh 'bashbrew --library .doi/library fetch --all'
}
stage('Sources') {
sh '''
# we only need to regenerate "sources.json" if ".doi", ".scripts", or "subset.txt" have changed since we last generated it
# we only need to regenerate "sources.json" if ".doi" or ".scripts" have changed since we last generated it

needsBuild=
if [ ! -s commits.json ] || [ ! -s sources.json ]; then
Expand All @@ -78,17 +67,14 @@ node {

doi="$(git -C .doi log -1 --format='format:%H')"
scripts="$(git -C .scripts log -1 --format='format:%H')"
subset="$(sha256sum subset.txt | cut -d' ' -f1)"
export doi scripts subset
jq -n '{ doi: env.doi, scripts: env.scripts, subset: env.subset }' | tee commits.json
export doi scripts
jq -n '{ doi: env.doi, scripts: env.scripts }' | tee commits.json
if [ -z "$needsBuild" ] && ! git diff --exit-code commits.json; then
needsBuild=1
fi

if [ -n "$needsBuild" ]; then
images="$(cat subset.txt)"
[ -n "$images" ]
.scripts/sources.sh $images > sources.json
.scripts/sources.sh --all > sources.json
fi
'''
}
Expand Down
9 changes: 0 additions & 9 deletions doi.jq
Original file line number Diff line number Diff line change
@@ -1,12 +1,3 @@
# our old "subset.txt", but inverted
# this way, we can "fail open" correctly (new images should be part of meta by default)
# see "Jenkinsfile.meta" for how/why this turns into "subset.txt"
def repos_anti_subset:
[
empty
]
;

# a helper for "build_should_sbom"
def _sbom_subset:
[
Expand Down