Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Take accelerator builds into account in CI that checks for missing installations #753

Merged
merged 4 commits into from
Sep 27, 2024
Merged

Take accelerator builds into account in CI that checks for missing installations #753

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
0b5de15
add mapping for cpu->accelerators and check for missing accelerator b…
bedroge Sep 27, 2024
4a5808c
remove entries without an accelerator
bedroge Sep 27, 2024
eaf656a
replace hardcoded nvidia in easystack dir by "dirname $accel"
bedroge Sep 27, 2024
3c5520a
only run on *-software.eessi.io branches again
bedroge Sep 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 35 additions & 1 deletion .github/workflows/test-software.eessi.io.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,25 @@
name: Check for missing software installations in software.eessi.io
on:
push:
branches: [ "*-software.eessi.io" ]
# branches: [ "*-software.eessi.io" ]
branches: "*"
pull_request:
workflow_dispatch:
permissions:
contents: read # to fetch code (actions/checkout)
env:
EESSI_ACCELERATOR_TARGETS: |
aarch64/generic:
aarch64/neoverse_n1:
aarch64/neoverse_v1:
bedroge marked this conversation as resolved.
Show resolved Hide resolved
x86_64/generic:
x86_64/amd/zen2:
- nvidia/cc80
x86_64/amd/zen3:
- nvidia/cc80
x86_64/amd/zen4:
x86_64/intel/haswell:
x86_64/intel/skylake_avx512:
jobs:
check_missing:
runs-on: ubuntu-latest
Expand Down Expand Up @@ -48,6 +62,8 @@ jobs:
export EESSI_PREFIX=/cvmfs/software.eessi.io/versions/${{matrix.EESSI_VERSION}}
export EESSI_OS_TYPE=linux
env | grep ^EESSI | sort

# first check the CPU-only builds for this CPU target
echo "just run check_missing_installations.sh (should use easystacks/software.eessi.io/${{matrix.EESSI_VERSION}}/eessi-${{matrix.EESSI_VERSION}}-*.yml)"
for easystack_file in $(ls easystacks/software.eessi.io/${{matrix.EESSI_VERSION}}/eessi-${{matrix.EESSI_VERSION}}-eb-*.yml); do
echo "check missing installations for ${easystack_file}..."
Expand All @@ -56,6 +72,24 @@ jobs:
if [[ ${ec} -ne 0 ]]; then echo "missing installations found for ${easystack_file}!" >&2; exit ${ec}; fi
done

# now check the accelerator builds for this CPU target
accelerators=$(echo "${EESSI_ACCELERATOR_TARGETS}" | yq ".${EESSI_SOFTWARE_SUBDIR_OVERRIDE}[]")
if [ -z ${accelerators} ]; then
echo "no accelerator targets defined for ${EESSI_SOFTWARE_SUBDIR_OVERRIDE}"
else
for accel in ${accelerators}; do
module use ${EESSI_SOFTWARE_PATH}/accel/${accel}/modules/all
echo "checking missing installations for accelerator ${accel} using modulepath: ${MODULEPATH}"
for easystack_file in $(ls easystacks/software.eessi.io/${{matrix.EESSI_VERSION}}/accel/nvidia/eessi-${{matrix.EESSI_VERSION}}-eb-*.yml); do
bedroge marked this conversation as resolved.
Show resolved Hide resolved
echo "check missing installations for ${easystack_file}..."
./check_missing_installations.sh ${easystack_file}
ec=$?
if [[ ${ec} -ne 0 ]]; then echo "missing installations found for ${easystack_file}!" >&2; exit ${ec}; fi
done
module unuse ${EESSI_SOFTWARE_PATH}/accel/${accel}/modules/all
done
fi

- name: Test check_missing_installations.sh with missing package (GCC/8.3.0)
run: |
export EESSI_SOFTWARE_SUBDIR_OVERRIDE=${{matrix.EESSI_SOFTWARE_SUBDIR_OVERRIDE}}
Expand Down