-
Notifications
You must be signed in to change notification settings - Fork 50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Map tests to software builds #673
Map tests to software builds #673
Conversation
…software that is build works
… software was built
Instance
|
Instance
|
bot: build repo:eessi.io-2023.06-software arch:zen3 |
Updates by the bot instance
|
Updates by the bot instance
|
New job on instance
|
Hmmm... I guess dumping the
|
bot: build repo:eessi.io-2023.06-software arch:zen3 |
Updates by the bot instance
|
Updates by the bot instance
|
New job on instance
|
…ved correctly. Also, end the test step
bot: build repo:eessi.io-2023.06-software arch:zen2 |
Updates by the bot instance
|
Updates by the bot instance
|
New job on instance
|
bot: build repo:eessi.io-2023.06-software arch:zen2 |
Updates by the bot instance
|
Updates by the bot instance
|
New job on instance
|
…sts are always added at the end, even if all test had other matches
bot: build repo:eessi.io-2023.06-software arch:zen2 |
Updates by the bot instance
|
Updates by the bot instance
|
New job on instance
|
This looks like it should be:
Just for the sake of argument, let me retry this with a change to the mapping file so that |
… EESSI_ESPRESSO and EESS_OSU tests
bot: build repo:eessi.io-2023.06-software arch:zen2 |
Updates by the bot instance
|
Updates by the bot instance
|
New job on instance
|
bot: build repo:eessi.io-2023.06-software arch:zen2 |
Updates by the bot instance
|
Updates by the bot instance
|
New job on instance
|
Ok, this looks exactly as it was intended:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lgtm, really nice improvement!
OSU-Micro-Benchmarks/*: | ||
- EESSI_OSU_Micro_Benchmarks | ||
GROMACS/*: | ||
- EESSI_GROMACS |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@casparvl We'll need to maintain this as we add more tests to the test suite, right?
Wouldn't it make more sense to make sure that we consistently use EESSI_<software_name>
, so we don't need this mapping file at all?
The tests from the EESSI test suite that are run should depend on which software was built. This PR implements that functionality. Roughly speaking, it does the following:
module_files.list.txt
(in the tempdir)software_to_test.yml
) that maps regular expressions matching the (new) module names to a (list of) test(s) that need to be ran for that particular module name. First match in theyml
file will be returned, so order is important.module_files.list.txt
andsoftware_to_test.yml
, loops over all build software, each time executing the regular expression matches on the content of thesoftware_to_test.yml
, thus finding the tests that should be run for that particular module. If that test wasn't part of an aggregate list yet, it'll append it. Finally, the list is turned into a ReFrame argument string of the form e.g.-n EESSI_GROMACS -n EESSI_ESPRESSO
, ... etc.