Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support mixed-arch HPC where leader and jobs need different Python interpreters #5180

Open
unito-bot opened this issue Dec 9, 2024 · 0 comments
Labels

Comments

@unito-bot
Copy link

Bruno P Kinoshita from CWL4HPC has an HPC setup in Barcelona with x86_64 head nodes and ARM worker nodes. Toil supports ARM but only thinks about having single-architecture clusters, and the HPC virtualenv setup we use doesn’t really think about having to run the workers under a different Python interpreter.

We should try and support this cluster setup, and let you run the Toil leader on the head node and the Toil workers on ARM.

The workaround it so make sure the cluster lets worker nodes submit new jobs, and then run the leader as a job on the ARM workers.

┆Issue is synchronized with this Jira Story
┆Issue Number: TOIL-1687

@unito-bot unito-bot added the cwl label Dec 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant