Specifying default nodeSelector and toleration for k8s.pod directive

[peterwong-prenetics]

Hello, I see that when using k8s as an executor. I would like to set up default pod nodeSelector and toleration using a config file specified through -c option in the command line when calling nextflow run. Here is the k8s scoped configuration

k8s {
    pod = [
        nodeSelector: 'purpose=nextflow',
        toleration: [
            [
                key:'purpose',
                operator:'Equal',
                value:'nextflow',
                effect:'NoSchedule'
            ]
        ]
    ]
}

However, it seems nextflow cannot pick up this configuration. The same configuration works if defined inside a process such as the following

process Test {
	pod =  [
		[
			nodeSelector: 'purpose=nextflow'
		], 
		[
			toleration: [
				key:'purpose',
				operator:'Equal',
				value:'nextflow',
				effect:'NoSchedule'
			]
		]
	]
	container "ubuntu:latest"

	output:
		stdout

	script:
		"""
		echo "Hello World!"
		"""
}

I would like to know whether there is an error in my configuration or this is an expected behaviour. I am using the latest nextflow verion 23.10.1.5891

[bentsherman]

Looks right to me, but another way you can try is to put it in the process config scope:

process {
    pod = [
        // ...
    ]
}

[peterwong-prenetics]

Thanks @bentsherman however I’d like to see if I can add default tolerations and/or node selectors in the config without specifying them in individual processes

[bentsherman]

You can do that as I have shown above, it will apply to all processes. The pod directive is additive, so if you also specify process-specific pod options, they will be appended to the global options from the config

[peterwong-prenetics]

This is the related PR #2763 that introduced this feature

[peterwong-prenetics]

So this works by using a list of 2 maps, one for each of nodeSelector and toleration.

k8s {
    pod = [
        [
            nodeSelector: 'purpose=pipeline'
        ],
        [
            toleration: [
                key:'purpose',
                operator:'Equal',
                value:'pipeline',
                effect:'NoSchedule'
            ]
        ]
    ]
}