Can I make a process take exclusive control of the CPU?

[rongcuid]

I would like certain processes to run exclusively. That is, when this process runs, no other process can execute in parallel. This is required as I am performing benchmarks. Is this possible in nextflow?

I have looked at the batch pattern and maxForks, but they seem to only limit the process itself, not other processes.

Example:

process exclusiveProcess {
  maxForks 1
  // maxSomething 1?
  "sleep 10"
}

process unrelatedProcess {
  "sleep 1"
}

workflow {
  exclusiveProcess | view
  unrelatedProcess | view
}

In the above example, the two processes are still executed in parallel. I want that exclusiveProcess actually runs exclusively.

[bentsherman]

You could set executor.cpus in your config file to limit how many CPUs are available to the pipeline, then set the cpus directive for the exclusive process such that it uses all the available CPUs and prevents any other task from running at the same time.

[bentsherman]

How did you do it

[rongcuid]

    maxForks 1
    cpus {Runtime.getRuntime().availableProcessors()}

[bentsherman]

I think that's fine. You probably still need to do Math.max(1, Runtime.getRuntime().availableProcessors() - 1) or else Nextflow might fail. And you shouldn't need the maxForks or curly braces

[rongcuid]

Here's my final solution:

process exclusiveProcess {
  cpus { Runtime.runtime.availableProcessors() }
  "sleep 10"
}

process unrelatedProcess {
  "sleep 1"
}

workflow {
  exclusiveProcess | view
  unrelatedProcess | view
}