PDK::Util::Bundler.ensure_bundle! causes parallel execution of pdk new test to fail

[hsnodgrass]

Describe the bug
When running pdk new test --unit in PowerShell 7 via the ForEach-Object -Parallel cmdlet, the function PDK::Util::Bundler.ensure_bundle! causes failures due to the function moving the Gemfile.lock while the other executing parallel commands are checking for Gemfile.lock and attempting to move it.

To Reproduce

• Using PowerShell 7, cd into a module directory
• Run the following command Get-ChildItem -Path .\manifests\path\to\your\manifests\dir | ForEach-Object -Parallel { pdk new test --unit ('module::path::to::your::manifests::dir::' + $_.name -replace '.pp','') } -ThrottleLimit 20
  • The path and the namespace should reflect your module
• Generating the tests will fail. You can verify that PDK::Util::Bundler.ensure_bundle! is the culprit by commenting out the following line in C:\Program Files\Puppet Labs\DevelopmentKit\private\ruby\<version>\lib\ruby\gems\<version>\gems\pdk-2.1.0\lib\pdk\cli\new:
  • # PDK::Util::Bundler.ensure_bundle!(puppet_env[:gemset]). Line 36 for me, may be different for you.
  • After commenting this line out, run the PowerShell command again and everything will work fine.

Expected behavior
Running pdk new test --unit in parallel should not fail. This would require either a command line flag to skip ensure_bundle! or alternate logic in ensure_bundle! that does not require moving Gemfile.lock.

Additional context

• Your PDK installation method: installed via Chocolatey on Windows 10
• Your PDK version: 2.1.0
• Your operating system: Windows 10 build 19042 / PowerShell Core 7.1.3

[jpogran]

Thanks for the great bug report @hsnodgrass!

Unfortunately the ruby pdk internals were not designed to handle being executed in parallel. As you have found, there are a significant number of file system and gem operations that happen that expect to have exclusive access to files and folders during the operation. There isn't anything we can do at this time to help with this code path.

The good news is that we already have work in flight that address this kind of work with a new approach. We'll be communicating this approach in the coming weeks.

[jpogran]

Hi @hsnodgrass I'd love to know if the new Puppet Content Templates can address this issue for you

[github-actions]

This issue has been marked stale because it is open with no recent activity. The PDK Team is actively prioritizing existing bugs and new features, if this issue is still important to you please comment and we will add this to our backlog to complete.

[github-actions]

This issue was closed because it is open with no recent activity. The PDK Team is actively prioritizing existing bugs and new features, if this issue is still important to you please comment and we will add this to our backlog to complete.

[chelnak]

Reopened. Should not have been closed.

[david22swan]

Unfortunately this is a known issue we have with PowerShell 7 and not something that we can fix at the moment, as said by James above:

Unfortunately the ruby pdk internals were not designed to handle being executed in parallel. As you have found, there are a significant number of file system and gem operations that happen that expect to have exclusive access to files and folders during the operation. There isn't anything we can do at this time to help with this code path.