Perl-module-CPAN installations hang waiting for input

[lexming]

The installation of Perl-bundle-CPAN (all versions) are currently broken on 5.0.x. Several extensions ask for question during installation and the process hangs waiting for input. Some of the offenders are GO, Net-SSLeay, MIME-Lite and XML-Twig.

Example for Net-SSLeay:

eb-shell> export OPENSSL_PREFIX=$EBROOTOPENSSL &&  perl Makefile.PL PREFIX=/user/brussel/101/vsc10122/easybuild/install/skylake/software/Perl-bundle-CPAN/5.38.2-GCCcore-13.3.0 --help
Do you want to run external tests?
These tests *will* *fail* if you do not have network connectivity. [n] n
*** If there are build errors, test failures or run-time malfunctions,
    try to use the same compiler and options to compile your OpenSSL,
    Perl, and Net::SSLeay.
*** Found OpenSSL-3.2.2 installed in /user/brussel/101/vsc10122/easybuild/install/skylake/software/OpenSSL/3
Generating a Unix-style Makefile
Writing Makefile for Net::SSLeay
Writing MYMETA.yml and MYMETA.json
eb-shell> exit

These extensions have been part of our installations of Perl fro a long time and nothing has changed on their side. This issue is related to some change on EasyBuild 5.0.x and seems to only occur in older Python 3.6:

• EB v4.9 on Python 3.6: working
• EB EB 5.0.x on Python 3.6: broken: broken
• EB 5.0.x on Python 3.9: working

There are two main methods how these perl modules prompt for input, manually or with prompt() from Module::Build. Regardless of the method used or any environment variables set, perl detects if it is running on an interactive shell or not. Whenever the shell is non-interactive, prompts are automatically handled by using the default answer without waiting for input.

Therefore, there is some bug on how run_shell_cmd works in EB 5.0.x with Python 3.6 that results in an shell that seems interactive to Perl, triggering the input prompts.

[lexming]

Fixed in easybuilders/easybuild-framework#4728

[boegel]

@lexming Hmm, I'm now suddenly seeing the installer of Perl modules waiting for input now, after the changes you've made in easybuilders/easybuild-framework#4728:

== installing extension Template 3.101 (74/414)...
  >> defining build environment for GCCcore/12.3.0 toolchain
  >> running shell command:
        tar xzf /data/gent/400/vsc40023/EasyBuild/sources/p/Perl-bundle-CPAN/extensions/Template-Toolkit-3.101.tar.gz
        [started at: 2024-12-23 13:03:27]
        [working dir: /tmp/vsc40023/easybuild_build/PerlbundleCPAN/5.36.1/GCCcore-12.3.0/Template]
        [output and state saved to /tmp/eb-iqk4h4fl/run-shell-cmd-output/tar-dahtl5tt]
  >> command completed: exit 0, ran in < 1s
  >> running shell command:
        perl Makefile.PL PREFIX=/user/gent/400/vsc40023/eb_arcaninescratch/RHEL8-eb5dev-rpath/zen2-ib/software/Perl-bundle-CPAN/5.36.1-GCCcore-12.3.0
        [started at: 2024-12-23 13:03:27]
        [working dir: /tmp/vsc40023/easybuild_build/PerlbundleCPAN/5.36.1/GCCcore-12.3.0/Template/Template-Toolkit-3.101]
        [output and state saved to /tmp/eb-iqk4h4fl/run-shell-cmd-output/perl-q71ajkqe]
Installing 'Template' extension (74/414) ━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:39:02

checking the output, I see this:

The Template Toolkit requires that the AppConfig module (version 1.56
or later) first be installed.  This is used by
the 'ttree' program for reading command line options and configuration
files.  It is available from CPAN:

    http://www.cpan.org/authors/Andy_Wardley/


                    Template Toolkit Version 3.101
                    =============================

Using Unix defaults.
Run 'perl Makefile.PL TT_HELP' for a summary of options.


Template::Stash::XS
-------------------

The Template::Stash module is a core part of the Template Toolkit,
implementing the magic for accessing data using the dot notation.

There is a high speed version, Template::Stash::XS, written in C.
This makes the Template Toolkit run about twice as fast as when using
the regular Template::Stash written in Perl.  If you've got a C
compiler on your system then you can elect to have the XS Stash built.
You can also specify that you want to use the XS Stash by default.

Note that as of version 2.15 the XS Stash now supports access to tied
hashes and arrays.

See 'perldoc Template::Config' for further details.

Do you want to build the XS Stash module? [y]

So it looks like we're not out of the woods yet here...

[lexming]

I can reproduce this new hang on my Python 3.9 environment, which is very puzzling because Template::Config never failed in my original report with Python 3.6. And I could expect a failure on some extension going ahead not reading input due to stdin=None, but not the opposite, a new prompt for input appearing.

Anyhow, clearly using the default None value for the STDIN handle is not reliable. Using subprocess.DEVNULL is the actual solution. It ensures that those shells are always treated as non-interactive.

So the situation is as follows on my side:

• original code of EB 5.0 with subprocess.PIPE on stdin:
  • Python 3.6 hangs on GO
  • Python 3.9 works
• PR#4728 with None on stdin (same approach as EB v4.9):
  • Python 3.6 works
  • Python 3.9 hangs on Template::Config
• New PR#4731 with subprocess.DEVNULL on stdin:
  • Python 3.6 works
  • Python 3.9 works