Failing Tests in Test_OpenQasmDevice.cpp when installing on MacOS

[ystade]

Issue description

Expected behavior

When following the installation guide on https://docs.pennylane.ai/projects/catalyst/en/stable/dev/installation.html#minimal-building-from-source-guide for MacOS, after the step make test all tests should pass.

Actual behavior

• After executing make test the last lines of the output are the following:

-------------------------------------------------------------------------------
Test PSWAP and ISWAP with BuilderType::Braket
-------------------------------------------------------------------------------
/Users/yannick/Documents/TUM/CDA/Research/playgrounds/Catalyst/catalyst/runtime/tests/Test_OpenQasmDevice.cpp:529
...............................................................................

/Users/yannick/Documents/TUM/CDA/Research/playgrounds/Catalyst/catalyst/runtime/tests/Test_OpenQasmDevice.cpp:529: FAILED:
due to unexpected exception with message:
  ModuleNotFoundError: No module named 'braket'

At:
    <string>(3): <module>

===============================================================================
test cases:  33 |  28 passed |  5 failed
assertions: 177 | 151 passed | 26 failed

make[1]: *** [test] Error 26
make: *** [test-runtime] Error 2

System information

Machine: Mac M1, Mac M2, Mac M3, with MacOS 15.0.1 installed.
Python used in a virtualenv: 3.11.9 and 3.12.6

The output of import pennylane as qml; qml.about():

Name: PennyLane
Version: 0.39.0.dev26
Summary: PennyLane is a cross-platform Python library for quantum computing, quantum machine learning, and quantum chemistry. Train a quantum computer the same way as a neural network.
Home-page: https://github.com/PennyLaneAI/pennylane
Author:
Author-email:
License: Apache License 2.0
Location: /Users/yannick/.pyenv/versions/catalyst/lib/python3.11/site-packages
Requires: appdirs, autograd, autoray, cachetools, networkx, numpy, packaging, pennylane-lightning, requests, rustworkx, scipy, toml, typing-extensions
Required-by: amazon-braket-pennylane-plugin, PennyLane-Catalyst, PennyLane_Lightning, PennyLane_Lightning_Kokkos

Platform info:           macOS-15.0.1-arm64-arm-64bit
Python version:          3.11.9
Numpy version:           2.0.0
Scipy version:           1.13.0
Installed devices:
- default.clifford (PennyLane-0.39.0.dev26)
- default.gaussian (PennyLane-0.39.0.dev26)
- default.mixed (PennyLane-0.39.0.dev26)
- default.qubit (PennyLane-0.39.0.dev26)
- default.qutrit (PennyLane-0.39.0.dev26)
- default.qutrit.mixed (PennyLane-0.39.0.dev26)
- default.tensor (PennyLane-0.39.0.dev26)
- null.qubit (PennyLane-0.39.0.dev26)
- reference.qubit (PennyLane-0.39.0.dev26)
- lightning.qubit (PennyLane_Lightning-0.39.0.dev38)
- nvidia.custatevec (PennyLane-Catalyst-0.9.0.dev30)
- nvidia.cutensornet (PennyLane-Catalyst-0.9.0.dev30)
- oqc.cloud (PennyLane-Catalyst-0.9.0.dev30)
- softwareq.qpp (PennyLane-Catalyst-0.9.0.dev30)
- lightning.kokkos (PennyLane_Lightning_Kokkos-0.39.0.dev38)
- braket.aws.ahs (amazon-braket-pennylane-plugin-1.30.1)
- braket.aws.qubit (amazon-braket-pennylane-plugin-1.30.1)
- braket.local.ahs (amazon-braket-pennylane-plugin-1.30.1)
- braket.local.qubit (amazon-braket-pennylane-plugin-1.30.1)

Additional information

• On Ubuntu this behavour could not be reproduced and everything has worked as expected.
• After installing the libraries into the system python version the test went through on one machine. Hence, the assumption is that for those tests the system python version is used instead of the version from the virtual environment.

[josh146]

Thanks for reporting this @ystade! This looks like it is because one of the tests depends on the braket plugin being installed (pip install amazon-braket-pennylane-plugin).

Ideally though we should ensure that this test is skipped if braket is not installed.

[dime10]

This is true, we normally skip pytests if an optional package is not installed. In this case it looks to be a C++ that was added not too long ago 🤔

@ystade Normally all test and build dependencies are covered by pip install -r requirements.txt, did you run this line before building?

[burgholzer]

The problem here seems to be very macOS specific. Everything succeeds on a fresh install of Catalyst on Ubuntu.
It seems to be that these C++ tests which call Python via an embedded interpreter for some reason do not use the site-packages from the virtual environment that was used to build catalyst.
All the required dependencies (from the requirements.txt) are installed in that virtual environment. But they are not installed in the system python site-packages.
This is supported by the fact that manually installing amazon-braked-sdk and numpy==2.0 in the global site packages fixes the errors in the tests, i.e., running, e.g.,

python3.12 -m pip install --user --break-system-packages amazon-braket-sdk "numpy==2.0"

where python3.12 links to the exact version that was used to create the virtual environment (in which all dependencies, including catalyst, have been installed into as part of make all).
Naturally, that is not a recommended fix and the C++ tests should pick up the right Python site-package directory.
Note again, that Linux builds work out of the box and pick the packages from the virtual environment when running these tests.

[dime10]

Ah, thanks for surfacing the actual issue @burgholzer! We'll look into it.