Testing

Unit Tests

IDTxl uses the pytest package for testing.

This is the folder structure of the IDTxl package:

IDTxl/
    setup.py   # your setuptools Python package metadata
    idtxl/     # actual code
        __init__.py
        multivariate_te.py
        ...
    test/      # unit and system tests
        test_multivariate_te.py
        ...
    demos/     # demo scripts
        demo_multivariate_te.py
        ...

Go to the test folder and call

$ py.test   # execute pytest with Python 3

This starts the py.test module, which will automatically collect all test_*.py or *_test.py files and execute them (see here for py.test's test detection rules).

Some more ways to invoke unit tests with py.test (go here for a full documentation):

    $ pytest -x            # stop after first failure
    $ pytest --maxfail=2   # stop after two failures
    $ pytest test_mod.py   # run tests in given module

If pytest-cov is installed a report on code coverage can be created by calling

    py.test --cov idtxl

inside the test folder. (Running pytest --cov-report html --cov idtxl creates html output in a subfolder htmlcov.)

System Tests

The test folder also contains system tests in files systemtest_*.py, which are tests of whole algorithms or analyses and which generally run longer than unit tests (order of hours to days).

Testing demo scripts

Go to the folder demos and run py.test. py.test will use the local pytest.ini file which switches the automatic test detection from files with prefix test_*.py to demo_*.py. Accordingly, py.test will run all demo scripts. Note, that py.test will produce no console output while running the scripts.