Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests for speechbrain tutorial #1007

Open
wants to merge 31 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
0c1d329
base files for speechbrain tutorial
Simnol22 Sep 27, 2022
f97b7f8
writing rst files for tutorial, using valid loss
Simnol22 Oct 3, 2022
ced901f
adding results
Simnol22 Oct 3, 2022
fdfa5c9
adding speechbrain tutorial to index
Simnol22 Oct 3, 2022
5c7a8c9
calling main.py instead of train.py for this tutorial
Simnol22 Oct 3, 2022
2b2700c
init commit
Simnol22 Oct 3, 2022
a49799c
fixing doc test
Simnol22 Oct 3, 2022
5b3fe25
adding tests for speechbrain model
Simnol22 Oct 13, 2022
b4a3855
passing pipeline
Simnol22 Oct 13, 2022
e6a2322
fixing doc8 check
Simnol22 Oct 13, 2022
c7c46be
fixing last line
Simnol22 Oct 13, 2022
a73c0e1
adding last line
Simnol22 Oct 13, 2022
d10b015
Adding speechbrain in requirements
Simnol22 Oct 13, 2022
99de397
good name for experiment + removing prints
Simnol22 Oct 13, 2022
d16f397
changing experiment name, running pipeline
Simnol22 Oct 14, 2022
3a9fdb2
Trying simpler test for pipeline
Simnol22 Oct 19, 2022
c7fe640
Turn strings into proper comments
bouthilx Dec 14, 2022
dc4ea45
adding link to sb repository and a download function
Simnol22 Dec 14, 2022
f3abe10
adding verbose and prints for debugging github pipeline
Simnol22 Dec 15, 2022
32aafa4
testing with just download data
Simnol22 Dec 16, 2022
ac1a723
Adding some more code
Simnol22 Dec 16, 2022
23fa7cd
only 1 test to debug
Simnol22 Dec 16, 2022
69f0d90
only 1 test to debug removing comments
Simnol22 Dec 16, 2022
943efe1
removing dashboard test aswell
Simnol22 Dec 16, 2022
92e6c49
removing dependexy githubA
Simnol22 Dec 16, 2022
a411619
removing need:
Simnol22 Dec 16, 2022
02f5357
running only necessary test
Simnol22 Dec 16, 2022
beaa361
all but report_objective
Simnol22 Dec 16, 2022
f8c0a84
all but report_objective
Simnol22 Dec 16, 2022
af97eb9
test without fit fct
Simnol22 Dec 16, 2022
dc31016
Update for now. Waiting for runners with more RAM to merge
Simnol22 Dec 19, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
314 changes: 3 additions & 311 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,82 +13,14 @@ on:
workflow_dispatch:

jobs:
pre-commit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
- run: pip install pre-commit
- run: pre-commit --version
- run: pre-commit install
- run: pre-commit run --all-files
pretest:
runs-on: ubuntu-latest
strategy:
matrix:
toxenv: [pylint, doc8, docs]

steps:
- uses: actions/checkout@v2
- name: Set up Python 3.9
uses: actions/setup-python@v2
with:
python-version: 3.9
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install tox tox-gh-actions
- name: Run pre-tests with Tox
run: tox -e ${{ matrix.toxenv }}

test-dashboard-build:
needs: pretest
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v2
with:
node-version: 16
- name: Compile Dashboard
run: |
cd dashboard/src
yarn
yarn build
rm -rf ../build
mv build ..
cd ../../
- name: Set up Python 3.9
uses: actions/setup-python@v2
with:
python-version: 3.9
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -e .[test]
- name: Get gecko driver
run: |
wget https://github.com/mozilla/geckodriver/releases/download/v0.31.0/geckodriver-v0.31.0-linux64.tar.gz
tar -xvf geckodriver-v0.31.0-linux64.tar.gz
- name: Create gunicorn_tmp_dir to match orion_config.yaml
run: mkdir -p gunicorn_tmp_dir
- name: Launch backend
run: orion serve -c .github/workflows/orion/orion_config.yaml &
- name: Launch frontend
run: orion frontend &
- name: Install Firefox
uses: browser-actions/setup-firefox@latest
- name: Test frontend
run: PATH=$PATH:$(pwd) pytest tests/functional/serving/test_frontend.py

test:
needs: [pre-commit, pretest]
test:
runs-on: ${{ matrix.platform }}
strategy:
max-parallel: 4
matrix:
platform: [ubuntu-latest]
python-version: [3.7, 3.8, 3.9, '3.10']
python-version: [3.7]
env:
PLATFORM: ${{ matrix.platform }}
steps:
Expand All @@ -102,7 +34,7 @@ jobs:
python -m pip install --upgrade pip
pip install tox tox-gh-actions
- name: Test with tox (and all extra dependencies)
run: tox -e py-all
run: tox -e py-all -- tests/functional/example/test_speechbrain_tutorial.py
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
with:
Expand All @@ -111,243 +43,3 @@ jobs:
env_vars: PLATFORM,PYTHON
name: codecov-umbrella
fail_ci_if_error: false

test-long-algos:
needs: [pre-commit, pretest]
runs-on: ${{ matrix.platform }}
strategy:
matrix:
platform: [ubuntu-latest]
python-version: [3.7, 3.8, 3.9]
algo: [nevergrad, hebo, ax]
env:
PLATFORM: ${{ matrix.platform }}
steps:
- uses: actions/checkout@v1

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install tox tox-gh-actions

- name: Test long algorithms with tox
run: tox -e algo -- tests/unittests/algo/long/${{ matrix.algo }}

- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
with:
file: ./coverage.xml
flags: unittests
env_vars: PLATFORM,PYTHON
name: codecov-umbrella
fail_ci_if_error: false

test_no_extras:
needs: [pre-commit, pretest]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- name: Set up Python 3.9
uses: actions/setup-python@v2
with:
python-version: 3.9
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install tox tox-gh-actions
- name: Test with tox
run: tox -e py
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
with:
file: ./coverage.xml
flags: unittests
env_vars: PLATFORM,PYTHON
name: codecov-umbrella
fail_ci_if_error: false
mongodb:
needs: [pre-commit, pretest]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- name: Set up Python 3.9
uses: actions/setup-python@v2
with:
python-version: 3.9
- name: Setup MongoDB
uses: supercharge/[email protected]
with:
mongodb-version: 6.0

- name: Install MongoShell
run: |
wget -qO - https://www.mongodb.org/static/pgp/server-6.0.asc | sudo apt-key add -
echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/6.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-6.0.list
sudo apt-get update
sudo apt-get install -y mongodb-org

- name: Configure MongoDB
run: |
mongosh orion_test --eval 'db.createUser({user:"user",pwd:"pass",roles:["readWrite"]});'

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install tox tox-gh-actions
- name: Test with tox
run: tox -e mongodb
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
with:
file: ./coverage.xml
flags: backward
env_vars: PLATFORM,PYTHON
name: codecov-umbrella
fail_ci_if_error: false
backward-compatibility:
needs: [pre-commit, pretest]
runs-on: ubuntu-latest
strategy:
max-parallel: 2
matrix:
orion_db_type: [mongodb, pickleddb]
env:
ORION_DB_TYPE: ${{ matrix.orion_db_type }}
steps:
- uses: actions/checkout@v1
- name: Set up Python 3.9
uses: actions/setup-python@v2
with:
python-version: 3.9
- name: Setup MongoDB
uses: supercharge/[email protected]
with:
mongodb-version: 6.0

- name: Install MongoShell
run: |
wget -qO - https://www.mongodb.org/static/pgp/server-6.0.asc | sudo apt-key add -
echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/6.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-6.0.list
sudo apt-get update
sudo apt-get install -y mongodb-org

- name: Configure MongoDB
run: |
mongosh orion_test --eval 'db.createUser({user:"user",pwd:"pass",roles:["readWrite"]});'

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install tox tox-gh-actions
- name: Test with tox
run: tox -e backward-compatibility
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
with:
file: ./coverage.xml
flags: backward
env_vars: PLATFORM,PYTHON
name: codecov-umbrella
fail_ci_if_error: false
pypi:
needs: [test, backward-compatibility, test-long-algos, mongodb, test_no_extras, test-dashboard-build]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- name: Set up Python 3.9
uses: actions/setup-python@v2
with:
python-version: 3.9
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install tox tox-gh-actions
- name: Test packaging
run: tox -e packaging
- name: Build
run: tox -e build
- name: Test dashboard build deployment on normal install
run: |
# Get package path
export ORION_PACKAGE=$( realpath `find dist/ -type f` )
echo Package path: ${ORION_PACKAGE}
# Move to another folder (to prevent any confusion at installation with repo folder)
cd ~
echo Normal install
pip install ${ORION_PACKAGE}
# Get prefix
export ORION_PREFIX=$( python -c "import sys; print(sys.prefix);" )
echo Check if dashboard build is installed
if ( [ -d "${ORION_PREFIX}/orion-dashboard" ] ); then true; else false; fi
if ( [ -f "${ORION_PREFIX}/orion-dashboard/build/index.html" ] ); then true; else false; fi
if ( ls ${ORION_PREFIX}/orion-dashboard/build/static/js/main.*.js ); then true; else false; fi
echo Check if frontend script can find dashboard build
python -c "from orion.core.cli.frontend import get_dashboard_build_path; get_dashboard_build_path();"
echo Clean-up
pip uninstall -y orion
echo Check if dashboard build is correctly removed
# NB: It seems orion-dashboard build is not deleted,
# but it should be empty after uninstall
if ( [ -f "${ORION_PREFIX}/orion-dashboard/build/index.html" ] ); then false; fi
if ( ls ${ORION_PREFIX}/orion-dashboard/build/static/js/main.*.js ); then false; fi
echo End
cd -
- name: Test dashboard build deployment on user install
run: |
# Get package path
export ORION_PACKAGE=$( realpath `find dist/ -type f` )
echo Package path: ${ORION_PACKAGE}
# Move to another folder (to prevent any confusion at installation with repo folder)
cd ~
echo User install
pip install --user ${ORION_PACKAGE}
# Get prefix
export ORION_PREFIX=$( python -c "import site; print(site.USER_BASE);" )
echo Check if dashboard build is installed
if ( [ -d "${ORION_PREFIX}/orion-dashboard" ] ); then true; else false; fi
if ( [ -f "${ORION_PREFIX}/orion-dashboard/build/index.html" ] ); then true; else false; fi
if ( ls ${ORION_PREFIX}/orion-dashboard/build/static/js/main.*.js ); then true; else false; fi
echo Check if frontend script can find dashboard build
python -c "from orion.core.cli.frontend import get_dashboard_build_path; get_dashboard_build_path();"
echo Clean-up
pip uninstall -y orion
echo Check if dashboard build is correctly removed
# NB: In user install case, it seems folder orion-dashboard itself is not deleted,
# but it should be empty after uninstall
if ( [ -f "${ORION_PREFIX}/orion-dashboard/build/index.html" ] ); then false; fi
if ( ls ${ORION_PREFIX}/orion-dashboard/build/static/js/main.*.js ); then false; fi
echo End
cd -
- name: Publish distribution 📦 to Test PyPI
if: startsWith(github.ref, 'refs/tags')
uses: pypa/gh-action-pypi-publish@master
with:
user: __token__
password: ${{ secrets.test_pypi_password }}
repository_url: https://test.pypi.org/legacy/
- name: Publish distribution 📦 to PyPI
if: startsWith(github.ref, 'refs/tags')
uses: pypa/gh-action-pypi-publish@master
with:
user: __token__
password: ${{ secrets.pypi_password }}
conda:
needs: [test, backward-compatibility, test-long-algos, test-dashboard-build]
runs-on: ubuntu-latest
env:
ANACONDA_TOKEN: ${{ secrets.anaconda_token }}
steps:
- uses: actions/checkout@v1
- name: Set up Python 3.9
uses: actions/setup-python@v2
with:
python-version: 3.9
- name: Build conda
run: ./conda/ci_build.sh
- name: Publish distribution 📦 to Conda
if: startsWith(github.ref, 'refs/tags')
run: ./conda/upload.sh
Loading