Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add nf-test to CI #51

Merged
merged 10 commits into from
Oct 29, 2024
Merged

Add nf-test to CI #51

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
0f7ff12
Add CI config for parallellized and economical nf-test runs, ripped f…
kedhammar Oct 29, 2024
389c50b
Add nf-test config to trigger full test on changes to critical files
kedhammar Oct 29, 2024
38b0e45
prettier lint
kedhammar Oct 29, 2024
b452763
Bump changelog
kedhammar Oct 29, 2024
89ca390
Add modules testdata base path to test config and trim out resource l…
kedhammar Oct 29, 2024
3443734
nf-core subworkflows update to apply fix
kedhammar Oct 29, 2024
ee42aa9
simplify CI, always run
kedhammar Oct 29, 2024
4705376
try specifying cwd as arg in nf-test command
kedhammar Oct 29, 2024
222a986
harmonize syntax and correct comment
kedhammar Oct 29, 2024
69825df
remove multiqc versions file from snapshot assertion, will fail if te…
kedhammar Oct 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions .github/workflows/nf-test.yml
View file
Open in desktop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not in the ci.yml file?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ci.yml file runs the whole pipeline if I understand correctly.

I'd prefer to have them as separate GHA checks, if they do different things.

Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: nf-test

on: [push, pull_request]

jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
shard: [1, 2, 3, 4]
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up JDK 11
uses: actions/setup-java@v2
with:
java-version: "11"
distribution: "adopt"

- name: Setup Nextflow latest-edge
uses: nf-core/setup-nextflow@v1
with:
version: "latest-edge"

- name: Install nf-test
run: |
wget -qO- https://get.nf-test.com | bash
sudo mv nf-test /usr/local/bin/

- name: Run Tests (Shard ${{ matrix.shard }}/${{ strategy.job-total }})
run: nf-test test --ci --shard ${{ matrix.shard }}/${{ strategy.job-total }} .
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ Initial release of nf-core/seqinspector, created with the [nf-core](https://nf-c
- [#20](https://github.com/nf-core/seqinspector/pull/20) Use tags to generate group reports
- [#13](https://github.com/nf-core/seqinspector/pull/13) Generate reports per run, per project and per lane.
- [#49](https://github.com/nf-core/seqinspector/pull/49) Merge with template 3.0.2.
- [#51](https://github.com/nf-core/seqinspector/pull/51) Add nf-test to CI.

### `Fixed`

Expand Down
4 changes: 2 additions & 2 deletions modules.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
"nf-core": {
"utils_nextflow_pipeline": {
"branch": "master",
"git_sha": "3aa0aec1d52d492fe241919f0c6100ebf0074082",
"git_sha": "56372688d8979092cafbe0c5c3895b491166ca1c",
"installed_by": ["subworkflows"]
},
"utils_nfcore_pipeline": {
Expand All @@ -31,7 +31,7 @@
},
"utils_nfschema_plugin": {
"branch": "master",
"git_sha": "bbd5a41f4535a8defafe6080e00ea74c45f4f96c",
"git_sha": "2fd2cd6d0e7b273747f32e465fdc6bcc3ae0814e",
"installed_by": ["subworkflows"]
}
}
Expand Down
10 changes: 6 additions & 4 deletions subworkflows/nf-core/utils_nextflow_pipeline/tests/main.workflow.nf.test
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions subworkflows/nf-core/utils_nfschema_plugin/tests/main.nf.test
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion tests/MiSeq.main.nf.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ nextflow_pipeline {
path("$outputDir/multiqc/global_report/multiqc_data/multiqc_citations.txt"),
path("$outputDir/multiqc/global_report/multiqc_data/multiqc_fastqc.txt"),
path("$outputDir/multiqc/global_report/multiqc_data/multiqc_general_stats.txt"),
path("$outputDir/multiqc/global_report/multiqc_data/multiqc_software_versions.txt"),
).match()
}
)
Expand Down
5 changes: 2 additions & 3 deletions tests/MiSeq.main.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,12 @@
"content": [
"multiqc_citations.txt:md5,4c806e63a283ec1b7e78cdae3a923d4f",
"multiqc_fastqc.txt:md5,7b1b7fd457b60404768045b148d4c0a8",
"multiqc_general_stats.txt:md5,5b28a83b14cb2fe88d084d08900ebdbf",
"multiqc_software_versions.txt:md5,a3698a2d32e8695c38d50e3d17de5fe3"
"multiqc_general_stats.txt:md5,5b28a83b14cb2fe88d084d08900ebdbf"
],
"meta": {
"nf-test": "0.9.0",
"nextflow": "24.10.0"
},
"timestamp": "2024-10-28T13:18:10.3675973"
}
}
}
5 changes: 0 additions & 5 deletions tests/NovaSeq6000.main.nf.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,27 +21,22 @@ nextflow_pipeline {
path("$outputDir/multiqc/global_report/multiqc_data/multiqc_citations.txt"),
path("$outputDir/multiqc/global_report/multiqc_data/multiqc_fastqc.txt"),
path("$outputDir/multiqc/global_report/multiqc_data/multiqc_general_stats.txt"),
path("$outputDir/multiqc/global_report/multiqc_data/multiqc_software_versions.txt"),

path("$outputDir/multiqc/group_reports/lane1/multiqc_data/multiqc_citations.txt"),
path("$outputDir/multiqc/group_reports/lane1/multiqc_data/multiqc_fastqc.txt"),
path("$outputDir/multiqc/group_reports/lane1/multiqc_data/multiqc_general_stats.txt"),
path("$outputDir/multiqc/group_reports/lane1/multiqc_data/multiqc_software_versions.txt"),

path("$outputDir/multiqc/group_reports/group1/multiqc_data/multiqc_citations.txt"),
path("$outputDir/multiqc/group_reports/group1/multiqc_data/multiqc_fastqc.txt"),
path("$outputDir/multiqc/group_reports/group1/multiqc_data/multiqc_general_stats.txt"),
path("$outputDir/multiqc/group_reports/group1/multiqc_data/multiqc_software_versions.txt"),

path("$outputDir/multiqc/group_reports/group2/multiqc_data/multiqc_citations.txt"),
path("$outputDir/multiqc/group_reports/group2/multiqc_data/multiqc_fastqc.txt"),
path("$outputDir/multiqc/group_reports/group2/multiqc_data/multiqc_general_stats.txt"),
path("$outputDir/multiqc/group_reports/group2/multiqc_data/multiqc_software_versions.txt"),

path("$outputDir/multiqc/group_reports/test/multiqc_data/multiqc_citations.txt"),
path("$outputDir/multiqc/group_reports/test/multiqc_data/multiqc_fastqc.txt"),
path("$outputDir/multiqc/group_reports/test/multiqc_data/multiqc_general_stats.txt"),
path("$outputDir/multiqc/group_reports/test/multiqc_data/multiqc_software_versions.txt"),
).match()
},
)
Expand Down
9 changes: 2 additions & 7 deletions tests/NovaSeq6000.main.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,28 +4,23 @@
"multiqc_citations.txt:md5,4c806e63a283ec1b7e78cdae3a923d4f",
"multiqc_fastqc.txt:md5,3730f9046b20ac5c17a86db0a33f8d5d",
"multiqc_general_stats.txt:md5,25abe0f6a35eb4a3b056fc3cf5c13732",
"multiqc_software_versions.txt:md5,a3698a2d32e8695c38d50e3d17de5fe3",
"multiqc_citations.txt:md5,4c806e63a283ec1b7e78cdae3a923d4f",
"multiqc_fastqc.txt:md5,8284e25ccc21041cf3b5a32eb6a51e78",
"multiqc_general_stats.txt:md5,90ee35137492b80aab36ef67f72d8921",
"multiqc_software_versions.txt:md5,a3698a2d32e8695c38d50e3d17de5fe3",
"multiqc_citations.txt:md5,4c806e63a283ec1b7e78cdae3a923d4f",
"multiqc_fastqc.txt:md5,f38ffdc112c73af3a41ed15848a3761f",
"multiqc_general_stats.txt:md5,d62a2fc39e674d98783d408791803148",
"multiqc_software_versions.txt:md5,a3698a2d32e8695c38d50e3d17de5fe3",
"multiqc_citations.txt:md5,4c806e63a283ec1b7e78cdae3a923d4f",
"multiqc_fastqc.txt:md5,7ff71ceb8ecdf086331047f8860c3347",
"multiqc_general_stats.txt:md5,2f09b8f199ac40cf67ba50843cebd29c",
"multiqc_software_versions.txt:md5,a3698a2d32e8695c38d50e3d17de5fe3",
"multiqc_citations.txt:md5,4c806e63a283ec1b7e78cdae3a923d4f",
"multiqc_fastqc.txt:md5,519ff344a896ac369bba4d5c5b8be7b5",
"multiqc_general_stats.txt:md5,6a1c16f068d7ba3a9225a17eb570ed9a",
"multiqc_software_versions.txt:md5,a3698a2d32e8695c38d50e3d17de5fe3"
"multiqc_general_stats.txt:md5,6a1c16f068d7ba3a9225a17eb570ed9a"
],
"meta": {
"nf-test": "0.9.0",
"nextflow": "24.10.0"
},
"timestamp": "2024-10-28T13:19:13.226135825"
}
}
}
1 change: 0 additions & 1 deletion tests/PromethION.main.nf.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ nextflow_pipeline {
path("$outputDir/multiqc/global_report/multiqc_data/multiqc_citations.txt"),
path("$outputDir/multiqc/global_report/multiqc_data/multiqc_fastqc.txt"),
path("$outputDir/multiqc/global_report/multiqc_data/multiqc_general_stats.txt"),
path("$outputDir/multiqc/global_report/multiqc_data/multiqc_software_versions.txt"),
).match()
},
)
Expand Down
5 changes: 2 additions & 3 deletions tests/PromethION.main.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,12 @@
"content": [
"multiqc_citations.txt:md5,4c806e63a283ec1b7e78cdae3a923d4f",
"multiqc_fastqc.txt:md5,1a4b472e13cadc770832b0e20d1de7b0",
"multiqc_general_stats.txt:md5,409cefc7f17f95d176ced6032bf8fb32",
"multiqc_software_versions.txt:md5,a3698a2d32e8695c38d50e3d17de5fe3"
"multiqc_general_stats.txt:md5,409cefc7f17f95d176ced6032bf8fb32"
],
"meta": {
"nf-test": "0.9.0",
"nextflow": "24.10.0"
},
"timestamp": "2024-10-28T13:19:57.261730412"
}
}
}
8 changes: 1 addition & 7 deletions tests/nextflow.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,8 @@ params {
config_profile_name = 'nf-test profile'
config_profile_description = 'Configuration profile to use for nf-test.'

// Limit resources so that this can run on GitHub Actions
max_cpus = 2
max_memory = '3.GB'
max_time = '2.h'

pipelines_testdata_base_path = 'https://raw.githubusercontent.com/nf-core/test-datasets/'
modules_testdata_base_path = 's3://ngi-igenomes/testdata/nf-core/modules/'

validationSchemaIgnoreParams = 'genomes,igenomes_base,pipelines_testdata_base_path'


}
10 changes: 5 additions & 5 deletions workflows/seqinspector.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,10 +87,10 @@ workflow SEQINSPECTOR {
.mix(ch_multiqc_extra_files)
.collect(),
ch_multiqc_config.toList(),
Channel.empty().toList(),
[],
ch_multiqc_logo.toList(),
Channel.empty().toList(),
Channel.empty().toList()
[],
[]
)

ch_tags = ch_multiqc_files
Expand Down Expand Up @@ -138,9 +138,9 @@ workflow SEQINSPECTOR {
)

emit:
global_report = MULTIQC_GLOBAL.out.report.toList() // channel: /path/to/multiqc_report.html
global_report = MULTIQC_GLOBAL.out.report.toList() // channel: [ /path/to/multiqc_report.html ]
grouped_reports = MULTIQC_PER_TAG.out.report.toList() // channel: [ /path/to/multiqc_report.html ]
versions = ch_versions // channel: [ path(versions.yml) ]
versions = ch_versions // channel: [ path(versions.yml) ]
}

/*
Expand Down
Loading