Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add nf-test to CI #51

Merged
merged 10 commits into from
Oct 29, 2024
Merged

Add nf-test to CI #51

merged 10 commits into from
Oct 29, 2024

Conversation

kedhammar
Copy link

@kedhammar kedhammar commented Oct 29, 2024 •
edited
Loading

PR checklist

  • This comment contains a description of changes (with reason).
  • [ ] If you've fixed a bug or added code that should be tested, add tests!
  • [ ] If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • [ ] If necessary, also make a PR on the nf-core/seqinspector branch on the nf-core/test-datasets repository.
  • Make sure your code lints (nf-core lint).
  • Ensure the test suite passes (nf-test test main.nf.test -profile test,docker).
  • [ ] Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
  • [ ] Usage Documentation in docs/usage.md is updated.
  • [ ] Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • [] README.md` is updated (including new tool citations and authors/contributors).

@kedhammar kedhammar self-assigned this Oct 29, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 29, 2024 •
edited
Loading

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 69825df

+| ✅ 189 tests passed       |+
!| ❗  21 tests had warnings |!

❗ Test warnings:

  • readme - README contains the placeholder zenodo.XXXXXXX. This should be replaced with the zenodo doi (after the first release).
  • pipeline_todos - TODO string in main.nf: Remove this line if you don't need a FASTA file
  • pipeline_todos - TODO string in nextflow.config: Specify your pipeline's command line flags
  • pipeline_todos - TODO string in nextflow.config: Optionally, you can add a pipeline-specific nf-core config at https://github.com/nf-core/configs
  • pipeline_todos - TODO string in README.md: TODO nf-core:
  • pipeline_todos - TODO string in README.md: Include a figure that guides the user through the major workflow steps. Many nf-core
  • pipeline_todos - TODO string in README.md: Fill in short bullet-pointed list of the default steps in the pipeline
  • pipeline_todos - TODO string in README.md: Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file.
  • pipeline_todos - TODO string in README.md: Add bibliography of tools and data used in your pipeline
  • pipeline_todos - TODO string in usage.md: Add documentation about anything specific to running your pipeline. For general topics, please point to (and add to) the main nf-core website.
  • pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
  • pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
  • pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
  • pipeline_todos - TODO string in test_full.config: Specify the paths to your full test data ( on nf-core/test-datasets or directly in repositories, e.g. SRA)
  • pipeline_todos - TODO string in test_full.config: Give any required params for the test so that command line flags are not needed
  • pipeline_todos - TODO string in test.config: Specify the paths to your test data on nf-core/test-datasets
  • pipeline_todos - TODO string in test.config: Give any required params for the test so that command line flags are not needed
  • pipeline_todos - TODO string in base.config: Check the defaults for all processes
  • pipeline_todos - TODO string in base.config: Customise requirements for specific processes.
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
  • pipeline_todos - TODO string in awsfulltest.yml: You can customise AWS full pipeline tests as required

✅ Tests passed:

Run details

  • nf-core/tools version 3.0.2
  • Run at 2024-10-29 14:49:10

@kedhammar
Copy link
Author

Since I've opted for conservative nf-test runs I'm running it manually to make sure we have full coverage.

I’m running into a single, very simple assertion error for the utils_nextflow_pipeline subworkflow.

  Test [dd2e962c] 'Should print version' Assertion failed: 

assert workflow.stdout.contains("nextflow_workflow v9.9.9")
       |        |      |
       workflow []     false

FAILED (8.614s)

  Assertion failed: 
  
  1 of 2 assertions failed
  
  Nextflow stdout:
  
  Nextflow stderr:

This is after running nf-core submodules update as well. I've asked about it in the nf-test slack channel now but it might not be our problem.

@kedhammar kedhammar requested review from Aratz and maxulysse October 29, 2024 10:56
@kedhammar
Copy link
Author

Ping @maxulysse

@kedhammar
Copy link
Author

kedhammar commented Oct 29, 2024 •
edited
Loading

I’m running into a single, very simple assertion error for the utils_nextflow_pipeline subworkflow.

Got confirmation from @mirpedrol it's an issue on the nf-core/modules repo

@Aratz
Copy link
Collaborator

Aratz commented Oct 29, 2024

Awesome! I've tried to make some changes on my own repo to test it but I get this weird error in the ci 🤔 Is this something you've encountered yourself?
https://github.com/Aratz/seqinspector/actions/runs/11573234518/job/32214895326

@Aratz
Copy link
Collaborator

Aratz commented Oct 29, 2024

I think we might want to run all the tests all the time actually: if you modify some files, run the test and they fail, and then do some other modifications that don't trigger the tests again, it will not run anything and still show a green mark, which may be misleading, see https://github.com/Aratz/seqinspector/actions/runs/11573281505/job/32215045767, I just wanted to reduce the number of shards to 2 but it didn't run the tests because modifying the ci file does not trigger the tests...

maxulysse
maxulysse reviewed Oct 29, 2024
View reviewed changes
.github/workflows/nf-test.yml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not in the ci.yml file?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ci.yml file runs the whole pipeline if I understand correctly.

I'd prefer to have them as separate GHA checks, if they do different things.

@kedhammar
Copy link
Author

I’m running into a single, very simple assertion error for the utils_nextflow_pipeline subworkflow.

Got confirmation from @mirpedrol it's an issue on the nf-core/modules repo

Fixed here nf-core/modules#6887 and applied in 3443734.

All tests now pass on gitpod 🚀

Aratz
Aratz approved these changes Oct 29, 2024
View reviewed changes
Copy link
Collaborator

@Aratz Aratz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

Just one thing is that now it's running the tests for the modules we're using, even though these are supposed to be validated in https://github.com/nf-core/modules/. I think for now it's fine but it might get very slow as we add more and more modules. Maybe we can look into how to restrict nf-test. I tried to look it up now but didn't manage to go anywhere...

@kedhammar kedhammar merged commit a2d589f into nf-core:dev Oct 29, 2024
10 checks passed
@kedhammar kedhammar deleted the ci-nf-test branch October 29, 2024 15:49
@kedhammar kedhammar mentioned this pull request Oct 29, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@maxulysse maxulysse maxulysse left review comments

@Aratz Aratz Aratz approved these changes

Assignees

@kedhammar kedhammar

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@kedhammar @Aratz @maxulysse