Skip to content

Commit

Permalink
Merge pull request #17 from nf-core/dev
Browse files Browse the repository at this point in the history
PR for 1.0.0 release
  • Loading branch information
apeltzer authored Dec 9, 2019
2 parents f3f58f3 + 0946c41 commit 884e541
Show file tree
Hide file tree
Showing 26 changed files with 918 additions and 426 deletions.
4 changes: 2 additions & 2 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ We try to manage the required tasks for nf-core/scrnaseq using GitHub issues, yo

However, don't be put off by this template - other more general issues and suggestions are welcome! Contributions to the code are even more welcome ;)

> If you need help using or modifying nf-core/scrnaseq then the best place to ask is on the pipeline channel on [Slack](https://nf-core-invite.herokuapp.com/).
> If you need help using or modifying nf-core/scrnaseq then the best place to ask is on the pipeline channel on [Slack](https://nfcore.slack.com/channels/scrnaseq).
## Contribution workflow

Expand Down Expand Up @@ -46,4 +46,4 @@ These tests are run both with the latest available version of Nextflow and also

## Getting help

For further information/help, please consult the [nf-core/scrnaseq documentation](https://github.com/nf-core/scrnaseq#documentation) and don't hesitate to get in touch on the pipeline channel on [Slack](https://nf-core-invite.herokuapp.com/).
For further information/help, please consult the [nf-core/scrnaseq documentation](https://github.com/nf-core/scrnaseq#documentation) and don't hesitate to get in touch on the pipeline channel on [Slack](https://nfcore.slack.com/channels/scrnaseq).
4 changes: 0 additions & 4 deletions .github/markdownlint.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,5 @@
# Markdownlint configuration file
default: true,
line-length: false
no-multiple-blanks: 0
blanks-around-headers: false
blanks-around-lists: false
header-increment: false
no-duplicate-header:
siblings_only: true
16 changes: 16 additions & 0 deletions .github/workflows/branch.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: nfcore/scrnaseq branch protection
# This workflow is triggered on PRs to master branch on the repository
on:
pull_request:
branches:
- master

jobs:
test:
runs-on: ubuntu-latest
steps:
# PRs are only ok if coming from an nf-core dev branch
- uses: actions/checkout@v1
- name: Check PRs
run: |
[[ $(git remote get-url origin) == *nf-core/scrnaseq ]] && [ ${GITHUB_BASE_REF} = "master" ] && [ ${GITHUB_HEAD_REF} = "dev" ]
26 changes: 26 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: scrnaseq CI
# This workflow is triggered on pushes and PRs to the repository.
on: [push, pull_request]

jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
nxf_ver: ['19.10.0', '']
profile: ['test,docker', 'test_kallisto,docker', 'test,docker --aligner star']
steps:
- uses: actions/checkout@v1
- name: Install Nextflow
run: |
export NXF_VER=${{ matrix.nxf_ver }}
wget -qO- get.nextflow.io | bash
sudo mv nextflow /usr/local/bin/
- name: Download and tag image
run: |
docker pull nfcore/scrnaseq:dev
docker tag nfcore/scrnaseq:dev nfcore/scrnaseq:1.0.0
- name: "Test suites for all alignment methods"
run: |
nextflow run ${GITHUB_WORKSPACE} -profile ${{ matrix.profile }}
53 changes: 53 additions & 0 deletions .github/workflows/linting.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: nfcore/scrnaseq linting
# This workflow is triggered on pushes and PRs to the repository.
on: [push, pull_request]

jobs:
Markdown:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- uses: actions/setup-node@v1
with:
node-version: '10'
- name: Install markdownlint
run: |
npm install -g markdownlint-cli
- name: Run Markdownlint
run: |
markdownlint ${GITHUB_WORKSPACE} -c ${GITHUB_WORKSPACE}/.github/markdownlint.yml
YAML:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- uses: actions/setup-node@v1
with:
node-version: '10'
- name: Install yamllint
run: |
npm install -g yaml-lint
- name: Run yamllint
run: |
yamllint $(find ${GITHUB_WORKSPACE} -type f -name "*.yml")
nf-core:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- name: Install Nextflow
run: |
wget -qO- get.nextflow.io | bash
sudo mv nextflow /usr/local/bin/
- uses: actions/setup-python@v1
with:
python-version: '3.6'
architecture: 'x64'
- name: Install pip
run: |
sudo apt install python3-pip
pip install --upgrade pip
- name: Install nf-core tools
run: |
pip install nf-core
- name: Run nf-core lint
run: |
nf-core lint ${GITHUB_WORKSPACE}
16 changes: 10 additions & 6 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ matrix:

before_install:
# PRs to master are only ok if coming from dev branch
- '[ $TRAVIS_PULL_REQUEST = "false" ] || [ $TRAVIS_BRANCH != "master" ] || ([ $TRAVIS_PULL_REQUEST_SLUG = $TRAVIS_REPO_SLUG ] && [ $TRAVIS_PULL_REQUEST_BRANCH = "dev" ])'
- '[ $TRAVIS_PULL_REQUEST = "false" ] || [ $TRAVIS_BRANCH != "master" ] || ([ $TRAVIS_PULL_REQUEST_SLUG = $TRAVIS_REPO_SLUG ] && ([ $TRAVIS_PULL_REQUEST_BRANCH = "dev" ] || [ $TRAVIS_PULL_REQUEST_BRANCH = "patch" ]))'
# Pull the docker image first so the test doesn't wait for this
- docker pull nfcore/scrnaseq:dev
# Fake the tag locally so that the pipeline runs properly
# Looks weird when this is :dev to :dev, but makes sense when testing code for a release (:dev to :1.0.1)
- docker tag nfcore/scrnaseq:dev nfcore/scrnaseq:dev
- docker tag nfcore/scrnaseq:dev nfcore/scrnaseq:1.0.0

install:
# Install Nextflow
Expand All @@ -24,19 +24,23 @@ install:
# Install nf-core/tools
- pip install --upgrade pip
- pip install nf-core
# Install Conda
- wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
- bash Miniconda3-latest-Linux-x86_64.sh -b -f -p $HOME/miniconda
- export PATH="$HOME/miniconda/bin:$PATH"
# Reset
- mkdir ${TRAVIS_BUILD_DIR}/tests && cd ${TRAVIS_BUILD_DIR}/tests
# Install markdownlint-cli
- sudo apt-get install npm && npm install -g markdownlint-cli

env:
- NXF_VER='0.32.0' # Specify a minimum NF version that should be tested and work
- NXF_VER='' # Plus: get the latest NF version and check that it works
- NXF_VER='19.10.0' NXF_ANSI_LOG=0
- NXF_VER='' NXF_ANSI_LOG=0

script:
# Lint the pipeline code
- nf-core lint ${TRAVIS_BUILD_DIR}
# Lint the documentation
- markdownlint ${TRAVIS_BUILD_DIR} -c ${TRAVIS_BUILD_DIR}/.github/markdownlint.yml
# Run the pipeline with the test profile
# REFERENCE: Run the basic pipeline with the test profile
- nextflow run ${TRAVIS_BUILD_DIR} -profile test,docker
# All other tests run via GithUB Actions now
8 changes: 7 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,10 @@
# nf-core/scrnaseq: Changelog

## v1.0dev - [date]
## v1.0.0 - 2019-11-28 "Tiny Aluminium Crab"

Initial release of nf-core/scrnaseq, created with the [nf-core](http://nf-co.re/) template.
This includes the following workflow options:

* Salmon Alevin + AlevinQC
* STARSolo
* Kallisto / BUStools
2 changes: 1 addition & 1 deletion CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ This Code of Conduct applies both within project spaces and in public spaces whe

## Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team on [Slack](https://nf-core-invite.herokuapp.com/). The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team on [Slack](https://nf-co.re/join/slack). The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.

Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.

Expand Down
7 changes: 3 additions & 4 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
FROM nfcore/base
LABEL authors="Peter J Bailey, Alexander Peltzer" \
FROM nfcore/base:1.7
LABEL authors="Peter J Bailey, Alexander Peltzer, Olga Botvinnik" \
description="Docker image containing all requirements for nf-core/scrnaseq pipeline"

COPY environment.yml /
RUN conda env create -f /environment.yml && conda clean -a
ENV PATH /opt/conda/envs/nf-core-scrnaseq-1.0dev/bin:$PATH
ENV PATH /opt/conda/envs/nf-core-scrnaseq-1.0.0/bin:$PATH
10 changes: 0 additions & 10 deletions Makefile

This file was deleted.

21 changes: 12 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,17 @@
# nf-core/scrnaseq
# ![nf-core/scrnaseq](docs/images/nfcore-scrnaseq_logo.png)

**A pipeline for the processing of single-cell rnaseq data**.
**A fully automated Nextflow pipeline for Droplet-based (e.g. 10x Genomics) single-cell RNA-Seq data**.

[![Build Status](https://travis-ci.com/nf-core/scrnaseq.svg?branch=master)](https://travis-ci.com/nf-core/scrnaseq)
[![Nextflow](https://img.shields.io/badge/nextflow-%E2%89%A50.32.0-brightgreen.svg)](https://www.nextflow.io/)
[![Build Status](https://github.com/nf-core/scrnaseq/workflows/scrnaseq%20CI/badge.svg)](https://github.com/nf-core/scrnaseq/workflows/scrnaseq%20CI/badge.svg)
[![Nextflow](https://img.shields.io/badge/nextflow-%E2%89%A519.10.0-brightgreen.svg)](https://www.nextflow.io/)

[![install with bioconda](https://img.shields.io/badge/install%20with-bioconda-brightgreen.svg)](http://bioconda.github.io/)
[![Docker](https://img.shields.io/docker/automated/nfcore/scrnaseq.svg)](https://hub.docker.com/r/nfcore/scrnaseq)

[![Join us on Slack](https://img.shields.io/badge/slack-nfcore/scrnaseq-blue.svg)](https://nfcore.slack.com/channels/scrnaseq)

## Introduction

The pipeline is built using [Nextflow](https://www.nextflow.io), a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It comes with docker containers making installation trivial and results highly reproducible.

Work in progress - this is a community effort in building a pipeline capable to support:
Expand All @@ -30,15 +33,15 @@ The nf-core/scrnaseq pipeline comes with documentation about the pipeline, found
4. [Output and how to interpret the results](docs/output.md)
5. [Troubleshooting](https://nf-co.re/usage/troubleshooting)

<!-- TODO nf-core: Add a brief overview of what the pipeline does and how it works -->

## Credits

nf-core/scrnaseq was originally written by [Peter J. Bailey](https://github.com/PeterBailey) with some help from [Olga Botvinnik](https://github.com/olgabot) and [Alex Peltzer](https://github.com/apeltzer).
The `nf-core/scrnaseq` was initiated by [Peter J. Bailey](https://github.com/PeterBailey) (Salmon Alevin, AlevinQC) with major contributions from [Olga Botvinnik](https://github.com/olgabot) (STARsolo, Testdata) and [Alex Peltzer](https://github.com/apeltzer) (Kallisto/BusTools workflow).

## Citation

<!-- If you use nf-core/scrnaseq for your analysis, please cite it as follows: -->

You can cite the `nf-core` pre-print as follows:
Ewels PA, Peltzer A, Fillinger S, Alneberg JA, Patel H, Wilm A, Garcia MU, Di Tommaso P, Nahnsen S. **nf-core: Community curated bioinformatics pipelines**. *bioRxiv*. 2019. p. 610741. [doi: 10.1101/610741](https://www.biorxiv.org/content/10.1101/610741v1).

The basic benchmarks that were used as motivation for incorporating the three available modular workflows can be found in [this publication](https://www.biorxiv.org/content/10.1101/673285v2).

We offer all three paths for the processing of scRNAseq data so it remains up to the user to decide which pipeline workflow is chosen for a particular analysis question.
Binary file not shown.
11 changes: 0 additions & 11 deletions bin/alevin_qc.r
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,6 @@ if (length(args) < 3) {
stop("Usage: alevin_qc.r <baseDir> <sampleId> <outDir>", call.=FALSE)
}

print(R.Version())

if (!require("BiocManager")){
install.packages("BiocManager", dependencies=TRUE, repos='http://cloud.r-project.org/')
BiocManager::install()}

if (!require("alevinQC")) {
BiocManager::install("alevinQC")
}


require(alevinQC)
require(tximport)

Expand Down
Empty file modified bin/postprocessing.r
100644 → 100755
Empty file.
5 changes: 4 additions & 1 deletion bin/scrape_software_versions.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,22 @@
from collections import OrderedDict
import re

# TODO nf-core: Add additional regexes for new tools in process get_software_versions
regexes = {
'nf-core/scrnaseq': ['v_pipeline.txt', r"(\S+)"],
'Nextflow': ['v_nextflow.txt', r"(\S+)"],
'STAR': ['v_star.txt', r"(\S+)"],
'Salmon': ['v_salmon.txt', r"salmon (\S+)"],
'Kallisto': ['v_kallisto.txt', r"kallisto, version (\S+)"],
'BUStools': ['v_bustools.txt', r"bustools (\S+)"],
'MultiQC': ['v_multiqc.txt', r"multiqc, version (\S+)"],
}
results = OrderedDict()
results['nf-core/scrnaseq'] = '<span style="color:#999999;\">N/A</span>'
results['Nextflow'] = '<span style="color:#999999;\">N/A</span>'
results['STAR'] = '<span style="color:#999999;\">N/A</span>'
results['Salmon'] = '<span style="color:#999999;\">N/A</span>'
results['Kallisto'] = '<span style="color:#999999;\">N/A</span>'
results['BUStools'] = '<span style="color:#999999;\">N/A</span>'
results['MultiQC'] = '<span style="color:#999999;\">N/A</span>'

# Search each file using its regex
Expand Down
99 changes: 99 additions & 0 deletions bin/t2g.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
#!/usr/bin/env python
#This was downloaded on 2019-06-23 from https://github.com/bustools/getting_started/releases/
#All credit goes to the original authors from the Kallisto/BUStools team!
# BSD 2-Clause License
#
# Copyright (c) 2017, Nicolas Bray, Harold Pimentel, Páll Melsted and Lior Pachter
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright notice, this
# list of conditions and the following disclaimer.
#
# * Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

import sys, argparse

def create_transcript_list(input, use_name = True, use_version = False):
r = {}
for line in input:
if len(line) == 0 or line[0] == '#':
continue
l = line.strip().split('\t')
if l[2] == 'transcript':
info = l[8]
d = {}
for x in info.split('; '):
x = x.strip()
p = x.find(' ')
if p == -1:
continue
k = x[:p]
p = x.find('"',p)
p2 = x.find('"',p+1)
v = x[p+1:p2]
d[k] = v


if 'transcript_id' not in d or 'gene_id' not in d:
continue

tid = d['transcript_id'].split(".")[0]
gid = d['gene_id'].split(".")[0]
if use_version:
if 'transcript_version' not in d or 'gene_version' not in d:
continue

tid += '.' + d['transcript_version']
gid += '.' + d['gene_version']
gname = None
if use_name:
if 'gene_name' not in d:
continue
gname = d['gene_name']

if tid in r:
continue

r[tid] = (gid, gname)
return r



def print_output(output, r, use_name = True):
for tid in r:
if use_name:
output.write("%s\t%s\t%s\n"%(tid, r[tid][0], r[tid][1]))
else:
output.write("%s\t%s\n"%(tid, r[tid][0]))


if __name__ == "__main__":


parser = argparse.ArgumentParser(add_help=True, description='Creates transcript to gene info from GTF files\nreads from standard input and writes to standard output')
parser.add_argument('--use_version', '-v', action='store_true', help='Use version numbers in transcript and gene ids')
parser.add_argument('--skip_gene_names', '-s', action='store_true', help='Do not output gene names')
args = parser.parse_args()



input = sys.stdin
r = create_transcript_list(input, use_name = not args.skip_gene_names, use_version = args.use_version)
output = sys.stdout
print_output(output, r)
Loading

0 comments on commit 884e541

Please sign in to comment.