Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate CleanVcf into GATK #733

Draft
wants to merge 31 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
be51214
Initial commit
kjaisingh Oct 17, 2024
28d2062
Merge branch 'main' into kj_sv_cleanvcf
kjaisingh Oct 30, 2024
d20d5b8
E2E working in local integration tests
kjaisingh Oct 31, 2024
7eaa215
Furhter changes - reverted changes to 1b, removed 5
kjaisingh Nov 1, 2024
1ad3c1c
More files removed...
kjaisingh Nov 1, 2024
cbcf76e
Resolve merge
kjaisingh Nov 1, 2024
db6b9b8
Minor changes to merge with latest changes
kjaisingh Nov 1, 2024
e70ec78
Modified java_mem_gb to use select_first
kjaisingh Nov 1, 2024
2cb19ca
Clean up CleanVcf.wdl inputs
kjaisingh Nov 1, 2024
0956b92
Forgot comma
kjaisingh Nov 1, 2024
48d81a8
Forgot comma
kjaisingh Nov 1, 2024
f4112a2
Removed unnecessary params
kjaisingh Nov 1, 2024
ad93a92
Added runtime_override_clean_vcf_5
kjaisingh Nov 1, 2024
559a2ce
Minor changes
kjaisingh Nov 4, 2024
df05435
Merge branch 'main' into kj_sv_cleanvcf
kjaisingh Nov 12, 2024
e197f0a
Resolved merge conflicts
kjaisingh Nov 22, 2024
43db008
WIP
kjaisingh Nov 24, 2024
e6e519d
Initial preprocess script
kjaisingh Nov 25, 2024
ccd5dde
Created postprocessing script
kjaisingh Nov 25, 2024
d60b01b
Minor removal of > character
kjaisingh Nov 25, 2024
38eefb2
Python linting fixes
kjaisingh Nov 25, 2024
be7f3bf
WIP - refactored WDLs to use new set of tools
kjaisingh Dec 5, 2024
fec8d59
WIP
kjaisingh Dec 6, 2024
e27026b
Updated pre/postprocess to use pipeline docker
kjaisingh Dec 11, 2024
a92904d
Finished testing cleanvcfpreprocess
kjaisingh Dec 12, 2024
766acf6
Undo header.txt addition
kjaisingh Dec 12, 2024
05a17c2
Modified postprocess function, works now
kjaisingh Dec 12, 2024
d403387
Removed header.txt again
kjaisingh Dec 12, 2024
170ec7c
Updated header writing
kjaisingh Dec 13, 2024
eed4c57
Decommissioned post-process script
kjaisingh Dec 13, 2024
078155c
Updated vargq values
kjaisingh Dec 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/.dockstore.yml
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,7 @@ workflows:
filters:
branches:
- main
- kj_sv_cleanvcf
tags:
- /.*/

Expand Down
12 changes: 0 additions & 12 deletions dockerfiles/sv-pipeline/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -70,13 +70,9 @@ RUN plink2 || true
# -Compile StitchFragmentedCNVs Java program
# -Compile StitchFragmentedCNVs unit tests
# -Compile VCFParser unit tests
# -Compile and test CleanVCFPart1 Java program
# -Compile and test CleanVCFPart1 unit tests
ENV STITCH_JAR="/opt/sv-pipeline/java/build/StitchFragmentedCNVs.jar"
ARG STITCH_UNIT_TEST_JAR="/opt/sv-pipeline/java/build/StitchFragmentedCNVsUnitTest.jar"
ARG VCF_PARSER_UNIT_TEST_JAR="/opt/sv-pipeline/java/build/VCFParserUnitTest.jar"
ENV CLEAN_VCF_PART_1_JAR="/opt/sv-pipeline/java/build/CleanVCFPart1.jar"
ARG CLEAN_VCF_PART_1_UNIT_TEST_JAR="/opt/sv-pipeline/java/build/CleanVCFPart1UnitTest.jar"
ARG BUILD_DEPS="openjdk-8-jdk"
ARG DEBIAN_FRONTEND=noninteractive
RUN export APT_TRANSIENT_PACKAGES=$(diff_of_lists.sh "$BUILD_DEPS" $APT_REQUIRED_PACKAGES) && \
Expand All @@ -97,14 +93,6 @@ RUN export APT_TRANSIENT_PACKAGES=$(diff_of_lists.sh "$BUILD_DEPS" $APT_REQUIRED
echo "Running VCFParserUnitTest..." && \
java -enableassertions -jar $VCF_PARSER_UNIT_TEST_JAR && \
rm -rf build/classes/* $VCF_PARSER_UNIT_TEST_JAR && \
javac -d build/classes org/broadinstitute/svpipeline/CleanVCFPart1.java org/broadinstitute/svpipeline/VCFParser.java && \
jar cfe build/CleanVCFPart1.jar "org.broadinstitute.svpipeline.CleanVCFPart1" -C build/classes . && \
rm -rf build/classes/* && \
javac -d build/classes org/broadinstitute/svpipeline/CleanVCFPart1UnitTest.java org/broadinstitute/svpipeline/CleanVCFPart1.java org/broadinstitute/svpipeline/VCFParser.java && \
jar cfe build/CleanVCFPart1UnitTest.jar "org.broadinstitute.svpipeline.CleanVCFPart1UnitTest" -C build/classes . && \
echo "Running CleanVCFPart1UnitTest..." && \
java -enableassertions -jar $CLEAN_VCF_PART_1_UNIT_TEST_JAR && \
rm -rf build/classes/* $CLEAN_VCF_PART_1_UNIT_TEST_JAR && \
apt-get -qqy remove --purge $APT_TRANSIENT_PACKAGES && \
apt-get -qqy autoremove --purge && \
apt-get -qqy clean && \
Expand Down
6 changes: 3 additions & 3 deletions inputs/values/dockers.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@
"samtools_cloud_docker": "us.gcr.io/broad-dsde-methods/gatk-sv/samtools-cloud:2024-10-25-v0.29-beta-5ea22a52",
"sv_base_docker": "us.gcr.io/broad-dsde-methods/gatk-sv/sv-base:2024-10-25-v0.29-beta-5ea22a52",
"sv_base_mini_docker": "us.gcr.io/broad-dsde-methods/gatk-sv/sv-base-mini:2024-10-25-v0.29-beta-5ea22a52",
"sv_pipeline_docker": "us.gcr.io/broad-dsde-methods/gatk-sv/sv-pipeline:2024-11-15-v1.0-488d7cb0",
"sv_pipeline_qc_docker": "us.gcr.io/broad-dsde-methods/gatk-sv/sv-pipeline:2024-11-15-v1.0-488d7cb0",
"sv_pipeline_docker": "us-central1-docker.pkg.dev/talkowski-training/kj-development/sv-pipeline:kj-clean-vcf-170ec7c38b004f02639b5f82270113b74162fe79",
"sv_pipeline_qc_docker": "us-central1-docker.pkg.dev/talkowski-training/kj-development/sv-pipeline:kj-clean-vcf-170ec7c38b004f02639b5f82270113b74162fe79",
"wham_docker": "us.gcr.io/broad-dsde-methods/gatk-sv/wham:2024-10-25-v0.29-beta-5ea22a52",
"igv_docker": "us.gcr.io/broad-dsde-methods/gatk-sv/igv:mw-xz-fixes-2-b1be6a9",
"duphold_docker": "us.gcr.io/broad-dsde-methods/gatk-sv/duphold:mw-xz-fixes-2-b1be6a9",
Expand All @@ -28,5 +28,5 @@
"sv_utils_docker": "us.gcr.io/broad-dsde-methods/gatk-sv/sv-utils:2024-10-25-v0.29-beta-5ea22a52",
"gq_recalibrator_docker": "us.gcr.io/broad-dsde-methods/markw/gatk:mw-tb-form-sv-filter-training-data-899360a",
"str": "us.gcr.io/broad-dsde-methods/gatk-sv/str:2023-05-23-v0.27.3-beta-e537bdd6",
"denovo": "us.gcr.io/broad-dsde-methods/gatk-sv/denovo:2024-11-15-v1.0-488d7cb0"
"denovo": "us-central1-docker.pkg.dev/talkowski-training/kj-development/denovo:kj-clean-vcf-170ec7c38b004f02639b5f82270113b74162fe79"
}

This file was deleted.

This file was deleted.

Loading
Loading