BasicQC1-error stream

[GoogleCodeExporter]

Hi Ivo,

I ran  also the BasicQC1. Here the problem is still the script from my 
understanding.  Please find attached the module and the script to double check 
if there is some info missing(I changed the paths and nothing else more).

thanks,

Fed

Original issue reported on code.google.com by [email protected] on 20 Jul 2011 at 6:49

Attachments:

• Basic_QC1_UCI.pipe
• [Screen shot 2011-07-20 at 11.23.42 AM.png](https://storage.googleapis.com/google-code-attachments/informatics/issue-4/comment-0/Screen shot 2011-07-20 at 11.23.42 AM.png)
• [Screen shot 2011-07-19 at 6.01.05 PM.png](https://storage.googleapis.com/google-code-attachments/informatics/issue-4/comment-0/Screen shot 2011-07-19 at 6.01.05 PM.png)
• samtools_calmd_pipeline.sh

[GoogleCodeExporter]

Sorry the *.sh script did not get attached. Perhaps change the file extension 
to TXT or wrap it in a ZIP file and repost.

What was the reason to suspect the script was the problem? It seems like 
SAMtools is generating a floating point runtime exception, no?

Original comment by [email protected] on 20 Jul 2011 at 11:42

• Changed state: Started

[GoogleCodeExporter]

I attach here the zipped script. I was asking bc a line of the script was 
recalled..so I interpreted that was a script problem. This "floating point 
runtime exception" means that is a problem of samtools? I will try also with 
the new samtools build.

fed

Original comment by [email protected] on 20 Jul 2011 at 11:51

Attachments:

• samtools_calmd_pipeline.sh.zip

[GoogleCodeExporter]

I ran this module also with an upgraded version of samtools, but I have back 
the same error.

Original comment by [email protected] on 21 Jul 2011 at 9:58

[GoogleCodeExporter]

Hi,

I haven't had any feedback from andrew on this issue, let me know if on your 
server you are experiencing the same issues. I re-run on fgene2 also, but I got 
back the same floating error(see attached).

Even using command line:

/applications/samtools-0.1.7_x86_64-linux/samtools calmd -b 
/projects/pipelineCache/pipeline/2011August19_10h51m17s724ms/FixMateInformation_
1.Output-1 
/projects1/Reference_genomes/hg18_ensembl/gatk-canonical/gatk-hg18_ensembl.fa 
/projects/pipelineCache/pipeline/2011August19_10h51m17s724ms/SamToolsCamldMDtag_
1.OutputNo-DuplicatesBAMfile-1.bam
Floating exception

Fed

Original comment by [email protected] on 19 Aug 2011 at 8:13

Attachments:

• [Screen shot 2011-08-19 at 1.09.41 PM.png](https://storage.googleapis.com/google-code-attachments/informatics/issue-4/comment-4/Screen shot 2011-08-19 at 1.09.41 PM.png)
• [Screen shot 2011-08-19 at 1.09.26 PM.png](https://storage.googleapis.com/google-code-attachments/informatics/issue-4/comment-4/Screen shot 2011-08-19 at 1.09.26 PM.png)
• Basic_QC1_UCI.pipe
• Basic_QC2_UCI.pipe

[GoogleCodeExporter]

Sorry for not responding. We are experiencing the same floating point error on 
our servers. Still looking into solutions to this.

Sam

Original comment by [email protected] on 22 Aug 2011 at 7:12

[GoogleCodeExporter]

Thanks,

Fed

Original comment by [email protected] on 26 Aug 2011 at 8:00

[GoogleCodeExporter]

The solution to this problem is that you need full permission to the reference 
file. I cannot change the permissions to this file so instead, I created a 
symbolic link to the it and used the link as my data source input. 

This is the command for symbolic linkage: "ln -s 
/projects1/Reference_genomes/hg18_ensembl/gatk-canonical/gatk-hg18_ensembl.fa 
/projects1/test_pipeline/NEW_ORGANIZATION/Basic_QC/gatk-hg18_ensembl.fa"

I then put 
/projects1/test_pipeline/NEW_ORGANIZATION/Basic_QC/gatk-hg18_ensembl.fa in the 
data source. 

I modified the workflow significantly to make it simpler and attached it here. 
I could even add a module that does the linking automatically, or you can just 
make sure you have full permissions to the reference file. Whatever you want.  

Sam

Original comment by [email protected] on 14 Sep 2011 at 9:46

Attachments:

• fgene1_BasicQC1.pipe

[GoogleCodeExporter]

More accurately, you need write permission to the directory that the reference 
file is in. You don't need full permission to the reference file. This is why 
symbolically linking the reference file to a folder you can write to fixes the 
problem.

Original comment by [email protected] on 14 Sep 2011 at 10:42

[GoogleCodeExporter]

Oh Sam,

thank you so much! I'm gonna test it right away!

fed

Original comment by [email protected] on 15 Sep 2011 at 5:11

[GoogleCodeExporter]

Hi,

what does it mean when a run is backlogged? is it due to any errors? It's 
happening to me for both the QC modules..

fed

Original comment by [email protected] on 15 Sep 2011 at 5:26

Attachments:

• [Screen shot 2011-09-15 at 10.25.27 AM.png](https://storage.googleapis.com/google-code-attachments/informatics/issue-4/comment-10/Screen shot 2011-09-15 at 10.25.27 AM.png)

[GoogleCodeExporter]

It just means the server is very busy and cannot accept another job. Although, 
you may still be able to run other jobs with smaller memory requirements. 

Original comment by [email protected] on 15 Sep 2011 at 6:12

[GoogleCodeExporter]

ok, tks a lot I'll shift to any other fgenes

Original comment by [email protected] on 15 Sep 2011 at 6:16

[GoogleCodeExporter]

Hi,

the run was perfect until the indexing. The output file until then are perfect:

fgene3 [/usr/pl_cache/pipeline/2011September16_14h21m09s250ms] ll
total 19648
-rw-rw-rw- 1 pipeline pipeline 6738598 Sep 16 14:21 
SamToolsRemoveDuplicates_1.OutputNo-DuplicatesBAMfile-1.bam
-rw-rw-rw- 1 pipeline pipeline 6592761 Sep 16 14:21 
SamToolsSort_1.OutputSortedBAMfile-1.bam
-rw-rw-rw- 1 pipeline pipeline 6738662 Sep 16 14:21 
SamToolsView_1.OutputBAMfile-1.bam
drwxrwxrwx 2 pipeline pipeline    4096 Sep 16 16:08 streams

But looking to the indexing process even if it is green, I have the error that 
it cannot open the BAM (see pic). If I open up che error logs of the output 
folders i have the error saying that it cannot copy the .bai file BUT the .bai 
file (as you can see) has not been produced. Note that I have the permissions 
on the output folders I have set.

Fed

Original comment by [email protected] on 17 Sep 2011 at 12:25

Attachments:

• fgene1_BasicQC1.pipe
• [Screen shot 2011-09-16 at 5.18.39 PM.png](https://storage.googleapis.com/google-code-attachments/informatics/issue-4/comment-13/Screen shot 2011-09-16 at 5.18.39 PM.png)
• [Screen shot 2011-09-16 at 5.18.24 PM.png](https://storage.googleapis.com/google-code-attachments/informatics/issue-4/comment-13/Screen shot 2011-09-16 at 5.18.24 PM.png)

[GoogleCodeExporter]

Hi,

the run was perfect until the indexing. The output file until then are perfect:

fgene3 [/usr/pl_cache/pipeline/2011September16_14h21m09s250ms] ll
total 19648
-rw-rw-rw- 1 pipeline pipeline 6738598 Sep 16 14:21 
SamToolsRemoveDuplicates_1.OutputNo-DuplicatesBAMfile-1.bam
-rw-rw-rw- 1 pipeline pipeline 6592761 Sep 16 14:21 
SamToolsSort_1.OutputSortedBAMfile-1.bam
-rw-rw-rw- 1 pipeline pipeline 6738662 Sep 16 14:21 
SamToolsView_1.OutputBAMfile-1.bam
drwxrwxrwx 2 pipeline pipeline    4096 Sep 16 16:08 streams

But looking to the indexing process even if it is green, I have the error that 
it cannot open the BAM (see pic). If I open up che error logs of the output 
folders i have the error saying that it cannot copy the .bai file BUT the .bai 
file (as you can see) has not been produced. Note that I have the permissions 
on the output folders I have set.

Fed

Original comment by [email protected] on 17 Sep 2011 at 12:26

Attachments:

• fgene1_BasicQC1.pipe
• [Screen shot 2011-09-16 at 5.18.39 PM.png](https://storage.googleapis.com/google-code-attachments/informatics/issue-4/comment-14/Screen shot 2011-09-16 at 5.18.39 PM.png)
• [Screen shot 2011-09-16 at 5.18.24 PM.png](https://storage.googleapis.com/google-code-attachments/informatics/issue-4/comment-14/Screen shot 2011-09-16 at 5.18.24 PM.png)

[GoogleCodeExporter]

It looks like the Calmd module outputs the BAM file to the standard output. 
Thus, it isn't passing any file to the Index module. I wrote a wrapper script 
to redirect stdout to the output parameter and put it here: 
/projects1/samtools_calmd.sh. You can copy it wherever you want and use the 
path to that script as the executable in the calmd module. 

Sam

Original comment by [email protected] on 17 Sep 2011 at 3:02

[GoogleCodeExporter]

Use the same script for the calmd module in Basic QC2.

Original comment by [email protected] on 17 Sep 2011 at 3:02

[GoogleCodeExporter]

ok, tks running now.

Fed

Original comment by [email protected] on 17 Sep 2011 at 3:15

[GoogleCodeExporter]

Oh yes, BasicQC1 went ok!

Original comment by [email protected] on 17 Sep 2011 at 4:25