mergeSAM.log error

[hcsun1]

Dear HiC-Pro Authors,

Thank you for your software. However, I encountered an error while running my own data:

Pairing of R1 and R2 tags ...
Logs: logs/Fuji/mergeSAM.log
Makefile:144: bowtie_pairing] Error 1

Upon further investigation, I found the following error in the mergeSAM.log file.

I have tried many solutions, including installing the HiC-Pro Singularity image, but I still haven't been able to resolve this issue.

Could you kindly assist me with this problem? I greatly appreciate your help.

Best regards,
Haochen Sun

[nservant]

Could you check the read names in R1/R2 files. They should be identical, and it seems that recently some sequencers add /1, /2 at the end of the read names .... which make HiC-Pro crashing

[qqyanyang]

Could you check the read names in R1/R2 files. They should be identical, and it seems that recently some sequencers add /1, /2 at the end of the read names .... which make HiC-Pro crashing

Excuse me，I have a same error when I running HiC-pro process. My error like this:
Run HiC-Pro 3.1.0

Bowtie2 alignment step1 ...
Logs: logs/iPSC_P13_Hi-C_R1r1/mapping_step1.log

---

Bowtie2 alignment step2 ...
Logs: logs/iPSC_P13_Hi-C_R1r1/mapping_step2.log

---

Combine R1/R2 alignment files ...
Logs: logs/iPSC_P13_Hi-C_R1r1/mapping_combine.log

---

Mapping statistics for R1 and R2 tags ...
Logs: logs/iPSC_P13_Hi-C_R1r1/mapping_stats.log

---

Pairing of R1 and R2 tags ...
Logs: logs/iPSC_P13_Hi-C_R1r1/mergeSAM.log
make: *** [path../../HiC-Pro_3.1.0/bin/../scripts//Makefile:144: bowtie_pairing] Error 1

And this is my mergeSAMlog:
python_path/../bin/python /../HiC-Pro_3.1.0/scripts/mergeSAM.py -q 0 -t -v -f bowtie_results/bwt2/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_1_hg38.bwt2merged.bam -r bowtie_results/bwt2/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_2_hg38.bwt2merged.bam -o bowtie_results/bwt2/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_hg38.bwt2pairs.bam
[E::idx_find_and_load] Could not retrieve index file for 'bowtie_results/bwt2/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_1_hg38.bwt2merged.bam'
[E::idx_find_and_load] Could not retrieve index file for 'bowtie_results/bwt2/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_2_hg38.bwt2merged.bam'

mergeBAM.py

forward= bowtie_results/bwt2/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_1_hg38.bwt2merged.bam

reverse= bowtie_results/bwt2/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_2_hg38.bwt2merged.bam

output= bowtie_results/bwt2/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_hg38.bwt2pairs.bam

min mapq= 0

report_single= False

report_multi= False

verbose= True

Merging forward and reverse tags ...

Forward and reverse reads not paired. Check that BAM files have the same read names and are sorted.

And my mapping_conbine.log is :
../bin/samtools merge -@ 144 -n -f bowtie_results/bwt2/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_1_hg38.bwt2merged.bam bowtie_results/bwt2_global/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_1_hg38.bwt2glob.bam bowtie_results/bwt2_local/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_1_hg38.bwt2glob.unmap_bwt2loc.bam
../bin/samtools merge -@ 144 -n -f bowtie_results/bwt2/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_2_hg38.bwt2merged.bam bowtie_results/bwt2_global/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_2_hg38.bwt2glob.bam bowtie_results/bwt2_local/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_2_hg38.bwt2glob.unmap_bwt2loc.bam
../bin/samtools sort -@ 144 -m 1000M -n -T tmp/iPSC_P13_Hi-C_R1r1_1_hg38 -o bowtie_results/bwt2/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_1_hg38.bwt2merged.sorted.bam bowtie_results/bwt2/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_1_hg38.bwt2merged.bam
../bin/samtools sort -@ 144 -m 1000M -n -T tmp/iPSC_P13_Hi-C_R1r1_2_hg38 -o bowtie_results/bwt2/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_2_hg38.bwt2merged.sorted.bam bowtie_results/bwt2/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_2_hg38.bwt2merged.bam
../bin/HiC-Pro_3.1.0/scripts/hic.inc.sh: Line 86: 2519760 Killed ../bin/samtools sort -@ 144 -m 1000M -n -T tmp/iPSC_P13_Hi-C_R1r1_1_hg38 -o bowtie_results/bwt2/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_1_hg38.bwt2merged.sorted.bam bowtie_results/bwt2/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_1_hg38.bwt2merged.bam
[bam_sort_core] merging from 288 files and 144 in-memory blocks...
mv bowtie_results/bwt2/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_2_hg38.bwt2merged.sorted.bam bowtie_results/bwt2/iPSC_P13_Hi-C_R1r1/iPSC_P13_Hi-C_R1r1_2_hg38.bwt2merged.bam

With your answer for hcsun1, I while try your solvement, but how can I check the read names in my files? Like that:
iPSC_P13_Hi-C_R1r1_1.fq.gz
iPSC_P13_Hi-C_R1r1_2.fq.gz
If you need the config-hicpro.txt ,I while support it.Thanks a lot

[lilongjuan]

I also encountered the same problem during the run（mapping）. I have checked my bam file in bwt2. Some of the content is as follows：(hicpro) iy1702@iyun17:$ samtools view /media/desk15/iy1702/newstools/HiC-Pro_3.1.0/output_data/bowtie_results/bwt2/sample1/sample1_R1.bam | head -n 10 | cut -f1
SRR12062646.1
SRR12062646.2
SRR12062646.3
SRR12062646.4
SRR12062646.5
SRR12062646.6
SRR12062646.7
SRR12062646.8
SRR12062646.9
SRR12062646.10
(hicpro) iy1702@iyun17:$ samtools view /media/desk15/iy1702/newstools/HiC-Pro_3.1.0/output_data/bowtie_results/bwt2/sample1/sample1_R2.bam | head -n 10 | cut -f1
SRR12062646.1
SRR12062646.1
SRR12062646.2
SRR12062646.2
SRR12062646.3
SRR12062646.3
SRR12062646.4
SRR12062646.4
SRR12062646.5
SRR12062646.5
，Indeed, as you said, how should I specifically solve this problem? Thank you!

[qqyanyang]

Hello, the problem I encountered before, I think it should be caused by insufficient computer memory. After I added memory, there was no similar error report. However, I have not encountered your problem before. If I can help you with anything, we can discuss it via email. At 2024-11-29 16:06:37, "lilongjuan" ***@***.***> wrote: I also encountered the same problem during the run（mapping）. I have checked my bam file in bwt2. Some of the content is as follows：(hicpro) ***@***.***:$ samtools view /media/desk15/iy1702/newstools/HiC-Pro_3.1.0/output_data/bowtie_results/bwt2/sample1/sample1_R1.bam | head -n 10 | cut -f1 SRR12062646.1 SRR12062646.2 SRR12062646.3 SRR12062646.4 SRR12062646.5 SRR12062646.6 SRR12062646.7 SRR12062646.8 SRR12062646.9 SRR12062646.10 (hicpro) ***@***.***:$ samtools view /media/desk15/iy1702/newstools/HiC-Pro_3.1.0/output_data/bowtie_results/bwt2/sample1/sample1_R2.bam | head -n 10 | cut -f1 SRR12062646.1 SRR12062646.1 SRR12062646.2 SRR12062646.2 SRR12062646.3 SRR12062646.3 SRR12062646.4 SRR12062646.4 SRR12062646.5 SRR12062646.5 ，Indeed, as you said, how should I specifically solve this problem? Thank you! — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[lilongjuan]

Hello, the problem I encountered before, I think it should be caused by insufficient computer memory. After I added memory, there was no similar error report. However, I have not encountered your problem before. If I can help you with anything, we can discuss it via email. At 2024-11-29 16:06:37, "lilongjuan" @.> wrote: I also encountered the same problem during the run（mapping）. I have checked my bam file in bwt2. Some of the content is as follows：(hicpro) @.:$ samtools view /media/desk15/iy1702/newstools/HiC-Pro_3.1.0/output_data/bowtie_results/bwt2/sample1/sample1_R1.bam | head -n 10 | cut -f1 SRR12062646.1 SRR12062646.2 SRR12062646.3 SRR12062646.4 SRR12062646.5 SRR12062646.6 SRR12062646.7 SRR12062646.8 SRR12062646.9 SRR12062646.10 (hicpro) @.:$ samtools view /media/desk15/iy1702/newstools/HiC-Pro_3.1.0/output_data/bowtie_results/bwt2/sample1/sample1_R2.bam | head -n 10 | cut -f1 SRR12062646.1 SRR12062646.1 SRR12062646.2 SRR12062646.2 SRR12062646.3 SRR12062646.3 SRR12062646.4 SRR12062646.4 SRR12062646.5 SRR12062646.5 ，Indeed, as you said, how should I specifically solve this problem? Thank you! — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: @.>

Hello, thank you very much for your reply, but I don't particularly understand which problem you encountered before. Can you tell me more about it? I am very eager to get your help to solve the problem I am currently encountering. Thank you again.

[qqyanyang]

Hello, the problem I encountered before, I think it should be caused by insufficient computer memory. After I added memory, there was no similar error report. However, I have not encountered your problem before. If I can help you with anything, we can discuss it via email. At 2024-11-29 16:06:37, "lilongjuan" @.> wrote: I also encountered the same problem during the run（mapping）. I have checked my bam file in bwt2. Some of the content is as follows：(hicpro) _@**._:$ samtools view /media/desk15/iy1702/newstools/HiC-Pro_3.1.0/output_data/bowtie_results/bwt2/sample1/sample1_R1.bam | head -n 10 | cut -f1 SRR12062646.1 SRR12062646.2 SRR12062646.3 SRR12062646.4 SRR12062646.5 SRR12062646.6 SRR12062646.7 SRR12062646.8 SRR12062646.9 SRR12062646.10 (hicpro) _@.:$ samtools view /media/desk15/iy1702/newstools/HiC-Pro_3.1.0/output_data/bowtie_results/bwt2/sample1/sample1_R2.bam | head -n 10 | cut -f1 SRR12062646.1 SRR12062646.1 SRR12062646.2 SRR12062646.2 SRR12062646.3 SRR12062646.3 SRR12062646.4 SRR12062646.4 SRR12062646.5 SRR12062646.5 ，Indeed, as you said, how should I specifically solve this problem? Thank you! — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: @.**_>

Hello, thank you very much for your reply, but I don't particularly understand which problem you encountered before. Can you tell me more about it? I am very eager to get your help to solve the problem I am currently encountering. Thank you again.

Hello，the problem I encountered before was that the operation stopped before the mergesam stage, and there would be a log file (there would be a log in the commit step, but I deleted it before, sorry), but then I speculated that it was a memory overflow, and the relationship between threads and memory is still worth considering in hicpro, so I think if you check the logs file and find that there is a similar problem, you can adjust the threads and memory in the config file, which may help you.

[lilongjuan]

Hello, thank you very much for your patient answer. I do have the problem you mentioned, but I also made some modifications before I encountered this problem as you said. thank you very much for your reply!

…

---- Replied Message ---- | From | ***@***.***> | | Date | 12/8/2024 18:31 | | To | ***@***.***> | | Cc | ***@***.***>, ***@***.***> | | Subject | Re: [nservant/HiC-Pro] mergeSAM.log error (Issue #649) | Hello, the problem I encountered before, I think it should be caused by insufficient computer memory. After I added memory, there was no similar error report. However, I have not encountered your problem before. If I can help you with anything, we can discuss it via email. At 2024-11-29 16:06:37, "lilongjuan" @.> wrote: I also encountered the same problem during the run（mapping）. I have checked my bam file in bwt2. Some of the content is as follows：(hicpro) _@**._:$ samtools view /media/desk15/iy1702/newstools/HiC-Pro_3.1.0/output_data/bowtie_results/bwt2/sample1/sample1_R1.bam | head -n 10 | cut -f1 SRR12062646.1 SRR12062646.2 SRR12062646.3 SRR12062646.4 SRR12062646.5 SRR12062646.6 SRR12062646.7 SRR12062646.8 SRR12062646.9 SRR12062646.10 (hicpro) ***@***.***:$ samtools view /media/desk15/iy1702/newstools/HiC-Pro_3.1.0/output_data/bowtie_results/bwt2/sample1/sample1_R2.bam | head -n 10 | cut -f1 SRR12062646.1 SRR12062646.1 SRR12062646.2 SRR12062646.2 SRR12062646.3 SRR12062646.3 SRR12062646.4 SRR12062646.4 SRR12062646.5 SRR12062646.5 ，Indeed, as you said, how should I specifically solve this problem? Thank you! — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: @.**_> Hello, thank you very much for your reply, but I don't particularly understand which problem you encountered before. Can you tell me more about it? I am very eager to get your help to solve the problem I am currently encountering. Thank you again. Hello，the problem I encountered before was that the operation stopped before the mergesam stage, and there would be a log file (there would be a log in the commit step, but I deleted it before, sorry), but then I speculated that it was a memory overflow, and the relationship between threads and memory is still worth considering in hicpro, so I think if you check the logs file and find that there is a similar problem, you can adjust the threads and memory in the config file, which may help you. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>