Skip to content
This repository has been archived by the owner on Jul 17, 2023. It is now read-only.

No reads passing filters for dragmap-aligned bams #299

Open
Rohan-Abraham opened this issue Oct 18, 2022 · 0 comments
Open

No reads passing filters for dragmap-aligned bams #299

Rohan-Abraham opened this issue Oct 18, 2022 · 0 comments

Comments

@Rohan-Abraham
Copy link

Rohan-Abraham commented Oct 18, 2022 •
edited
Loading

Hi,

I've been running into an error in which I find that after generating bams aligned with the dragmap (1.3.0) aligner they fail to pass manta filters. Many of these same samples ran through manta previously without error using an older (bwa 0.7.17) aligner which was part of our old pipeline. The full error log is as follows:

Command error:
  [2022-10-14T06:38:08.701509Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskRunner:getAlignmentStats_generateStats_000] Retrying task: 'getAlignmentStats_generateStats_000'. Total prior task failures: 1
  [2022-10-14T06:38:08.714303Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskRunner:getAlignmentStats_generateStats_000] Task initiated on local node
  [2022-10-14T06:58:16.373405Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [StatusUpdate] ===== MantaWorkflow StatusUpdate =====
  [2022-10-14T06:58:16.374348Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [StatusUpdate] Workflow specification is complete?: False
  [2022-10-14T06:58:16.374397Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [StatusUpdate] Task status (waiting/queued/running/complete/error): 895/0/1/380/0
  [2022-10-14T06:58:16.374442Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [StatusUpdate] Longest ongoing queued task time (hrs): 0.0000
  [2022-10-14T06:58:16.374470Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [StatusUpdate] Longest ongoing queued task name: ''
  [2022-10-14T06:58:16.374506Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [StatusUpdate] Longest ongoing running task time (hrs): 0.9997
  [2022-10-14T06:58:16.374539Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [StatusUpdate] Longest ongoing running task name: 'getAlignmentStats_generateStats_000'
  [2022-10-14T07:13:47.843926Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskRunner:getAlignmentStats_generateStats_000] [WARNING] Task: 'getAlignmentStats_generateStats_000' failed but qualifies for retry. Total task failures (including this one): 2. Task command: '/opt/manta-1.5.0/libexec/GetAlignmentStats --ref GRCh38_full_analysis_set_plus_decoy_hla.fa --output-file workspace/alignmentStats.xml.tmpdir/alignmentStats.xml.000.xml --align-file B46156_1_lane_dupsFlagged.md.recal.bam'
  [2022-10-14T07:15:17.934525Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskRunner:getAlignmentStats_generateStats_000] Retrying task: 'getAlignmentStats_generateStats_000'. Total prior task failures: 2
  [2022-10-14T07:15:17.945695Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskRunner:getAlignmentStats_generateStats_000] Task initiated on local node
  [2022-10-14T07:50:47.333070Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskManager] [ERROR] Failed to complete command task: 'getAlignmentStats_generateStats_000' launched from master workflow, error code: 1, command: '/opt/manta-1.5.0/libexec/GetAlignmentStats --ref GRCh38_full_analysis_set_plus_decoy_hla.fa --output-file workspace/alignmentStats.xml.tmpdir/alignmentStats.xml.000.xml --align-file B46156_1_lane_dupsFlagged.md.recal.bam'
  [2022-10-14T07:50:47.333977Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskManager] [ERROR] [getAlignmentStats_generateStats_000] Error Message:
  [2022-10-14T07:50:47.335612Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskManager] [ERROR] [getAlignmentStats_generateStats_000] Last 15 stderr lines from task (of 15 total lines):
  [2022-10-14T07:50:47.335612Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskManager] [ERROR] [2022-10-14T07:50:47.297525Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] FATAL_ERROR: 2022-Oct-14 00:50:47 /opt/manta-1.5.0/manta-1.5.0.release_src/src/c++/lib/manta/ReadGroupStatsUtil.cpp(247): Throw in function void ReadGroupOrientTracker::finalize(const ReadCounter&)
  [2022-10-14T07:50:47.335612Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskManager] [ERROR] [2022-10-14T07:50:47.300090Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] Dynamic exception type: boost::exception_detail::clone_impl<illumina::common::GeneralException>
  [2022-10-14T07:50:47.335612Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskManager] [ERROR] [2022-10-14T07:50:47.300830Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] std::exception::what: Too few high-confidence read pairs (0) to determine pair orientation for read group '' in bam file 'B46156_1_lane_dupsFlagged.md.recal.bam'
  [2022-10-14T07:50:47.335612Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskManager] [ERROR] [2022-10-14T07:50:47.301608Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] 	At least 100 high-confidence read pairs are required to determine pair orientation.
  [2022-10-14T07:50:47.335612Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskManager] [ERROR] [2022-10-14T07:50:47.302203Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] 	Total sampled reads: 826442797
  [2022-10-14T07:50:47.335612Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskManager] [ERROR] [2022-10-14T07:50:47.302992Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] 	Total sampled paired reads: 826442797
  [2022-10-14T07:50:47.335612Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskManager] [ERROR] [2022-10-14T07:50:47.303619Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] 	Total sampled paired reads passing MAPQ filter: 771169106
  [2022-10-14T07:50:47.335612Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskManager] [ERROR] [2022-10-14T07:50:47.304222Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] 	Total sampled high-confidence read pairs passing all filters: 0
  [2022-10-14T07:50:47.335612Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskManager] [ERROR] [2022-10-14T07:50:47.304853Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] 
  [2022-10-14T07:50:47.335612Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskManager] [ERROR] [2022-10-14T07:50:47.305660Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] 
  [2022-10-14T07:50:47.335612Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskManager] [ERROR] [2022-10-14T07:50:47.306386Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] 
  [2022-10-14T07:50:47.335612Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskManager] [ERROR] [2022-10-14T07:50:47.307216Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] cmdline:	/opt/manta-1.5.0/libexec/GetAlignmentStats --ref GRCh38_full_analysis_set_plus_decoy_hla.fa --output-file workspace/alignmentStats.xml.tmpdir/alignmentStats.xml.000.xml --align-file B46156_1_lane_dupsFlagged.md.recal.bam
  [2022-10-14T07:50:47.335612Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskManager] [ERROR] [2022-10-14T07:50:47.307957Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] version:	1.5.0
  [2022-10-14T07:50:47.335612Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskManager] [ERROR] [2022-10-14T07:50:47.308655Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] buildTime:	2021-05-20T08:31:11.839897Z
  [2022-10-14T07:50:47.335612Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskManager] [ERROR] [2022-10-14T07:50:47.309807Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] compiler:	g++-4.8.5
  [2022-10-14T07:50:47.335907Z] [n238.numbers.bcgsc.ca] [56341_1] [TaskManager] [ERROR] Shutting down task submission. Waiting for remaining tasks to complete.
  [2022-10-14T07:51:04.384867Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [ERROR] Workflow terminated due to the following task errors:
  [2022-10-14T07:51:04.385517Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [ERROR] Failed to complete command task: 'getAlignmentStats_generateStats_000' launched from master workflow, error code: 1, command: '/opt/manta-1.5.0/libexec/GetAlignmentStats --ref GRCh38_full_analysis_set_plus_decoy_hla.fa --output-file workspace/alignmentStats.xml.tmpdir/alignmentStats.xml.000.xml --align-file B46156_1_lane_dupsFlagged.md.recal.bam'
  [2022-10-14T07:51:04.385572Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [ERROR] [getAlignmentStats_generateStats_000] Error Message:
  [2022-10-14T07:51:04.385620Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [ERROR] [getAlignmentStats_generateStats_000] Last 15 stderr lines from task (of 15 total lines):
  [2022-10-14T07:51:04.385620Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [ERROR] [2022-10-14T07:50:47.297525Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] FATAL_ERROR: 2022-Oct-14 00:50:47 /opt/manta-1.5.0/manta-1.5.0.release_src/src/c++/lib/manta/ReadGroupStatsUtil.cpp(247): Throw in function void ReadGroupOrientTracker::finalize(const ReadCounter&)
  [2022-10-14T07:51:04.385620Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [ERROR] [2022-10-14T07:50:47.300090Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] Dynamic exception type: boost::exception_detail::clone_impl<illumina::common::GeneralException>
  [2022-10-14T07:51:04.385620Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [ERROR] [2022-10-14T07:50:47.300830Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] std::exception::what: Too few high-confidence read pairs (0) to determine pair orientation for read group '' in bam file 'B46156_1_lane_dupsFlagged.md.recal.bam'
  [2022-10-14T07:51:04.385620Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [ERROR] [2022-10-14T07:50:47.301608Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] 	At least 100 high-confidence read pairs are required to determine pair orientation.
  [2022-10-14T07:51:04.385620Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [ERROR] [2022-10-14T07:50:47.302203Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] 	Total sampled reads: 826442797
  [2022-10-14T07:51:04.385620Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [ERROR] [2022-10-14T07:50:47.302992Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] 	Total sampled paired reads: 826442797
  [2022-10-14T07:51:04.385620Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [ERROR] [2022-10-14T07:50:47.303619Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] 	Total sampled paired reads passing MAPQ filter: 771169106
  [2022-10-14T07:51:04.385620Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [ERROR] [2022-10-14T07:50:47.304222Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] 	Total sampled high-confidence read pairs passing all filters: 0
  [2022-10-14T07:51:04.385620Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [ERROR] [2022-10-14T07:50:47.304853Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] 
  [2022-10-14T07:51:04.385620Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [ERROR] [2022-10-14T07:50:47.305660Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] 
  [2022-10-14T07:51:04.385620Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [ERROR] [2022-10-14T07:50:47.306386Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] 
  [2022-10-14T07:51:04.385620Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [ERROR] [2022-10-14T07:50:47.307216Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] cmdline:	/opt/manta-1.5.0/libexec/GetAlignmentStats --ref GRCh38_full_analysis_set_plus_decoy_hla.fa --output-file workspace/alignmentStats.xml.tmpdir/alignmentStats.xml.000.xml --align-file B46156_1_lane_dupsFlagged.md.recal.bam
  [2022-10-14T07:51:04.385620Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [ERROR] [2022-10-14T07:50:47.307957Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] version:	1.5.0
  [2022-10-14T07:51:04.385620Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [ERROR] [2022-10-14T07:50:47.308655Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] buildTime:	2021-05-20T08:31:11.839897Z
  [2022-10-14T07:51:04.385620Z] [n238.numbers.bcgsc.ca] [56341_1] [WorkflowRunner] [ERROR] [2022-10-14T07:50:47.309807Z] [n238.numbers.bcgsc.ca] [56341_1] [getAlignmentStats_generateStats_000] compiler:	g++-4.8.5

I have replicated the same error using Genome in a Bottle bams aligned using dragmap 1.3.0 and run through manta 1.5.0. I noticed issue #118 noted a similar problem and found a workaround by running:

samtools view -F 1024 -h some.bam | perl -wlane 'if(not(m/^[@]/)&& defined($F[5])){$F[5] =~ s/\d*H//g;print join("\t",@F);}else{print $_;}' | samtools view -Sb > some.clean.bam

samtools index some.clean.bam

Though unfortunately this did not work in my case, and resulted in the same error as above. Cheers,

Rohan
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Rohan-Abraham