Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

#232 cgw looping infinitely on a scaffold

scaffolder
closed-wont-fix
Jason Miller
5
2015-02-03
2013-01-02
eernst
No

Representative log output follows:

MergeScaffoldsAggressive()-- iter 25469 -- continue because we merged scaffolds.
ExamineUsableSEdges()- maxWeightEdge from 0 to 7 at idx 680 out of 21962
* Considering edges with weight >= 5.25 (maxWeightEdge 7 weightScale 0.7500)
isQualityScaffoldMergingEdge()-- Merge scaffolds 513317 (22130.0bp) and 513318 (2898.0bp): gap -6381.0bp +- 519.1bp weight 7 AB_AB edge
isQualityScaffoldMergingEdge()-- scaffold 513317 instrumenter happy 64.0 gap 34.7 misorient close 0.0 correct 119.0 far 40.0 oriented close 0.0 far 1.0 missing 175.1 external 28.2
isQualityScaffoldMergingEdge()-- scaffold 513318 instrumenter happy 3.0 gap 0.0 misorient close 0.0 correct 12.0 far 2.0 oriented close 0.0 far 0.0 missing 21.7 external 33.3
isQualityScaffoldMergingEdge()-- scaffold (new) instrumenter happy 67.0 gap 58.5 misorient close 0.0 correct 131.0 far 45.0 oriented close 0.0 far 5.0 missing 191.6 external 28.9
isQualityScaffoldMergingEdge()-- before: 0.141 satisfied (66/370 good/bad mates) after: 0.141 satisfied (66/372 good/bad mates)
isQualityScaffoldMergingEdge()-- ARE happy enough to merge 101 (0.141 >= 0.985) || (0.141 >= 0.141) || ((66 > 66) && (1.000 <= 0.300))
isQualityScaffoldMergingEdge()-- NEW pass (29783/0)
ExamineSEdgeForUsability_Interleaved()-- Interleaving succeeded without contig overlaps; will merge (-6368 +- 519).
InsertScaffoldContentsIntoScaffold()-- Insert scaffold 513317 (22130bp) into scaffold 513319 (0bp) at offset 0.000 +/- 0.000 orient F
InsertScaffoldContentsIntoScaffold()-- Insert CI 13770611 626bp fwd 0 +- 0 626 +- 16 was 0 +- 0 626 +- 16
InsertScaffoldContentsIntoScaffold()-- Insert CI 13747324 14229bp fwd 1483 +- 378623 15712 +- 378993 was 1483 +- 378623 15712 +- 378993
InsertScaffoldContentsIntoScaffold()-- Insert CI 13590456 1097bp rev 18710 +- 1014506 19807 +- 1014534 was 18710 +- 1014506 19807 +- 1014534
InsertScaffoldContentsIntoScaffold()-- Insert CI 13747325 2322bp fwd 19808 +- 1175564 22130 +- 1175624 was 19808 +- 1175564 22130 +- 1175624
InsertScaffoldContentsIntoScaffold()-- Insert scaffold 513318 (2898bp) into scaffold 513319 (22130bp) at offset 15762.000 +/- 624.173 orient F
InsertScaffoldContentsIntoScaffold()-- Insert CI 13759266 2898bp fwd 15762 +- 389592 18660 +- 389667 was 0 +- 0 2898 +- 75
KickOutNonOverlappingContig: Removing contig 13759266 to scaffold 513320 because we found no overlaps to it
RecomputeOffsetsInScaffold() returned RECOMPUTE_DELETE_CONTIG on scaffold 513319; scaffold modified, keep trying
dpbtrf failed with info=3; no solution found, giving up.
RecomputeOffsetsInScaffold() returned RECOMPUTE_LAPACK on scaffold 513319; no more work will help.
BuildSEdges()-- build scaffold edges for scaffold 513319
MergeScaffoldsAggressive()-- iter 25470 -- continue because we merged scaffolds.
ExamineUsableSEdges()- maxWeightEdge from 0 to 7 at idx 680 out of 21962
* Considering edges with weight >= 5.25 (maxWeightEdge 7 weightScale 0.7500)
isQualityScaffoldMergingEdge()-- Merge scaffolds 513319 (22130.0bp) and 513320 (2898.0bp): gap -6381.0bp +- 519.1bp weight 7 AB_AB edge
isQualityScaffoldMergingEdge()-- scaffold 513319 instrumenter happy 64.0 gap 34.7 misorient close 0.0 correct 119.0 far 40.0 oriented close 0.0 far 1.0 missing 175.1 external 28.2
isQualityScaffoldMergingEdge()-- scaffold 513320 instrumenter happy 3.0 gap 0.0 misorient close 0.0 correct 12.0 far 2.0 oriented close 0.0 far 0.0 missing 21.7 external 33.3
isQualityScaffoldMergingEdge()-- scaffold (new) instrumenter happy 67.0 gap 58.5 misorient close 0.0 correct 131.0 far 45.0 oriented close 0.0 far 5.0 missing 191.6 external 28.9
isQualityScaffoldMergingEdge()-- before: 0.141 satisfied (66/370 good/bad mates) after: 0.141 satisfied (66/372 good/bad mates)
isQualityScaffoldMergingEdge()-- ARE happy enough to merge 101 (0.141 >= 0.985) || (0.141 >= 0.141) || ((66 > 66) && (1.000 <= 0.300))
isQualityScaffoldMergingEdge()-- NEW pass (29784/0)
ExamineSEdgeForUsability_Interleaved()-- Interleaving succeeded without contig overlaps; will merge (-6368 +- 519).
InsertScaffoldContentsIntoScaffold()-- Insert scaffold 513319 (22130bp) into scaffold 513321 (0bp) at offset 0.000 +/- 0.000 orient F
InsertScaffoldContentsIntoScaffold()-- Insert CI 13770611 626bp fwd 0 +- 0 626 +- 16 was 0 +- 0 626 +- 16
InsertScaffoldContentsIntoScaffold()-- Insert CI 13747324 14229bp fwd 1483 +- 378623 15712 +- 378993 was 1483 +- 378623 15712 +- 378993
InsertScaffoldContentsIntoScaffold()-- Insert CI 13590456 1097bp rev 18710 +- 1014506 19807 +- 1014534 was 18710 +- 1014506 19807 +- 1014534
InsertScaffoldContentsIntoScaffold()-- Insert CI 13747325 2322bp fwd 19808 +- 1175564 22130 +- 1175624 was 19808 +- 1175564 22130 +- 1175624
InsertScaffoldContentsIntoScaffold()-- Insert scaffold 513320 (2898bp) into scaffold 513321 (22130bp) at offset 15762.000 +/- 624.173 orient F
InsertScaffoldContentsIntoScaffold()-- Insert CI 13759266 2898bp fwd 15762 +- 389592 18660 +- 389667 was 0 +- 0 2898 +- 75
KickOutNonOverlappingContig: Removing contig 13759266 to scaffold 513322 because we found no overlaps to it
RecomputeOffsetsInScaffold() returned RECOMPUTE_DELETE_CONTIG on scaffold 513321; scaffold modified, keep trying
dpbtrf failed with info=3; no solution found, giving up.
RecomputeOffsetsInScaffold() returned RECOMPUTE_LAPACK on scaffold 513321; no more work will help.
BuildSEdges()-- build scaffold edges for scaffold 513321

Discussion

  • Jason Miller
    Jason Miller
    2013-03-08

    • assigned_to: nobody --> jasonmiller9704
    • milestone: --> scaffolder
    • labels: --> Does not terminate
     
  • Jason Miller
    Jason Miller
    2013-03-08

    Has there been any change to your run? If not, can you provide some details? Was this the 7-0 stage? What kind of data (eg Illumina PE)?

     
  • eernst
    eernst
    2013-05-10

    I apologize for the long delay. In the meantime, I've gotten some new data, attempted a new assembly, and am stuck in an infinite cgw loop once again. The libraries are: ~6X PBcR, and ~50X Illumina MP 5K and 8K.

    Scaffolding options are:
    kickOutNonOvlContigs = 1
    cgwDemoteRBP = 0
    cgwUseUnitigOverlaps = 0
    cgwMergeMissingThreshold = 0.5
    doExtendClearRanges = 1

    Log snippet attached.

     
    Last edit: eernst 2013-05-10
  • eernst
    eernst
    2013-05-30

    Has anyone else run into this issue? Any suggestions for workarounds?

    I tried restarting cgw from the last checkpoint with kickOutNonOvlContigs=0, and scaffolding progressed, but almost exclusively operating on a single scaffold which had grown to ~17Mbp over the course of 6 or 7 days, continuing to iterate over MergeScaffoldsAggressive() but actually making (excruciatingly slow) progress.

     
  • Brian Walenz
    Brian Walenz
    2013-05-31

    That sounds like something we can get around without code changes.

    Try setting cgwMinMergeWeight=3 (default 2). We've seen scaffolder get 'stuck' merging lots of tiny scaffolds into one larger one. This increases the number of mate pairs required to merge scaffolds.

    Try setting cgwMergeFilterLevel=2. This changes the algorithm to be a bit more selective about what merges it will do. I don't think it will help with the behavior you are seeing though.

    Both require the latest assembler, not CA release 7.

    I don't really have the time right now to dig around for the original infinite loop problem. Sorry.

     
  • Brian Walenz
    Brian Walenz
    2015-02-03

    Closing old case. Won't fix.

     
  • Brian Walenz
    Brian Walenz
    2015-02-03

    • status: open --> closed-wont-fix