PBSuite / Blog: Recent posts

Below is a quick rundown of the changes we've made to PBSuite.
The most important of which is about the single 5' end contig extension correction in PBJelly. It's important to rerun your genome upgrade with the new version if you had any significant number of single end extensions.
Otherwise, most of the changes were weird edge-cases that were fixed thanks to you, the active user community, giving much needed feedback. ... read more

I've spent the last couple of months really diving into the Structural Variant (SV) discovery tool PBHoney. It's been heavily refactored and improved so that PBHoney is the one stop shop for SV discovery with long reads. This tool is now extremely easy to use, blazingly fast, and incredibly accurate. We'll be publishing presenting PBHoney's performance on multiple human genomes in the next few months. For all the details on what's changed in this new version of PBSuite, check the CHANGELOG.txt. Here are some highlights of my favorite changes: ... read more

The big news is that PBJelly isn't as fussy about your networkx version. Hopefully this reduces the number of tickets... More importantly, PBJelly is more accurate than ever thanks to fixes to contig edge trimming AND better minRead functionality. PBHoney has better documentation and fewer bugs. Below is the information from the CHANGELOG

Jelly
general -
=Networkx v1.7 now supported!! We also hold support for v1.1 though
this is experimental. Please let me know if you see any bugs.
=Fixed docs/jellyExample/Protocol.xml (the blasr -bestn was incorrectly
set at 5)
=Added space support for Fasta/q headers. There shouldn't be much of a
problem if your headers have spaces.
Collection -
=Fixed a trimming bug that would occasionally take off more than
MAXTRIM bases
=Changed MAXTRIM to 100bp
=Added minRead support functionality - this works better now
=Changed logic structure for building FillMetrics (though this could
still be cleaner)
Assembly -
=Fixed bug that would occasionally allow inproper support during
assembly
Support -
=Cleaned up -h and added a little documentation to the code
summarizeAssembly -
=Even prettier output... read more

This version has a few minor bug-fixes in PBJelly, and functionality extension in PBHoney.

Below are details from the CHANGELOG.txt

Honey.py force
Added parser helpers for the Force output format as well as a 'genotyper'
Improved spot forcing by chaninging --minIns to --minErr. Now 'twilight' deletions aren't incorrectly forced
Added --bedPE Support which includes a couple of Bed Objects... which probably need to be moved to utils/FileHandlers.py... [read more](/p/pb-jelly/blog/2014/07/pbsuite-v14714/)

Some minor bug fixes and sanity checks for PBJelly.
Improved the documentation in PBHoney.
NOTE: 11 people downloaded 14.6.24 before I was able to identify and fix a small bug on line 91 of Jelly.py

PBJelly and PBHoney are not officially packaged within the same code. If you only want one of the tools, the presence of the second will not negatively affect anything you need.

Highlights of changes made to the PBSuite include:

• Improved functionality of PBHoney including 'force' calling, bug-fixes, better annotations and size estimations.
• PBJelly's output stage now has a parameter for the minimum number of reads required to support a gap fill before it's allowed to contribute to the improved assembly.
• New and improved tools for summarizing your data including BamStat.py, mapStat.py, quickN50.py, and summarizeAssembly.py.... read more

If you check in Files tab, you'll see two new downloads - PBSuite and PBHoney. I apologize that it's poorly organized and will be cleaning it up soon.

The reason for these new files is I've been building a new program called PBHoney. Because there are so many useful utilities in PBJell's code-base, I decided to build PBHoney as an extension of PBJelly. This has given birth to what this project will be called from here on out - the PBSuite (suite.. sweet.. get it!?).... read more

After tons of feedback from you wonderful users, we've discovered a handful of bugs that are fixed in our latest PBJelly release v14.1.14.
For anyone who saw Kim Worley's presentation at PAG, this is the version 2 you want.

Thank you users for your help in making Jelly better! With your assistance, I was able to better the code to ensure that Jelly's is more accurate and more stable. If you've been using the previous beta-version, I must insist you upgrade to the latest version to ensure you're making the best assemblies possible.

There is still one outstanding issue that has affected many users. I didn't realize that I was developing Jelly with a very old version of networkx (v1.1) in my environment. Unfortunately, the new version handles thing differently enough that Jelly can't work with it right now. I will be addressing this problem as soon as I can. Until then, you need to have networkx v1.1 in your python environment.... read more

After a long development cycle, The beta version of the new Jelly has arrived! Please download it, test it out, and send me your feedback. If you feel like waiting for the full release, you can expect a finalized program soon barring any major setbacks.

Below are the major changes in functionality. Soon we'll have a full report explaining what has changed and why. I'm not sure if this report will be an Applications Note in some journal or just a blog post. ... read more

Here are the changes you will find in the latest version of PBJelly
-- PBJelly's NCBISubmissionCreator now works and is much more user-friendly.
-- fillMetrics.json for any improved gap now has a [Left|Right|Span][Start|End] tag that keeps track of the coordinates of the contig that was used to left/right/span improve the gap.
-- Changed the untangling of reads in Support so it functions better.
-- Added exclusion.py for help with performing multiple iterations of PBJelly.py. exclusion.py will compare two support directories from PBJelly runs and remove reads that were previously identified as supporting a gap from being used in a successive iteration. Though, this is currently not optimal because there isn't a 1-to-1 correspondence between reads supporting the gaps and reads composing the contig that improves the gap (i.e. you lose some amount of data.)... read more

Unfortunately, testing for PBJelly 12.7.25 wasn't as rigorous as it needed to be and there were a number of bugs that slipped through. These have been addressed and are now in the CHANGELOG file distributed with PBJelly 12.8.2. Sorry for any inconvenience this may have caused.

Upgrades
-- Gaps that are within 25 bp of one another are now consolidated into a single gap.
-- Made some progress on the scripts that help with NCBI Submission of Upgraded Genomes (Still in Beta, though)... read more

    One limitation of PBJelly we identified is that mapping results were holding us back during or remap/resupport stage of assembly, especially when defining trim boundaries of negative gaps.
For example,
Consider the following fake contig PBJelly assembled around a negative gap and it's alignment to the reference.

PBJelly Contig - ...ATCGATCGA------TCGA-----------------GATCGATCG...
Reference      - ...ATCGATCGATCGAGATCGATCGGNNNNNNNNNNATCGATCGATCG...


    Previously, PBJelly would trim 4 bases off of the 3' end of the left contig (TCG) and 3 bases off of the 5' end of the right contig (ATC) before joining the contigs. However, if you look closely, the first and third gaps in the realignment are unnecessary. That is to say, one can realign the bases TCGA further upstream in a manner more continuous with the rest of the PBJelly contig. After realignment, 9 bases are identified for trimming from the 3' end of the left contig.

    In addition to the realignment improvement, we also have sped up the remap/resupport stage of assembly. PBJelly used to map the contigs it created to the entire original reference to find support. This is unnecessary because we've already identified all reads within a contig as supporting a particular gap. We really only need to remap to the gap-flanking region of interest in order to identify what part of our new contig fills the gap. Now PBJelly only remaps to the region of the gap that is being improved. This is the same improvement that allowed us to run PBJelly on the very large Sooty mangabey reference in a reasonable amount of time. It took approximately 2 days from start to finish to improve the Sooty mangabey by submitting up to 300 jobs to the cluster per step in the PBJelly workflow (i.e. splitting the assembly effort into 300 independent partitions - something easily done using the nJobs element in your PBJelly protocol settings.)

Jelly has been updated to work with SMRTAnalysis 1.3.1. See Version 12.5.29 and up!