You can subscribe to this list here.
2009 |
Jan
|
Feb
|
Mar
|
Apr
(1) |
May
(1) |
Jun
(1) |
Jul
(2) |
Aug
|
Sep
(2) |
Oct
|
Nov
(2) |
Dec
(2) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2010 |
Jan
(1) |
Feb
(2) |
Mar
(6) |
Apr
(3) |
May
(2) |
Jun
(4) |
Jul
(3) |
Aug
(4) |
Sep
(1) |
Oct
(5) |
Nov
(5) |
Dec
(4) |
2011 |
Jan
(2) |
Feb
(2) |
Mar
(6) |
Apr
(4) |
May
(2) |
Jun
(3) |
Jul
(2) |
Aug
(3) |
Sep
(3) |
Oct
(2) |
Nov
(1) |
Dec
(2) |
2012 |
Jan
(3) |
Feb
(3) |
Mar
(3) |
Apr
(2) |
May
(2) |
Jun
(2) |
Jul
(3) |
Aug
(2) |
Sep
(1) |
Oct
(2) |
Nov
(1) |
Dec
(2) |
2013 |
Jan
(2) |
Feb
(3) |
Mar
(2) |
Apr
(2) |
May
(3) |
Jun
(2) |
Jul
(3) |
Aug
(1) |
Sep
(2) |
Oct
(2) |
Nov
(2) |
Dec
(2) |
2014 |
Jan
(3) |
Feb
(1) |
Mar
(2) |
Apr
(3) |
May
(2) |
Jun
(2) |
Jul
(2) |
Aug
(2) |
Sep
(3) |
Oct
(2) |
Nov
(2) |
Dec
(2) |
2015 |
Jan
(1) |
Feb
(2) |
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(1) |
Dec
(1) |
2016 |
Jan
|
Feb
|
Mar
|
Apr
(1) |
May
|
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
|
Nov
|
Dec
|
2017 |
Jan
|
Feb
|
Mar
(1) |
Apr
|
May
(1) |
Jun
(1) |
Jul
|
Aug
|
Sep
(1) |
Oct
|
Nov
|
Dec
|
2018 |
Jan
(1) |
Feb
(1) |
Mar
|
Apr
(1) |
May
|
Jun
|
Jul
(1) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2019 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(1) |
2020 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
|
Nov
|
Dec
|
2021 |
Jan
|
Feb
|
Mar
(1) |
Apr
|
May
|
Jun
|
Jul
(1) |
Aug
|
Sep
|
Oct
(1) |
Nov
|
Dec
|
2022 |
Jan
|
Feb
(1) |
Mar
|
Apr
(1) |
May
|
Jun
|
Jul
|
Aug
(1) |
Sep
(1) |
Oct
|
Nov
|
Dec
|
2023 |
Jan
|
Feb
(1) |
Mar
|
Apr
|
May
|
Jun
|
Jul
(1) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(1) |
2024 |
Jan
(2) |
Feb
|
Mar
|
Apr
(1) |
May
|
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
|
Nov
|
Dec
|
From: John M. <jm...@sa...> - 2016-04-22 14:49:04
|
New 1.3.1 versions of Samtools, BCFtools, and HTSlib have been released. This is mainly a bug fix release, fixing a number of bugs and issues in December's 1.3 release. In particular: * Improved management of samtools sort temporary files. Running several separate "... | samtools sort | ..." pipelines simultaneously in v1.3 could lead to the sort commands corrupting each other's temporary files. This is now fixed, and the functionality of the -T PREFIX option is improved. * All samtools command are now more careful about checking for I/O errors, especially when writing output files. * BCFtools has fixes to numerous commands and plugins, and has a new GTisec plugin courtesy of David Laehnemann. For further details of these and other changes in these releases, see the release notes for each project linked to at <http://www.htslib.org/download/>, or see all three together at <http://sourceforge.net/projects/samtools/files/samtools/1.3.1/>. These releases can be found at <http://www.htslib.org/>. Development and source code of these projects are linked from that web site and can be found in various Git repositories under <https://github.com/samtools>. Release archives are available at both GitHub and SourceForge. Installing the tools will also install manual pages for them. Once installed, you should be able to use "man samtools" etc at your command line to read the documentation. Manual pages and other documentation are also available online at <http://www.htslib.org/doc/>. John -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. |
From: John M. <jm...@sa...> - 2015-12-16 00:47:10
|
New 1.3 versions of Samtools, BCFtools, and HTSlib have been released. These have numerous bug fixes, improvements, and new commands. The changes are too numerous to list here, but we would particularly like to draw your attention to these three items: * The old way of invoking the samtools sort subcommand has been superseded since samtools 1.0 and from this release no longer works: samtools sort [-f] [-o] in.bam out.prefix If you do use this obsolete way of specifying sort options, samtools will now in most cases fail and give you a hint about the new ways. If you are still using sort ‑f, ‑o, or out.prefix, convert to use -T PREFIX and/or -o FILE instead. See the manual page for details. * The samtools rmdup command has been restored to its basic functionality. It no longer immediately aborts, but it remains not recommended for most use. * When HTSlib has been built to use libcurl, most tools can now access remote files over HTTPS and via AWS S3 in addition to the previous HTTP, FTP, and iRODS. See HTSlib's INSTALL and Samtools's README for details. For further details of these and other changes in these releases, see the release notes for each project linked to at <http://www.htslib.org/download/>, or see all three together at <http://sourceforge.net/projects/samtools/files/samtools/1.3/>. These releases can be found at <http://www.htslib.org/>. Development and source code of these projects are linked from that web site and can be found in various Git repositories under <https://github.com/samtools>. Release archives are available at both GitHub and SourceForge. Installing the tools will also install manual pages for them. Once installed, you should be able to use "man samtools" etc at your command line to find details of new commands and functionality. Manual pages and other documentation are also available online at <http://www.htslib.org/doc/>. John -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. |
From: Tim F. <tf...@tf...> - 2015-11-24 18:27:41
|
In accordance with the policy and timelines that have been displayed on the project page[1] for the last few months, , HTSJDK will be discontinuing support for versions of Java prior to Java 8. The change will be made in GitHub today, and will effect binary releases of HTSJDK starting with the next release. This change is being made for two major reasons: 1) Oracle has discontinued providing public updates for Java 7 as of April 2015 2) The developers of the project (of which I am one) feel that the language features and additional APIs available in Java 8 provide sufficient value to discontinue supporting earlier versions While the announcement only effects HTSJDK, it is likely that Picard and other downstream libraries effected by the change will make a similar move in the coming weeks or months. -t [1] https://github.com/samtools/htsjdk <https://github.com/samtools/htsjdk> |
From: Jason C. <jc...@br...> - 2015-02-23 18:11:05
|
Picard release 1.129 23 February 2015 - Enable file source for Bam Index creation. - Separate out the actual work for FastqToSam so an outside caller can directly pass in FastqReaders - A recent change to the command line parsing causes problems with scala based CLPs. In cases where the new system fails, fall back to the original system - Allow tools to work with GA4GH resources specified via urls. This first batch includes AddOrReplaceReadGroups, MarkDuplicates and ViewSam. HTSJDK changes - removing SAMRecord.setAlignmentEnd it throws unsupported operation exception - adding getResourceDescription() to SamReader - fixed issue 155: CRAMFileWriter injects slice refMD5 now; added slice refMD5 check into CRAMIterator, for now only error message is show. - Made ReadTag.buf ThreadLocal. Fixes https://github.com/samtools/htsjdk/issues/153 In addition to the following changes to the htsjdk we are now publishing our releases to sonatype so they should be available at maven central within a few minutes of each major release. -J |
From: John M. <jm...@sa...> - 2015-02-04 14:56:47
|
New 1.2 versions of Samtools and BCFtools and a 1.2.1 version of HTSlib have been released. These provide improvements in CRAM support and variant calling, as well as a number of bug fixes and speed improvements. (The 1.2 HTSlib release was immediately superseded by 1.2.1 when a compatibilty issue with previous HTSlib releases was identified.) These releases can be found at <http://www.htslib.org/>. Development and source code of these projects are linked from that web site and can be found in various Git repositories under <https://github.com/samtools>. Release archives are now being made available at both GitHub and SourceForge. Installing the tools will also install manual pages for them. Once installed, you should be able to use "man samtools" etc at your command line to find details of new commands and functionality. Manual pages and other documentation are also available online at <http://www.htslib.org/doc/>. John Noteworthy changes in BCFtools: * new consensus command * new annotate plugins: fixploidy, vcf2sex, tag2tag * more features in convert command, amongst others new --hapsample function (thanks to Warren Kretzschmar) * support for complements in bcftools annotate --remove * support for -i/-e filtering expressions in isec * improved error reporting * call command changes: - the default prior increased from -P1e-3 to -P1.1e-3, some clear calls were missed with default settings previously - support for the new symbolic allele <*> - support for -f GQ - bug fixes, such as: proper trimming of DPR tag with -c; the -A switch does not add back records removed by -v and the behaviour has been made consistent with -c and -m * many bug fixes and improvements, such as - bug in filtering, FMT & INFO vs INFO & FMT - fixes in bcftools merge - filter update AN/AC with -S - isec outputs matching records for both VCFs in the Venn mode - annotate considers alleles when working with Number=A,R tags - new --set-id feature for annotate - convert can be used similarly to view Noteworthy changes in Samtools: * Flagstat now works on SAM, BAM, or CRAM files (rather than BAM only) * Stats calculates mismatches per cycle for unclipped length * Merge can now merge SAM input files * CRAM reference files are now cached by default (see HTSlib below and samtools(1) man page) * Tested against Intel-optimised zlib (https://github.com/jtkukunas/zlib; see README for details) * Fixed bugs #302, #309, #318, and #327 Noteworthy changes in HTSlib: * HTSlib now has a configure script which checks your build environment and allows for selection of optional extras. See INSTALL for details * By default, reference sequences are fetched from the EBI CRAM Reference Registry and cached in your $HOME cache directory. This behaviour can be controlled by setting REF_PATH and REF_CACHE enviroment variables (see the samtools(1) man page for details) * Numerous CRAM improvements: - Support for CRAM v3.0, an upcoming revision to CRAM supporting better compression and per-container checksums - EOF checking for v2.1 and v3.0 (similar to checking BAM EOF blocks) - Non-standard values for PNEXT and TLEN fields are now preserved - hts_set_fai_filename() now provides a reference file when encoding - Generated read names are now numbered from 1, rather than being labelled slice:record-in-slice - Multi-threading and speed improvements * New htsfile command for identifying file formats, and corresponding file format detection APIs * New tabix --regions FILE, --targets FILE options for filtering via BED files * Optional iRODS file access, disabled by default. Configure with --with-irods to enable accessing iRODS data objects directly via irods:DATAOBJ * All occurences of 2^29 in the source have been eliminated, so indexing and querying against reference sequences larger than 512Mbp works (when using CSI indices) * Support for plain GZIP compression in various places * VCF header editing speed improvements * Added seq_nt16_int[] (equivalent to the samtools API's bam_nt16_nt4_table[]) * Reinstated faidx_fetch_nseq(), which was accidentally removed from 1.1. Now faidx_fetch_nseq() and faidx_nseq() are equivalent; eventually faidx_fetch_nseq() will be deprecated and removed [#156] * Reinstated hts_file_type() and FT_* macros, which were available until 1.1 but briefly removed in 1.2. This function is deprecated and will be removed in a future release -- you should use hts_detect_format() etc instead * Fixed bugs #141, #152, #155, #158, #159, and various memory leaks (Numbers in the form #NNN refer to corresponding issue numbers in the relevant GitHub repository.) -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. |
From: George G. <gg...@br...> - 2015-01-12 17:08:50
|
Picard release 1.128 12 January 2015 - Refactoring CollectWgsMetrics in order to be able to easily create CollectRawWGSMetrics. - Create RawWGSMetrics by overriding some default option values and repackaging the metrics class. - Moved CollectIlluminaSummaryMetrics (will be deleted eventually) - Fixed some test files. - Removed the buggy CLP CollectIlluminaSummaryMetrics.java and its test - Modified CommandLineParser class (and test) to fix overridable option fields - Modified PicardCommandLine class - allow --list-commands to print all available commands to STDOUT - allow colors to be disabled with -Dpicard.cmdline.color_status=false - Modified Interval List Tool to add functionality so that it can use VCF as a source for intervals - Modified VCFConstants - removing GATK-specific constants - Added test (SamSpecInTest) to check min/max values for i tag in SAM/BAM - Added input validation functions that can deal with both file and URL inputs, in preparation for allowing GA4GH resources as inputs into Picard tools. - Made static methods in TextCigarCodec and BinaryCigarCodec - isCRAMFile moved to SamStreams. Use readBytes instead of dis readFully - Fixed race condition in SAMFileWriterImpl where synchronouslyClose() would be called multiple times if AbstractAsyncWriter.close() called twice simultaneously - Enhancements to VCFFileReader: - Added functionality so that fromVCF can create an intervalList with or without filtered variants (defaulting to without, as currently) - Added functionality to fromVCF so that the intervals will end at the END value from the info field (if available). - Make SortingVariantContextWriter public. - Currently, the only way to access this functionality is through VariantContextWriterFactory.sortOnTheFly, but VariantContextWriterFactory is deprecated. George |
From: Jason C. <jc...@br...> - 2014-12-15 18:09:34
|
Picard release 1.127 15 December 2014 - CRAM file support - Make parseOptionsFile public, with a little more control over behavior. - SamReader conversion of Picard public. - Added common argument for reference fasta to support CRAM input/output. - Reference fasta should be optional. - Use SamReaderFactory makeDefault with reference sequence whenever possible to allow for CRAM support. - Added read group tag option, resolves #108 #114 - Various aesthetic tweaks for HTML doc generation. - Fixed Faked Filter files. Need them to be of size one element for the faked filter file to have the same number of elements as other faked types. |
From: George G. <gg...@br...> - 2014-12-01 15:37:31
|
Picard release 1.126 1 December 2014 - Fixed error in FilterFileFaker. Changed cluster count in faked filter file to be 0. Added unit test to test for readability of faked files. - Modified FastqToSam to strip the "/2" when using STRIP_UNPAIRED_MATE_NUMBER - Modified ViewSam. Added option to show only the header or only the records. -George |
From: George G. <gg...@br...> - 2014-11-19 21:53:54
|
Picard release 1.125 19 November 2014 - Add new statistic 'Specificity' to GenotypeConcordance tool - Removed some sun.reflect imports from codebase George |
From: George G. <gg...@br...> - 2014-11-03 16:38:24
|
Picard release 1.124 3 November 2014 - Change Picard Command Line Programs to ALL execute from single JAR This makes a fundamental change to how we run Picard. We now have a single JAR (command line program), rather than multiple JAR (one per tool). In this brave new world, the single command line program is given the the command line program name (ex. SamView) to specify which tool to run. We also provide a convenient summary of all of the tools available. For the developers, we add a facility to add new command line programs to the single JAR, by extending the PicardCommandLine and adding the package name(s) for us to search for classes that extend CommandLineProgram. The usage is now put in CommandLineProgramProperties, where we can also specify short usages (for the summary help message), program versions, and program groups. The latter is useful for grouping tools that operate on common filetypes or have similar function (ex. SAM/BAM). We made a number of new tools public, coming from our internal toolset. These include: BaitDesigner, CalculateReadGroupChecksum, CheckTerminatorBlock, CollectIlluminaBasecallingMetrics, CollectIlluminaLaneMetrics, CollectJumpingLibraryMetrics, CollectOxoGMetrics, CollectQualityYieldMetrics, CollectRrbsMetrics, GatherVcfs, LiftOverIntervalList, ScatterIntervalsByNs, and SplitSamByLibrary. These may prove useful, or not. - Changed Output of GenotypeConcordance tool Additional metrics file output which contains the raw counts of contingency values (i.e. TP for true positives, FN for false negatives...) - New CLP 'CollectIlluminaSummaryMetrics' to collect coverage information according to Illumina-defined filters. - New CLP 'RenameSampleInVcf' to rename a sample in a VCF - New CLP 'FilterVcf' that provides simple hard filtering functionality for VCFs. - Added ability to open SamReader from string, specifying either a URL or a file path. This will make it easier treat INPUT parameters in Picard tools uniformly regardless of whether they designate a file or URL resoures. Url detection and decision of whether the resource is file based on url based is done in SamInputResource. - Some support for VCF v4.2 files added. Handles "Number=R" for INFO fileds in the header. Other VCF v4.2 specific additions (Eg: Source and Version fields in INFO header lines) are not handled, but appear to be silently ignored. Tested on output of samtools mpileup | bcftools call (1.0-17-gfaf4dd6, 1.0-55-gc661821, using htslib 1.0-11-g830ea73) - Bug fix: 'fixed the behavior of BCF2Utils.toList() when it's given an array' Old version return a List with a single element, which was the provided array. This broke BCF writing, in particular when a FORMAT annotation used arrays instead of Lists Made toList generic, rather than returning a List<Object> Removed the redundant BCF2FieldEncoder.toList() Added unit tests - Bug fix in Abstract AlignmentMerger Consider soft clipping at the ends of reads when clipping reads that overhang the reference - Added a button to the explain-flags.html page. This button switches the flag from the value of a read, to that of its mate (as much as can be inferred). Enjoy! George |
From: George G. <gg...@br...> - 2014-10-20 15:48:34
|
Picard release 1.123 20 October 2014 - Updates to "GenotypeConcordance" Command Line Program -- Update the genotype concordance scheme to have called variants that are filtered be counted as though they are missing -- Improvement to handle 'mismatching' ref alleles for indel comparison conditions. -- Changed default extension of metrics files - Improvements to htsjdk testing -- Cleanups regarding temporary files in tests -- Delete index files for temp files at the end of testing, and do not write temp files within source directory. - George |
From: George G. <gg...@br...> - 2014-10-08 15:29:36
|
Picard Release 1.122 8 October 2014 - New Command Line Program "GenotypeConcordance" -- Calculates the concordance between genotype data for two samples in two different VCFs - one being considered the truth (or reference) the other being considered the call. The concordance is broken into separate results sections for SNPs and indels. Summary and detailed statistics are reported. Note that for any pair of variants to compare, only the alleles for the samples under interrogation are considered and MNP, Symbolic, and Mixed classes of variants are not included. - New Command Line Program "UpdateVcfDictionary" -- Updates the sequence dictionary of a VCF from another file (SAM, BAM, VCF, dictionary, interval_list, fasta, etc). - New Command Line Program "VcfToIntervalList" -- Create an interval list from a VCF - New Command Line Program "MarkDuplicatesWithMateCigar" -- A new tool with which to mark duplicates: This tool can replace MarkDuplicates if the input SAM/BAM has Mate CIGAR (MC) optional tags pre-computed (see the tools RevertOriginalBaseQualitiesAndAddMateCigar and FixMateInformation). This allows the new tool to perform a streaming duplicate marking routine (i.e. a single-pass). This tool cannot be used with alignments that have large gaps or reference skips, which happens frequently in RNA-seq data. There were many refactors of the old MarkDuplicates and MarkDuplicatesWithMateCigar, since the share common code. EstimateLibraryComplexity was caught up in this too. Many, many, many unit tests were added to were added to prove equivalency of MarkDuplicatesWithMateCigar to MarkDuplicates. This also exposed a few one in a million corner cases in MarkDuplicates both in duplicate marking as well as optical duplicate detection. This results in MarkDuplicates needing to write slightly larger temporary files when running. SamFileTester was also improved to handle the various test cases for duplicate marking testing. - Updates to IntervalList: -- Added capacity to create a simple interval list from a string (the name of the contig) -- Added the capacity to subtract one interval list from another (currently it would only work if they were both wrapped inside a container) - Updates to SamLocusIterator -- Performance optimizations gaining about 35% speed up... - Updates to MarkDuplicates: -- Removed unnecessary storage of a string in the Read Ends in Mark -- Clarifed the size of ReadEndsForMarkDuplicates - Updated the minimum number of times that the BAIT_INTERVALS (in CalculateHsMetrics) and TARGET_INTERVALS (in CollectTargetedMetrics) must be set to one. - Moved CollectHiSeqPfFailMetrics into picard public - Updates to documentation generation (internal): -- changed link to IntervalList.java documentation -- updated how _includes/command-line-usage.html is generated - Moved SAMSequenceDictionaryExtractor and tests from picard to htsjdk - George |
From: John M. <jm...@sa...> - 2014-09-23 16:37:40
|
New 1.1 versions of htslib, samtools, and bcftools have been released. These are minor bug-fix releases, and include the following notable fixes and new functionality compared to 1.0: * Samtools fixmate and flagstat now consider supplementary reads * Sorting BAM files with thousands of reference contigs now completes in a reasonable amount of time * Fixed samtools idxstats when displaying statistics from indices generated by samtools 0.1.x * Fixed samtools calmd memory leak * Samtools fixmate now only adds a template cigar tag (ct:Z) when requested with -c, and never adds it repeatedly * Regularised script #! directives as #!/usr/bin/env perl etc * Fixed DPR annotation in samtools mpileup * New bcftools convert and plugin commands and annotate --rename-chrs option * BCFtools norm performance is improved and now averages QUALs and accumulates IDs and FILTERs * Improved bcftools filter expressions, query support for IUPAC ambiguity codes, and annotate support for genotype fields * Plugins for bcftools have now moved from annotate to the new plugin command These releases can be found at <http://www.htslib.org/>. Development and source code of these projects are linked from that web site and can be found in various Git repositories under <https://github.com/samtools>. Updates to the htslib.org web site are ongoing. In particular, we have recently added a sample CRAM workflow: see <http://www.htslib.org/workflow/#mapping_to_cram>. Installing the tools will also install manual pages for them. Once installed, you should be able to use "man bcftools" etc at your command line to find details of new commands and functionality. Manual pages are also available online at <http://www.htslib.org/doc/> and these will soon be updated to reflect the 1.1 release. John -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. |
From: Bradley T. <br...@br...> - 2014-09-22 19:39:30
|
Picard/HTSJDK Release 1.121 September 22, 2014 Picard: Added a static function to PedFile that allows the creation of a pedFile object from a Map<String,Sex>. Fixed BAMRecordCodec to recognize when qual array has been changed. Added a SortVcfs CLP that will sort VCF files by contig and genomic position. Supplemental alignments are now considered when fixing mate information in FixMateInformation and RevertOriginalBaseQualitiesAndAddMateCigar. RevertSam: Added ability to use original qualities to detect encoding scheme if they are being restored. Modified SANITIZE method to convert all non-Standard quality score encoding schemes to Standard. Fixed issue with validation stringency not being propagated. SamAlignmentMerger: Moved the test for duplicate @PG.IDs into AbstractAlignmentMerger to avoid opening the unmapped SAM file twice. SamToFastq: Removed redundant IOUtil.openFileForWriting() calls for the fastq writers. CommandLineProgram: Bugfix - the validation stringency command line option is now passed to the sam reader factory HTSJDK: Significant work was done towards making HTSJDK compatible with maven-based repositories such as Maven Central and Sonatype. VCFContigHeaderLine: Fixed bug in setSequenceDictionary where where the assembly value of the updated contigs was not being set properly in the VCF header. Fixed a bug in VCFUtils.smartMergeHeaders related to contig ordering. Made some useful additions to SAMTestUtil In build.xml, changed htsjdk.version.property.xml to htsjdk.version.properties. See https://github.com/samtools/htsjdk/issues/99 Replaced use of TestNG in SAMTestUtil. Only return the first error when validating a SAMRecord using STRICT, for increased performance. Added CustomReaderFactory for plugging in external implementations of SamReader capable of getting data from APIs such as Google Genomics. QualityEncodingDetector: Add the option of using original quality scores for determining the encoding scheme. Introduced several fixes related to JDK8 in SamFileHeaderMergerTest SamPairUtil: Added methods to help set mate cigars and mate information given a queryname sorted iterator. This is useful when we want to fix or update the mate information, including the mate cigar. IOUtil: Added methods to check for readability/writability of list of files Added copyrights to some recently added classes IntervalList: No longer enforce by default that intervals that will be merged have the same strand SamReaderFactory: Exposed a way to set the default validation stringency |
From: Bradley T. <br...@br...> - 2014-09-08 19:32:02
|
Picard release 1.120 September 8th, 2014 MAJOR CHANGE As part of our switch to Github, the Picard project page has moved here: http://broadinstitute.github.io/picard/ The latest release is available for download at the address above, or from Github here: https://github.com/broadinstitute/picard/releases The HTSJDK java library also has a new page: http://samtools.github.io/htsjdk/ Importantly, the following sites will no longer provide up-to-date picard documentation / binaries: http://picard.sourceforge.net/ http://sourceforge.net/projects/picard/files/ Release notes for version 1.120 - Made the release script push releases, javadoc, and the website documentation to github pages rather than sourceforge. - Added a tool to convert BED files to Picard Interval Lists. - Moved CircularByteBuffer to its own class - Modified MathUtil to have better named and more useful functions - Fixed oversight in SANITIZE block resulting in NPE if there's only one read group. - Added an option to VCFWriter to write the entire FORMAT field, even if the trailing values are missing and could therefore be trimmed. Also fixed ParsingUtils.split() to correctly handle the case where the only delimiter is the first character Brad |
From: Bradley T. <br...@br...> - 2014-08-25 20:46:59
|
Picard release 1.119 25 August 2014 - Updated AbstractOpticalDuplicateFinder to handle post-CASAVA 1.8 read names. Pass READ_NAME_REGEX=null to skip optical duplicate detection in MarkDuplicates. - Fixed RevertSam to harmonize different quality score encoding schemes when the sanitize option is set. - Modified Casava18ReadNameEncoder to correctly display filter status. - SamPairUtil: Added the ability to set the mate cigar tag on supplementary alignments. Updated AbstractAlignmentMerger to set mate cigars when merging supplementary alignments. - CollectWgsMetrics: Added 2 new metrics- the fraction of bases attaining 15X and 25X coverage respectively. - Added a new tool, CollectBaseDistributionByCycle, that computes the nucleotide distribution per cycle. - FileAppendStreamLRUCache: Added wrapping of FileOutputStreams with BufferedOutputStreams. - Fixed a bug in BCF2Writer where it was double-closing the output stream. - Fixed VariantContext to ignore symbolic alleles when running strict validation for a VCF. - MetricsBase: Fixed a NullPointerException - IndexStreamBuffer: Fixed a bug in which a data-exhaustion error was being thrown prematurely. -Brad -- Brad Taylor Associate Software Engineer, Genomics Platform Broad Institute 320 Charles Street, Cambridge MA 617-714-7721 |
From: John M. <jm...@sa...> - 2014-08-15 17:58:00
|
Major new versions of samtools and associated tools have been released. Significant changes include: * support for CRAM, a sequence data file format with better compression than BAM * support for BCF v2, a binary equivalent to the VCF format that is more flexible than the old BCF1 * bcftools has been split out from samtools as a fully-featured VCF/BCF manipulation toolkit * much work on the variant calling algorithms * replacement of samtools's SAM/BAM readers and writers with new implementations in htslib * a more flexible API for writing programs using htslib's API rather than the old samtools bam.h (which will remain supported and usable for a while yet) BCFtools has been significantly expanded to provide many tools for basic manipulation of VCF and BCF files. Both SAMtools and BCFtools now use HTSlib for their low-level SAM/BAM/CRAM and VCF/BCF file operations. Due to these reorganisations there are many small changes and improvements -- notably most tools automatically detect what kind of input files they have been provided with. These releases can be found at <http://www.htslib.org/>. Development and source code of these projects are linked from that web site and can be found in various Git repositories under <https://github.com/samtools>. The web site at <http://samtools.sourceforge.net> contains information for the old 0.1.x samtools/bcftools releases which is still useful but needs some updating for the 1.0 release. The <http://www.htslib.org/> website will be updated over the coming days and weeks with new workflows and updated information from the old website. It has been a long time since the last samtools release -- we anticipate making more frequent samtools, bcftools, and htslib releases in future. John -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. |
From: Alec W. <al...@br...> - 2014-07-28 15:35:06
|
Picard release 1.118 28 July 2014 - Fixed index out of bounds exception in CollectTargetedPcrMetrics. - Get IntelDeflater working again -- had been disabled since package name change. - Genotype.java: Add a function to determine if a genotype is heterozygous non-reference (e.g. C/T if the reference is A) - HardyWeinbergCalculation.java: Fixed HW statistic overflow at large sample numbers. - Update TribbleIndexedFeatureReader.java, TabixReader.java, and TabixFeatureReader.java to add new constructor with given index file. Update AbstractFeatureReader.java to pass index file forward to specific readers. Pass index file in AbstractFeatureReader to underlying readers. - Fixed race condition where synchronouslyClose() would be called multiple times if AbstractAsyncWriter.close() called twice simultaneously. Allowing close() to be called multiple times as per java.io.Closeable interface specifications. - FastqReader: added constructor, toString, removed extra-line. - Add constructor in VCFFileReader that allows to specify the index resource. - Fix problem with open TabixReader after calling close() on the TabixFeatureReader. - Fix signature of CloserUtil.close(List) so that is it more likely to be invoked. - When querying for a contig which doesn't exist in SAMFileReader, return an empty iterator. This matches the behavior in BAMFileReader. -Alec |
From: Alec W. <al...@br...> - 2014-07-14 17:07:47
|
Picard release 1.117 14 July 2014 - Added a divide function to Math.Utils. - Remove from Picard libraries that are found in htsjdk, since we can use those, as we are tightly coupled to htsjdk. - Updated RnaSeqMetricsCollector to make coverage collection optional. - Add travis-ci to Github project. - Explicitly implement Closeable on classes and interfaces with close(). - Enhance TabixReader: iterator moved to interface. - Added the Damerau-Levenshtein algorithm to calculate a distance between strings into StringUtil. - GitHub-51: Fix NPE if idxFn TabixReader ctor arg is non-null. - ReferenceSequenceFileWalker.java: Checks if ReferenceFile is indexed, and if so, uses it -Alec |
From: Alec W. <al...@br...> - 2014-06-30 15:06:21
|
There have been no substantive PIcard changes since release 1.115, so release 1.116 will be skipped. -Alec |
From: Alec W. <al...@br...> - 2014-06-16 14:35:28
|
Picard release 1.115 - EstimateLibraryComplexity: Control size of sorting collection with MAX_RECORDS_IN_RAM - Adds COMPRESS_OUTPUTS parameter to IlluminaBasecallsToFastq so that it can generate compressed FASTQ files (with file extension .fastq.gz). Default is false. - SamToFastq: Clarify what a "non-PF read" is. - Avoid logging a NullPointerException if IlluminaBasecallsToFastq is started with FORCE_GC=false. GC timer write to the 'log' instead of directly to System.out. There is really no reason to handle this information differently than other logging. - Adds BARCODES_DIR parameter to IlluminaBasecallsToFastq and IlluminaBasecallsToSam so that they can use the barcode files from ExtractIlluminaBarcodes when it has been run with OUTPUT_DIR different from the BASECALLS_DIR parameter. The use case is that the BASECALLS_DIR directory is read-only and the barcode files can't be written to that directory and the other tools are useless if they can't read the barcode files from a different directory. - Remove bzip2 classes from non-executable picard-<version>.jar - MarkDuplicates: Indicate in INPUT doc string that it may not be streamed. - Remove dependence on JAVA6_HOME environment variable. - Use a StringBuffer instead of synchronization when generating log messages. The only exception is if the stacktrace of a Throwable also needs to be written. Then we have to ensure that our message goes with the stacktrace so we need synchronization there. But this should not be a drawback since the Throwable.printStackTrace() itself synchronizes on the PrintStream. - Synchronize on the 'out' PrintStream instead of 'this' to avoid mixing up log statements from different threads. - Use the same buffer as the rest of htslib in the reference sequence / fasta file classes. - FastqRecord: added methods equals, hashcode, toString, length - Detect old style metric file and skip over header reading code. - Add additional boolean genotype functions to JEXLMap to add functionality to GATK tools using JEXL (like VariantFiltration) - Case of symbolic allele ID string now being preserved as per VCFv4.1 section 1.4.5 - BAM query optimization: Don't iterate over bins that are not actually interesting - Fix nullpointer exception when BAMFileSpan is null - IntervalTreeMap: added two methods to test interval overlapping - Add reference sequence index cache to speed up query by 20% for certain usecases - VariantContextBuilder: no longer crashing when an attribute is set without loading attributes or explicitly calling .attributes(null) first - Accepting VCF 4.2 breakend and inline symbolic insertion alleles - Better message when reference fasta index does not exist. - AbstractBAMFileIndex.java: 40% speed improvement for query under certain workloads -Alec |
From: Alec W. <al...@br...> - 2014-05-20 14:00:23
|
Picard release 1.114 20 May 2014 Note: This is the first release based on GitHub code. Generating this release was somewhat bumpy. As usual, please let us know if you experience any problems. - Complete reorganization of packages. Map of old to new package names is forthcoming, but the basic change is "net.sf.samtools" => "htsjdk.samtools" and "net.sf.picard" => "picard." - Enable `CollectTargetedPcrMetrics` and `CalculateHsMetrics` to target multiple baitsets. - Both command line programs' `TARGET_INTERVALS` argument accept multiple values. - `CalculateHsMetrics`'s `BAIT_INTERVALS` accepts multiple values. - Add new modes to `IntervalListTools`. - `INTERVAL_SUBDIVISION`, which is the default mode, and functions like the previous version of `IntervalListTools`. - `BALANCING_WITHOUT_INTERVAL_SUBDIVISION`, which emits interval lists without splitting them up, and sizes the output interval list sizes to be no smaller than the largest input interval. - Added an ASSUME_SORTED flag to FixMateInformation to allow grouped reads to be analyzed directly without prior sorting. - IlluminaBasecallsConverter: Flush sorting collections when a barcode within a tile has completed to avoid running out of memory - Add `SamReaderFactory` as a new facility for reading from SAM files. - Produces `SamReader` objects, which are similar to `SAMFileReader`. - Deprecate `SAMFileReader`, as `SamReaderFactory` is the preferred approach to creating things that read SAMs. `SAMFileReader` still functions as normal. - See `SamReaderFactory`'s javadoc for examples of how to use it. -Alec |
From: Alec W. <al...@br...> - 2014-05-05 14:50:26
|
Picard release 1.113 5 May 2014 NOTE: This is the last release that will be based on source code in the Sourceforge repository. The next release will use GitHub. However, binary distributions will continue to be done via Sourceforge. - SecondaryAlignmentFilter.java: Addition of a simple SamRecordFilter that filters out secondary alignments but not supplementary records. - Changes to SamLocusIterator to a) check the mapping quality only once per sam record instead of at every base and b) add the ability to filter out non-PF reads during pileup construction. - First version of CollectWgsMetrics a program that calculates metrics for evaluating whole genome shotgun sequencing experiments. - RevertOriginalBaseQualitiesAndAddMateCigar.java: do not check for OQ when we are not reverting OQs. - VariantContextWriterBuilder.java: Require a reference dictionary for Tribble index-on-the-fly but not Tabix. - Defaults.java: adding a reference fasta to the Defaults to facilitate CRAM reading/writing. - VCFEncoder.java: Make two methods public so Hadoop-BAM can use them. - New program ReplaceSamHeader. - New program AddCommentsToBam. - Optimize Illumina BCL reading. - Fix for CheckIlluminaDirectory when the number of clusters is different per tile -Alec |
From: Alec W. <al...@br...> - 2014-04-29 18:01:04
|
Hi Picard community, Some major changes to the Picard project will be happening this weekend 3 May -- 4 May. The motivations for these changes are the following: More clearly separate API code from programs to provide a compact usable API for next gen sequencing informatics in Java. Prepare for the merger of the sam-jdk and cram-jdk codebases. Switch from Subversion hosted on Sourceforge to Git hosted on GitHub, to facilitate collaborative software development. The following changes will be made: Picard source will be moving from Sourceforge to GitHub. Picard binaries will continue to be published to Sourceforge. The package structure will change: htsjdk package will contain SAM, BAM, Tribble and VCF code. Repository: https://github.com/samtools/htsjdk picard package will contain command-line tools. Repository: https://github.com/broadinstitute/picard There will be two library jars: htsjdk and picard, in contrast to the current organization where sam-jdk, tribble and vcf are in separate jars. There will be some movement of code from the current picard package into the htsjdk package. Other things to be aware of: Eventually htsjdk will support other file formats, e.g. CRAM. The new source repositories may be unstable for the next 2 weeks or so. Legacy source code will still be available on Sourceforge for a while. These changes should not affect users of the command-line tools, only users of the libraries and source code. Please let us know if you have questions or concerns. Thanks, Alec |
From: Alec W. <al...@br...> - 2014-04-21 12:40:06
|
Picard release 1.112 21 April 2014 - AbstractVCFCodec.java: Add ability to remap sample names for single-sample vcfs to the vcf codec. By calling setRemappedSampleName() on an instance of a vcf codec, you instruct it to replace the existing sample name in the vcf header with a new value. Attempting to use this feature with multi-sample or sites-only vcfs throws an error. - Make TextTagCodec.java and TagValueAndUnsignedArrayFlag.java public - Move BLOCK_COMPRESSED_EXTENSIONS to Tribble so all projects use the same set of extensions. - Create Tabix indices for block-compressed VCFs - Deprecate VariantContextWriterFactory. Convert uses of VariantContextWriterFactory to VariantContextWriterBuilder. -Alec |