Re: [Transdecoder-users] Announcement: Transdecoder release r20140704
Extracting likely coding regions from transcript sequences
Brought to you by:
bhaas
Menu
▾
▴
Re: [Transdecoder-users] Announcement: Transdecoder release r20140704
|
From: Martin M. <mmo...@gm...> - 2015-01-09 10:46:38
|
Hi Brian and Alexie, thank you for your answers. I made the following packages for Gentoo Linux. Here are some comments how the layout could be improved. 1. Gentoo picks TransDecoder_r20140704/3rd_party/ffindex-0.9.9.3 subdirectory contens from TransDecoder_r20140704.tar.gz and installs it as a separate package. Ideally, once I figure out where is the official source the source address will be changed. Somebody knowing transdecoder could document whether transdecoder requires a special version of this tool or not. 2. Gentoo picks TransDecoder_r20140704/3rd_party/parafly-r2013-01-21 subdirectory contens from TransDecoder_r20140704.tar.gz and installs it as a separate package. Passing extra arguments to configure cause a broken Makefile or config.h is created. In turn, gcc later on dies about unresolved MPI symbols because "-fopenmp" was not pulled into CFLAGS. I got around by passing only '--prefix=/usr' and deliberately discarding other arguments Gentoo applies on general (to specify for example CPU architecture, need for cross-compiling). I think it has to do with autoreconf being called for some reason but did not study this in more detail. Again, would be nice if parafly was available separately to download, and transdecoder documented which versions it eventually requires, ideally could check for that in configure. 3. Gentoo already has cd-hit package, like other distro's, again, installs the binaries into /usr/bin accessible via $PATH. 4. The main Makefile in TransDecoder_r20140704/ is after the above changes not necessary. Fetching Pfam 1.5TB data files and their indexing can be done by a standalone shell-script and the rest of the Makefile was just compiling and installing the 3rd_party tools. That is not needed for us now so the automated system drops the Makefile altogether. But how to install the thingie at all? What was missing was where to and how to install the transdecoder files. I placed 'TransDecoder *.pl util/*.pl util/*.sh' files into /usr/bin now. I have no idea, but maybe of them could be more hidden in some other path, like /usr/share/TransDecoder if they are not to be directly executed by end-user. Here is their current listing: /usr/bin/TransDecoder /usr/bin/cdna_alignment_orf_to_genome_orf.pl /usr/bin/compute_base_probs.pl /usr/bin/cufflinks_gtf_genome_to_cdna_fasta.pl /usr/bin/cufflinks_gtf_to_alignment_gff3.pl /usr/bin/cufflinks_gtf_to_bed.pl /usr/bin/ffindex_gather.sh /usr/bin/ffindex_resume.pl /usr/bin/gene_list_to_gff.pl /usr/bin/get_top_longest_fasta_entries.pl /usr/bin/gff3_file_to_bed.pl /usr/bin/gff3_file_to_proteins.pl /usr/bin/index_gff3_files_by_isoform.pl /usr/bin/nr_ORFs_gff3.pl /usr/bin/pfam_mpi.sh /usr/bin/pfam_runner.pl /usr/bin/remove_eclipsed_ORFs.pl /usr/bin/score_CDS_liklihood_all_6_frames.pl /usr/bin/seq_n_baseprobs_to_logliklihood_vals.pl Finally, 'PerlLib/*.pm' gets placed now into e.g. /usr/lib64/perl5/vendor_perl/5.16.3/ but it would be great if the *.pl tools could look for PERL5INC/TransDecoder/ instead, that is, to anticipate the modules in a subdirectory. Just to avoid filename clashes on the filesystem. Current layout is here: /usr/lib64/perl5/vendor_perl/5.16.3/Fasta_reader.pm /usr/lib64/perl5/vendor_perl/5.16.3/GFF3_utils.pm /usr/lib64/perl5/vendor_perl/5.16.3/Gene_obj.pm /usr/lib64/perl5/vendor_perl/5.16.3/Gene_obj_indexer.pm /usr/lib64/perl5/vendor_perl/5.16.3/Longest_orf.pm /usr/lib64/perl5/vendor_perl/5.16.3/Nuc_translator.pm /usr/lib64/perl5/vendor_perl/5.16.3/TiedHash.pm In summary, Gentoo Linux users can install cd-hit, ffindex, parafly and transdecoder from science overlay now. They may go into the official, main tree after some testing. Debian and other distro's sometimes take over the logic from Gentoo to build their binary packages. Gentoo stays with compilation from source code. Maybe other distro developers will contact you about the same: split out the 3rd-party apckages, document whether transdecoder requires a special version of any of them, to make it possible to install TransDecoder into a custom path. That usually boils down to Makefile containing $(DESTDIR) variable placed at the beginning of every target installation path, which is derived from the configure.ac and configure.in files, respectively. That allows one to install files into a *temporary* location like /tmp/blah/usr via "make install DESTDIR=/tmp/blah, and the files are guaranteed to work once extracted from the package tarball and placed into /usr. I occasionally contribute some packages to Gentoo but I cannot fix every package, one needs to know how the tools call each other, what version they require, etc. So, it is easier to contact authors and provide feedback what could be adjusted to make the package more easily integrated into a distribution. With best regards, -- Martin Mokrejs, Ph.D. Adapter/artefact removal from datasets based on the following technologies: 454 / IonTorrent / Evrogen MINT / Clontech SMART / ..., Illumina http://www.bioinformatics.cz/software/supported-protocols/ Ale...@cs... wrote: > Hi Martin > > > > Trinity and siblings are community open-source software. They are also not funded by any funding body to have professional developers on the the payroll, we do it because we care JSo, with the exception of responding to requests from biologists who can’t code, our interest is to invite clever computational people such as yourself to implement the fixes they suggest. > > > > Soon we will be on github and you will be able to fork and do a pull request but in the meantime just email us a patch from the current sourceforge HEAD. If Brian is happy he may just give you direct write access to the repository. > > > > Thanks! > > a > > > > *From:*Brian Haas [mailto:bh...@br...] > *Sent:* Friday, January 09, 2015 07:48 AM > *To:* Martin MOKREJŠ > *Cc:* tri...@li...; tra...@li... > *Subject:* Re: [Transdecoder-users] Announcement: Transdecoder release r20140704 > > > > Hi Martin, > > > > Not yet... after I get the next Trinity release out, I'll work on updating transdecoder. I don't think it's going to meet your expectations as far as how the build goes, but it'll be better than it is now. We can certainly continue discussions before the release goes out. I'll check with you beforehand. > > > > best, > > > > ~b > > > > On Thu, Jan 8, 2015 at 3:42 PM, Martin MOKREJŠ <mmo...@gm... <mailto:mmo...@gm...>> wrote: > > Hi Brian, > I just returned to this and ... I don't see a never transdecoder release at sourceforge. So, did you get to splitting the packages out or not yet? > Thank you, > Martin > > Brian Haas wrote: >> Hi Martin, >> >> responses below >> >> >> On Fri, Jul 4, 2014 at 9:50 AM, Martin MOKREJŠ <mmo...@gm... <mailto:mmo...@gm...> <mailto:mmo...@gm... <mailto:mmo...@gm...>>> wrote: >> >> >> >> Brian Haas wrote: >> > Thanks, Martin. I've CC'd the trinity-developers list. We'll take all your comments into consideration. It'll be some time before it all meets your expectations (if that's even an option for us). For now, our (perhaps my) goals have been quite simple: keep everything self-contained and minimize dependencies. I entirely agree about that evil wget (which is one of the reasons why I put the 'make simple' in there). This version will be incorporated as a plugin in the upcoming Trinity, along with Jellyfish-2 (even if currently to your dismay), and Trinity will do the 'make simple' to avoid pulling down pfam via wget. >> >> Sure, take your time. Also, the packaging is an issue due to many different LICENSEs used in all the bundled tools. Just split it all into sub-packages, that's the only way. >> >> >> Right... that's definitely a cause for concern, which is why we keep the third-party code isolated wherever possible, and should have a note in there somewhere to follow the different licenses for the plugins. We only leverage code that has very lenient licensing, as we do for our code. >> >> >> >> So where does trinityrnaseq_r20140413p1/trinity-plugins/jellyfish-1.1.11 come from? It doesn't seem to be from http://www.genome.umd.edu/jellyfish.html or ftp://ftp.genome.umd.edu/pub/jellyfish/jellyfish-2.1.3.tar.gz >> >> >> It should have come from the jellyfish website. In the upcoming Trinity release, we're just going to bundle in the .tar.gz files for the code, and have the makefile do the unpack/build as part of the Trinity build. This will go for RSEM-2.15 as well. >> >> One of the key issues here is that there are some versions of the tools where the usage has changed significantly (both the latest rsem and jellyfish), so that our new version of Trinity will only be readily compatible with those specific versions - which is why they get bundled in, and Trinity will look in the plugins area to find what it needs. >> >> After we get the next Trinity, Trinotate, and PASA releases out, we'll rethink our bundling strategy. >> >> best, >> >> ~b >> >> >> Martin >> >> >> > >> > cheers, >> > >> > ~brian >> > >> > > >> > On Fri, Jul 4, 2014 at 8:46 AM, Martin MOKREJŠ <mmo...@gm... <mailto:mmo...@gm...> <mailto:mmo...@gm... <mailto:mmo...@gm...>> <mailto:mmo...@gm... <mailto:mmo...@gm...> <mailto:mmo...@gm... <mailto:mmo...@gm...>>>> wrote: >> > >> > Brian Haas wrote: >> > > Greetings all, >> > > >> > > The latest release of TransDecoder is now available: >> > > >> > > http://sourceforge.net/projects/transdecoder/files/TransDecoder_r20140704.tar.gz/download >> > > >> > > including minor changes from the previous release to ensure better compatibility with other projects, including Trinity, PASA, and Trinotate >> > > >> > > Release notes: >> > > >> > > -added 'make simple' to build just the essential components involving parafly and cdhit >> > > >> > > -removed the 'cds.' prefix from the pep and cds sequence accessions. >> > >> > >> > Hi Brian, >> > I just tested the new and have some comments: >> > >> > 1. In the past the files were tar.bz2 instead of tar.gz as of now. It helps distro maintainers if the URLs and filenames remain stable. It is also a common habit that if one unpacks MyApp-2.4c.tar.gz that it extracts into MyApp-2.4c/ subdirectory. Although it seems your today's archive file complies with this I think trinity does not and not sure how long will it last. ;) >> > >> > 2. It is evil that the "make compile" step runs wget to download 1.4GB large PFAM file. Please put it under different "target" in your Makefile's. Not only, I already have the files on my system and I certainly do not want to waste my bandwidth. >> > >> > 3. I would like to add this to Gentoo Linux but that won't ever be allowed if the package is huge glue of other tools. For example, I have already cd-hit installed and installing TransDecoder would try to overwrite existing files, and will be denied. If you would like to get the package accepted into Linux distros and save developers time resolving the knotted layout, please introduce some configure- or Makefile-based checks and bail out if they are not installed. You can keep the crazy layout/setup as an alternative for users who think this is the right way to go (while it is not). >> > >> > 4. I wanted to post trinityrnaseq-users list about this but ... it is confusing that trinity and transdecoders place overlapping 3rd-party stuff under its own source tree. The http://trinityrnaseq.sourceforge.net/#installation page it totally quiet how all there hidden obstackles. I recommend you to sum up a simple listing of required/optional tools, their versions and URLs. If possible, drop them from the TransDecoder_r20140704/3rd_party and also from the plugins subdirectory somewhere under trinity*. >> > >> > 5. In the current setup, both transdecoder, trinity are un-manageable for a Linux distro. One cannot force some version dependencies, the tools download what they want to on their own instead of just running a compiler ... and tehy overwrite other applications files. >> > >> > 6. BTW, I realized quorum package looks for jellyfish-1.11 while on the web I found only jellyfish-2.x. Incidentally, I see jellyfish-1.11 under trinity*. Huh. Would you please tell me: whether trinity uses an "old" jellyfish version of teh same package? Or is that that incidentally same name? Why can't trinity use jellyfish-2.x installed already on the system. >> > >> > I wish it helps you and other devs to cleanup the interesting package, though I did not get to install it yet. >> > >> > Thank you, >> > Martin > > > > > > -- > > -- > Brian J. Haas > The Broad Institute > http://broadinstitute.org/~bhaas <http://broad.mit.edu/~bhaas> > > > |
