basedb-users

[base] array design GTF import - not 9 columns

[Bob M. <r.m...@im...>]

Hi all,

[mods please delete the mail I sent from my gmail account, or add me with
that account but with no delivery, thank]

Just upgraded from 2->3 and now trying out the NGS feaures.  Quick question
about GTF import for array designs...

When doing array design->import the importer is complaining:

"Error: A line must have at least 9 columns, at line: 1"
stack trace below
(using GTF reporter map importer (version 3.2.5-dev))

The same error came up when we first created the array design (because I
checked the "validate files" box.

I know the first line has fewer columns because it has some # commented
lines

so in the importer configuration I configured the "Ignore" field like
this:  ^#.*

But still the same error...

Even if I set Default error handling = skip it's the same.  Seems to be
something deeper than the usual tab delimited importer parser settings.

I guess we should make a new GTF file with no comments but I just wanted to
check if this was the intended behaviour?

cheers,
Bob

Re: [base] array design GTF import - not 9 columns

[Jari H. <jar...@me...>]

On 2014-05-23 18:17, Bob MacCallum wrote:
> [mods please delete the mail I sent from my gmail account, or add me
> with that account but with no delivery, thank]

Added to the list.

For the issue with the GTF I'll let Nicklas respond.


Cheers,

Jari

Re: [base] array design GTF import - not 9 columns

[Nicklas N. <nic...@me...>]

Thanks for testing this. It feels like it was a long time ago this was 
implemented and we have not really started to use it yet. Anyway, you 
are correct that the problem is something deeper than the usual importer.

To make GTF files parsable by the regular importers, there is a 
pre-parsing step that add headers and rewrites the "attributes" column 
(column 9) so that each attribute value is tab-separated instead of 
semicolon-separated.

It is the pre-parser that complains if a line doesn't have at least 9 
columns. I checked the code and found a "TODO" comment about 
implementing support for comments, but for some reason it has been 
overlooked. It should be easy enough to implement this, and I can 
probably fix this for the next (BASE 3.3) release.

But for now, the importer will choke on anything that is not a "Field" 
line as described by http://www.ensembl.org/info/website/upload/gff.html

/Nicklas


On 2014-05-23 18:17, Bob MacCallum wrote:
> Hi all,
>
> [mods please delete the mail I sent from my gmail account, or add me
> with that account but with no delivery, thank]
>
> Just upgraded from 2->3 and now trying out the NGS feaures.  Quick
> question about GTF import for array designs...
>
> When doing array design->import the importer is complaining:
>
> "Error: A line must have at least 9 columns, at line: 1"
> stack trace below
> (using GTF reporter map importer (version 3.2.5-dev))
>
> The same error came up when we first created the array design (because I
> checked the "validate files" box.
>
> I know the first line has fewer columns because it has some # commented
> lines
>
> so in the importer configuration I configured the "Ignore" field like
> this:  ^#.*
>
> But still the same error...
>
> Even if I set Default error handling = skip it's the same.  Seems to be
> something deeper than the usual tab delimited importer parser settings.
>
> I guess we should make a new GTF file with no comments but I just wanted
> to check if this was the intended behaviour?
>
> cheers,
> Bob
>
>
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.
> Get unparalleled scalability from the best Selenium testing platform available
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
>
>
>
> _______________________________________________
> The BASE general discussion mailing list
> bas...@li...
> unsubscribe: send a mail with subject "unsubscribe" to
> bas...@li...
>

Re: [base] array design GTF import - not 9 columns

[Bob M. <r.m...@im...>]

Thanks for the quick reply Nicklas,
We'll strip out the comments for now and keep on testing.
cheers,
Bob


On Mon, May 26, 2014 at 7:37 AM, Nicklas Nordborg <
nic...@me...> wrote:

> Thanks for testing this. It feels like it was a long time ago this was
> implemented and we have not really started to use it yet. Anyway, you
> are correct that the problem is something deeper than the usual importer.
>
> To make GTF files parsable by the regular importers, there is a
> pre-parsing step that add headers and rewrites the "attributes" column
> (column 9) so that each attribute value is tab-separated instead of
> semicolon-separated.
>
> It is the pre-parser that complains if a line doesn't have at least 9
> columns. I checked the code and found a "TODO" comment about
> implementing support for comments, but for some reason it has been
> overlooked. It should be easy enough to implement this, and I can
> probably fix this for the next (BASE 3.3) release.
>
> But for now, the importer will choke on anything that is not a "Field"
> line as described by http://www.ensembl.org/info/website/upload/gff.html
>
> /Nicklas
>
>
> On 2014-05-23 18:17, Bob MacCallum wrote:
> > Hi all,
> >
> > [mods please delete the mail I sent from my gmail account, or add me
> > with that account but with no delivery, thank]
> >
> > Just upgraded from 2->3 and now trying out the NGS feaures.  Quick
> > question about GTF import for array designs...
> >
> > When doing array design->import the importer is complaining:
> >
> > "Error: A line must have at least 9 columns, at line: 1"
> > stack trace below
> > (using GTF reporter map importer (version 3.2.5-dev))
> >
> > The same error came up when we first created the array design (because I
> > checked the "validate files" box.
> >
> > I know the first line has fewer columns because it has some # commented
> > lines
> >
> > so in the importer configuration I configured the "Ignore" field like
> > this:  ^#.*
> >
> > But still the same error...
> >
> > Even if I set Default error handling = skip it's the same.  Seems to be
> > something deeper than the usual tab delimited importer parser settings.
> >
> > I guess we should make a new GTF file with no comments but I just wanted
> > to check if this was the intended behaviour?
> >
> > cheers,
> > Bob
> >
> >
> >
> ------------------------------------------------------------------------------
> > "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> > Instantly run your Selenium tests across 300+ browser/OS combos.
> > Get unparalleled scalability from the best Selenium testing platform
> available
> > Simple to use. Nothing to install. Get started now for free."
> > http://p.sf.net/sfu/SauceLabs
> >
> >
> >
> > _______________________________________________
> > The BASE general discussion mailing list
> > bas...@li...
> > unsubscribe: send a mail with subject "unsubscribe" to
> > bas...@li...
> >
>
>
>
> ------------------------------------------------------------------------------
> The best possible search technologies are now affordable for all companies.
> Download your FREE open source Enterprise Search Engine today!
> Our experts will assist you in its installation for $59/mo, no commitment.
> Test it for FREE on our Cloud platform anytime!
>
> http://pubads.g.doubleclick.net/gampad/clk?id=145328191&iu=/4140/ostg.clktrk
> _______________________________________________
> The BASE general discussion mailing list
> bas...@li...
> unsubscribe: send a mail with subject "unsubscribe" to
> bas...@li...
>