[Mgb-users] RE: Question about MyGenBank

[Robert C. <rw...@al...>]

Hello Yucheng,

> From: Feng, Yucheng
> Subject: Question about MyGenBank
>
>   I downloaded mygenbank from http://sourceforge.net/projects/mgb/
>   Here is the problem:
>
>   When I run
>   > mygb_admin ftp (or -t ftp), I got the following error message:
>
> gunzip: stdin: invalid compressed data--format violated
> Caught a SIGPIPE at /Library/Perl/Net/FTP.pm line 503
> $ = Net::FTP::get('Net::FTP=GLOB(0x24a560)', 'gbbct1.seq.gz',
> 'GLOB(0xc780c)') called
> from file `.//mygb_admin' line 194
> $ = main::ftpNCBI('gbbct1') called from file `.//mygb_admin' line 180
> $ = main::ftp called from file `.//mygb_admin' line 94
> Abort

After a bit of sleuthing, I discovered one needs to add $ftp->binary() 
after every ftp login:

         $ftp->login("anonymous", "$ENV{USER}\@$ENV{HOST}");
         $ftp->binary();

My guess is that requiring to specify binary is a new "feature" of 
net::ftp.  See below for details on how I determined this.  I'll try to 
put a new MyGenBank package together during the PAG conference.

> Although the first file is downloaded and correctly gunzipped into
> $DATA/GenBank/, it stopped right there, and the 2nd file is not
> processed.

I would not trust any of the data you have downloaded thus far using 
MyGenBank.  I would scrap what you have, make the changes to the 
mygb_admin script, and then start again.

Regards,
- Robert

-----

Details:

I suspected that there was a either a bug in the libnet library or some 
setting was not being set.  To test this, I downloaded the 
gbbct1.seq.gz file using both ncftp and this minimal perl script:

#!/usr/bin/perl -w
use strict;
use Net::FTP;
my $file='gbbct1' ;
my $ftp = new Net::FTP("ftp.ncbi.nih.gov") ;
if (not defined $ftp) {warn "FTP error\n" ; die $! ; }
$ftp->login("anonymous", "guest\@localhost") or die $!;
$ftp->cwd("genbank");
open(GZIP, "> $file.seq.gz") or die $! ;
$ftp->get("$file.seq.gz",  \*GZIP) or die $! ;
close GZIP;

I then compared the two downloaded files:

-rw-r--r--  1 robertci  staff  68560715 Jan  8 03:20 gbbct1.seq.gz    # 
downloaded with perl script
-rw-r--r--  1 robertci  staff  68561549 Jan  8 02:26 gz/gbbct1.seq.gz # 
downloaded with ncftp

I then tried uncompressing each:

$ zcat gz/gbbct1.seq.gz >/dev/null     # file using ncftp had no errors
$ zcat gbbct1.seq.gz >/dev/null        # file using net::ftp had errors
zcat: gbbct1.seq.gz: invalid compressed data--format violated

It seems as though downloading using Net::FTP somehow corrupts the 
file.  I read through 'perldoc net::ftp' and noticed that one of the 
modules was "binary"  So, I added $ftp->binary(); to the ftp.pl script:

$ftp->cwd("genbank");
$ftp->binary();
open(GZIP, "> $file.seq.gz") or die $! ;

When I reran the script, everything worked just fine.  I then added it 
to the mygb_admin script and now the 'mygb_admin -t ftp' works just 
fine.