Re: [Babeldoc-devel] Question regarding FTP scanner

[Dejan K. <dej...@ya...>]

In current ftpScanner implementation with Commons Net
1.0 library there is only one problematic operation -
listing directory. This is because different servers
return different outputs that is not always parsed
correctly. Other operations like getting files,
changing directory, etc. works OK with all servers. 

Dejan


--- David Glick <dg...@co...> wrote:
> Actually, I threw together a Java-based shell and
> dropped it into my utility 
> jar.  Works like a charm.  FYI, it works just fine
> with the 1.0 commons-net 
> jar, so something else is problematic with the
> FtpScanner.  I'll (hopefully!) 
> have time to look at it middle of next week.
> 
> David
> 
> 
> On Sunday 19 October 2003 12:17 pm, Bruce McDonald
> wrote:
> > Moving to the "external application module" is
> good enough to get things
> > working assuming you have a decent shell (you use
> linux - so you are ok
> > there).
> >
> > On Saturday 18 October 2003 08:44 am, David Glick
> wrote:
> > > Hi Bruce,
> > >
> > > Sorry, I didn't explain myself well.  I'll
> certainly integrate the newest
> > > commons net in when I get a chance.  The
> question was whether to
> > > integrate it into the 'go-live' version Babeldoc
> we're using on
> > > Production (1.1.8-dev).. We've fully tested on
> it, and therefore we won't
> > > be upgrading to the 'real' release until after
> go-live.  The FTP piece is
> > > pretty isolated, so if I can drop it into
> 1.1.8-dev and everything starts
> > > working, that 's a level of integration I can
> handle.  If it doesn't
> > > work, I don't know if I'll have time to
> troubleshoot it, so I may end up
> > > calling the ftp transfer as an 'external to
> Babeldoc' process just to get
> > > over this hump.
> > >
> > > Regardless, I will integrate it into the current
> Babeldoc (after I get a
> > > reasonable amount of sleep... :-)).
> > >
> > >
> > > David
> > >
> > > On Friday 17 October 2003 10:08 pm, Bruce
> McDonald wrote:
> > > > Integrate.  This is what we need to get
> babeldoc stable.  We can build
> > > > the future in 1.3.  Dejan, can you check if
> your application still
> > > > works with the new code.  Anyone else check
> please.
> > > >
> > > > Ps.  I will be doing a release tonight !prior!
>  to the
> > > >
> > > > On Friday 17 October 2003 07:57 pm, David
> Glick wrote:
> > > > > Thanks, Dejan.  I downloaded and tested the
> latest snapshot directly
> > > > > using the ftp example, and it worked like a
> charm.  Since I'm now on
> > > > > a tight deadline, I need to decide whether
> it will be easier to
> > > > > integrate the newest version back into
> Babeldoc, or just run it
> > > > > externally for my needs...
> > > > >
> > > > > Life is ALWAYS interesting at crunch time!
> > > > >
> > > > > David
> > > > >
> > > > > On Friday 17 October 2003 3:15 pm, Dejan
> Krsmanovic wrote:
> > > > > > Hi David,
> > > > > > FtpScanner is using Jakarta Commons Net
> 1.0 package
> > > > > > (ex ORO Net Components 3.8.1) for ftp
> scannong. This
> > > > > > component was mad to work with Unix based
> servers and
> > > > > > you were supposed to implement support for
> other
> > > > > > servers. Since I needed to use it for MS
> FTP server I
> > > > > > have implement support for it.
> > > > > > AFAIK, Commons Net is improving and they
> have changed
> > > > > > the way how server listing is parsed, and
> also they
> > > > > > should support other systems (MS for
> example).
> > > > > > However, the code for that was not
> released yet but I
> > > > > > think it will be released very soon
> (actualy the
> > > > > > voting for Commons Net 1.1 relesase has
> already been
> > > > > > started on Jakarta's mailing list).
> > > > > >
> > > > > > So you have three options:
> > > > > > - Try to implement AS/400 support by
> yourself. You
> > > > > > need to parse your server's output to
> extract
> > > > > > filenames, etc  look at the code for
> > > > > > MsFtpFileListParser.
> > > > > > - wait for Commons Net 1.1
> > > > > > - use Commons Net CVS snapshot
> > > > > >
> > > > > > Note that  AS/400 might left unsupported
> by Commons
> > > > > > Net 1.1. Try to look at their CVS, there
> are parsers
> > > > > > for most systems...
> > > > > >
> > > > > > Dejan
> > > > > >
> > > > > > --- David Glick <dg...@co...> wrote:
> > > > > > > Gentlemen,
> > > > > > >
> > > > > > > I'm trying to set up the FtpScanner, but
> it is
> > > > > > > crashing.  The stack trace and
> > > > > > > config filre are shown below.  The stack
> trace show
> > > > > > > that MsFtpFileListParser
> > > > > > > was called,and the source code indicates
> that the
> > > > > > > FtpScanner has been
> > > > > > > optimized for Microsoft FTP.  Does the
> FtpScanner
> > > > > > > also work with
> > > > > > > non-Microsoft FTP servers?  The one I'm
> accessing is
> > > > > > > on an AS/400, but I've
> > > > > > > used manual FTP clients to it prevously
> without
> > > > > > > problems.
> > > > > > >
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > David
> > > > > > >
> > > > > > >
> > > > > > > === CONFIG.PROPERTIES ===
> > > > > > > #scan ftp site every 1 hour for files to
> place into
> > > > > > > the outbound pipeline
> > > > > > > ftpscan.type=ftp
> > > > > > > #ftpscan.period=3600000
> > > > > > > ftpscan.period=10000
> > > > > > > ftpscan.ftpHost=as400
> > > > > > > ftpscan.ftpUsername=AMUSER
> > > > > > > ftpscan.ftpPassword=XXXX  <== Sorry, I'd
> have to
> > > > > > > shoot you otherwise
> > > > > > > ftpscan.pipeline=test
> > > > > > > ftpscan.filter=EBPRDIMP.EBPRDRR
> > > > > > >
> > > > > > > === STACK TRACE ===
> > > > > > > <2003-10-17 12:44:02,550> ERROR
> [ftpscan] :
> > > > > > > [ScannerWorker.process] Error
> > > > > > > scanning...
> > > > > > > java.lang.NumberFormatException: For
> input string:
> > > > > > > "10/07/03"
> > > > > > >         at
> > > > > >
> > > > > >
>
java.lang.NumberFormatException.forInputString(NumberFormatExceptio
> > > > > >n. ja va
> > > > > >
> > > > > >:4 8)
> > > > > >:
> > > > > > >         at
> java.lang.Long.parseLong(Long.java:332)
> > > > > > >         at
> java.lang.Long.parseLong(Long.java:381)
> > > > > > >         at
> > > > > >
> > > > > >
>
com.babeldoc.scanner.worker.FtpScanner$1$MsFtpFileListParser.parseE
> > > > > >nt ry (U nk nown
> > > > > >
> > > > > > > Source)
> > > > > > >         at
> > > > > >
> > > > > >
>
com.babeldoc.scanner.worker.FtpScanner$1$MsFtpFileListParser.parseF
> > > > > >il eL is t( Unknown
> > > > > >
> > > > > > > Source)
> 
=== message truncated ===


__________________________________
Do you Yahoo!?
The New Yahoo! Shopping - with improved product search
http://shopping.yahoo.com