Hi,
I'm running babeldoc-1.1.9-dev and I have a problem with the FTP scanner. The problem is if I have one document in the FTP directory, the scanner does not pick it up. But immediately I place a second document in that same folder, the two documents are processed. I had the same problem with some of the previous versions.
Regards
Heti
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
First I used my PC (Windows 2000 Professional) as a FTP server, Then I connected to one of our client's FTP server which is a Linux server. I also connected to another Linux FTP server. In all the cases, when there is only one document, it does not work.
I also have the following environment variables:
BABELDOC_CMD_LINE_ARGS : scanner
BABELDOC_USER : C:\Babeldoc_src\babeldoc-bin-1.1.9-dev
BABELDOC_HOME : .;C:\Babeldoc_src\babeldoc-bin-1.1.9-dev\lib
I'm running the scanner from a batch file with the following command : babeldoc scanner -s ScannerConfig\scanner\config.properties
I hope I've given you enough information.
Regards
Heti
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
When the scanner connects to the ftp server to scan the remote directory, I've the following exception:
<2004-11-07 12:17:55,644> INFO [Thread-0] : Processing document...
<2004-11-07 12:17:55,664> INFO [Thread-0] : PipelineStage name: decompress
<2004-11-07 12:17:55,674> INFO [Thread-0] : PipelineStage name: addAttrib
<2004-11-07 12:17:55,684> INFO [Thread-0] : PipelineStage name: fWriter0
<2004-11-07 12:17:55,684> INFO [Thread-0] : PipelineStage name: router
<2004-11-07 12:19:38,662> ERROR [scanner] : [ScannerWorker.process] Error scanning...
com.babeldoc.scanner.ScannerException: Error connecting to ftp
at com.babeldoc.scanner.worker.FtpScanner.doScan(Unknown Source)
at com.babeldoc.scanner.ScannerWorker.process(Unknown Source)
at com.babeldoc.scanner.ScannerThread.run(Unknown Source)
Caused by: com.babeldoc.scanner.ScannerException: Error connecting to ftp
at com.babeldoc.scanner.worker.FtpScanner.processFtpFolder(Unknown Source)
... 3 more
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:168)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:254)
at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read(BufferedInputStream.java:235)
at org.apache.commons.net.telnet.TelnetInputStream.__read(TelnetInputStream.java:140)
at org.apache.commons.net.telnet.TelnetInputStream.run(TelnetInputStream.java:464)
at java.lang.Thread.run(Thread.java:595)
<2004-11-07 12:24:38,664> ERROR [scanner] : [ScannerWorker.process] Error scanning...
com.babeldoc.scanner.ScannerException: Error connecting to ftp
at com.babeldoc.scanner.worker.FtpScanner.doScan(Unknown Source)
at com.babeldoc.scanner.ScannerWorker.process(Unknown Source)
at com.babeldoc.scanner.ScannerThread.run(Unknown Source)
Caused by: com.babeldoc.scanner.ScannerException: Error connecting to ftp
at com.babeldoc.scanner.worker.FtpScanner.processFtpFolder(Unknown Source)
... 3 more
Caused by: java.net.SocketException: Connection reset by peer: socket write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at org.apache.commons.net.telnet.TelnetClient._flushOutputStream(TelnetClient.java:115)
at org.apache.commons.net.telnet.TelnetOutputStream.flush(TelnetOutputStream.java:176)
at java.io.FilterOutputStream.flush(FilterOutputStream.java:123)
at sun.nio.cs.StreamEncoder$CharsetSE.implFlush(StreamEncoder.java:410)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:152)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:213)
at java.io.BufferedWriter.flush(BufferedWriter.java:236)
at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:445)
at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:478)
at org.apache.commons.net.ftp.FTP.cwd(FTP.java:704)
at org.apache.commons.net.ftp.FTPClient.changeWorkingDirectory(FTPClient.java:668)
... 4 more
The pipeline works correctly when the scanner gets the files from server, but the above message appears at random.
Same results with different servers (I've a Windows Xp sistem and also a Linux system).
Massimo
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Is the cronSchedule value 1 * * * * the entry for starting the schedule of scanner every hour at minute 1 ? I'm tryng with this configuration but the scanner begins to scan ignorin the schedule setting.
Thanks in advance.
Massimo
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Try using latest CVS code. Latest version have a bug with cron scheduling which is fixed in CVS but not yet released.
BTW, you should not use period and cronSchedule options at the same time.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
if the ftp server crashes, is the FtpScanner capable to reconnect to the server and restablish connection?
I'm trying with my ftp server under Linux (I rebot the server when from a client I run the FtpScanner).
I think if the FtpScanner throws an exception, it can't reconnect...
Here my logs..
.................
<2004-11-22 00:22:05,106> INFO [Thread-12] : PipelineStage name: dbwriter
<2004-11-22 00:22:08,301> INFO [Thread-6] : PipelineStage name: fWriter3
<2004-11-22 00:22:08,301> INFO [Thread-6] : PipelineStage name: dbwriter
<2004-11-22 00:22:09,313> INFO [Thread-11] : PipelineStage name: fWriter3
<2004-11-22 00:22:09,323> INFO [Thread-11] : PipelineStage name: dbwriter
<2004-11-22 00:22:09,323> INFO [Thread-4] : PipelineStage name: fWriter3
<2004-11-22 00:22:09,453> INFO [Thread-4] : PipelineStage name: dbwriter
<2004-11-22 00:28:27,356> ERROR [scanner] : [ScannerWorker.process] Error scanning...
com.babeldoc.scanner.ScannerException: Error connecting to ftp
at com.babeldoc.scanner.worker.FtpScanner.doScan(FtpScanner.java:171)
at com.babeldoc.scanner.ScannerWorker.process(Unknown Source)
at com.babeldoc.scanner.ScannerThread.run(Unknown Source)
Caused by: com.babeldoc.scanner.ScannerException: Error connecting to ftp
at com.babeldoc.scanner.worker.FtpScanner.processFtpFolder(FtpScanner.java:521)
at com.babeldoc.scanner.worker.FtpScanner.doScan(FtpScanner.java:169)
... 2 more
Caused by: java.lang.NullPointerException
at org.apache.commons.net.telnet.TelnetClient._flushOutputStream(TelnetClient.java:115)
at org.apache.commons.net.telnet.TelnetOutputStream.flush(TelnetOutputStream.java:176)
at java.io.FilterOutputStream.flush(FilterOutputStream.java:123)
at java.io.FilterOutputStream.close(FilterOutputStream.java:140)
at org.apache.commons.net.telnet.TelnetClient.disconnect(TelnetClient.java:161)
at org.apache.commons.net.ftp.FTP.disconnect(FTP.java:401)
at org.apache.commons.net.ftp.FTPClient.disconnect(FTPClient.java:533)
at com.babeldoc.scanner.worker.FtpScanner.connectToHost(FtpScanner.java:267)
at com.babeldoc.scanner.worker.FtpScanner.processFtpFolder(FtpScanner.java:468)
... 3 more
<2004-11-22 00:28:57,359> ERROR [scanner] : [ScannerWorker.process] Error scanning...
com.babeldoc.scanner.ScannerException: Error connecting to ftp
at com.babeldoc.scanner.worker.FtpScanner.doScan(FtpScanner.java:171)
at com.babeldoc.scanner.ScannerWorker.process(Unknown Source)
at com.babeldoc.scanner.ScannerThread.run(Unknown Source)
Caused by: com.babeldoc.scanner.ScannerException: Error connecting to ftp
at com.babeldoc.scanner.worker.FtpScanner.processFtpFolder(FtpScanner.java:521)
at com.babeldoc.scanner.worker.FtpScanner.doScan(FtpScanner.java:169)
... 2 more
Caused by: java.lang.NullPointerException
at org.apache.commons.net.telnet.TelnetClient._flushOutputStream(TelnetClient.java:115)
at org.apache.commons.net.telnet.TelnetOutputStream.flush(TelnetOutputStream.java:176)
at java.io.FilterOutputStream.flush(FilterOutputStream.java:123)
at java.io.FilterOutputStream.close(FilterOutputStream.java:140)
at org.apache.commons.net.telnet.TelnetClient.disconnect(TelnetClient.java:161)
at org.apache.commons.net.ftp.FTP.disconnect(FTP.java:401)
at org.apache.commons.net.ftp.FTPClient.disconnect(FTPClient.java:533)
at com.babeldoc.scanner.worker.FtpScanner.processFtpFolder(FtpScanner.java:467)
... 3 more
<2004-11-22 00:29:27,362> ERROR [scanner] : [ScannerWorker.process] Error scanning...
com.babeldoc.scanner.ScannerException: Error connecting to ftp
..............
Massimo
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi,
I'm running babeldoc-1.1.9-dev and I have a problem with the FTP scanner. The problem is if I have one document in the FTP directory, the scanner does not pick it up. But immediately I place a second document in that same folder, the two documents are processed. I had the same problem with some of the previous versions.
Regards
Heti
I have never heard about such problem. What is in your config file?
Hi,
here is my scanner config file:
ftp_documents.type=ftp
ftp_documents.period=60000
ftp_documents.ftpHost=12.11.32.200
ftp_documents.ftpUsername=user1
ftp_documents.ftpPassword=
ftp_documents.ftpFolder=in
ftp_documents.includeSubfolders=false
ftp_documents.localBackupFolder=c:/ftpBackup
ftp_documents.pipeline=get-docs-xml
ftp_documents.binary=true
This configuration looks OK to me. What OS is used for FTP server?
First I used my PC (Windows 2000 Professional) as a FTP server, Then I connected to one of our client's FTP server which is a Linux server. I also connected to another Linux FTP server. In all the cases, when there is only one document, it does not work.
I also have the following environment variables:
BABELDOC_CMD_LINE_ARGS : scanner
BABELDOC_USER : C:\Babeldoc_src\babeldoc-bin-1.1.9-dev
BABELDOC_HOME : .;C:\Babeldoc_src\babeldoc-bin-1.1.9-dev\lib
I'm running the scanner from a batch file with the following command : babeldoc scanner -s ScannerConfig\scanner\config.properties
I hope I've given you enough information.
Regards
Heti
Hi all,
I've a similar problem with ftp scanner.
When the scanner connects to the ftp server to scan the remote directory, I've the following exception:
<2004-11-07 12:17:55,644> INFO [Thread-0] : Processing document...
<2004-11-07 12:17:55,664> INFO [Thread-0] : PipelineStage name: decompress
<2004-11-07 12:17:55,674> INFO [Thread-0] : PipelineStage name: addAttrib
<2004-11-07 12:17:55,684> INFO [Thread-0] : PipelineStage name: fWriter0
<2004-11-07 12:17:55,684> INFO [Thread-0] : PipelineStage name: router
<2004-11-07 12:19:38,662> ERROR [scanner] : [ScannerWorker.process] Error scanning...
com.babeldoc.scanner.ScannerException: Error connecting to ftp
at com.babeldoc.scanner.worker.FtpScanner.doScan(Unknown Source)
at com.babeldoc.scanner.ScannerWorker.process(Unknown Source)
at com.babeldoc.scanner.ScannerThread.run(Unknown Source)
Caused by: com.babeldoc.scanner.ScannerException: Error connecting to ftp
at com.babeldoc.scanner.worker.FtpScanner.processFtpFolder(Unknown Source)
... 3 more
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:168)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:254)
at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read(BufferedInputStream.java:235)
at org.apache.commons.net.telnet.TelnetInputStream.__read(TelnetInputStream.java:140)
at org.apache.commons.net.telnet.TelnetInputStream.run(TelnetInputStream.java:464)
at java.lang.Thread.run(Thread.java:595)
<2004-11-07 12:24:38,664> ERROR [scanner] : [ScannerWorker.process] Error scanning...
com.babeldoc.scanner.ScannerException: Error connecting to ftp
at com.babeldoc.scanner.worker.FtpScanner.doScan(Unknown Source)
at com.babeldoc.scanner.ScannerWorker.process(Unknown Source)
at com.babeldoc.scanner.ScannerThread.run(Unknown Source)
Caused by: com.babeldoc.scanner.ScannerException: Error connecting to ftp
at com.babeldoc.scanner.worker.FtpScanner.processFtpFolder(Unknown Source)
... 3 more
Caused by: java.net.SocketException: Connection reset by peer: socket write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at org.apache.commons.net.telnet.TelnetClient._flushOutputStream(TelnetClient.java:115)
at org.apache.commons.net.telnet.TelnetOutputStream.flush(TelnetOutputStream.java:176)
at java.io.FilterOutputStream.flush(FilterOutputStream.java:123)
at sun.nio.cs.StreamEncoder$CharsetSE.implFlush(StreamEncoder.java:410)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:152)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:213)
at java.io.BufferedWriter.flush(BufferedWriter.java:236)
at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:445)
at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:478)
at org.apache.commons.net.ftp.FTP.cwd(FTP.java:704)
at org.apache.commons.net.ftp.FTPClient.changeWorkingDirectory(FTPClient.java:668)
... 4 more
The pipeline works correctly when the scanner gets the files from server, but the above message appears at random.
Same results with different servers (I've a Windows Xp sistem and also a Linux system).
Massimo
Looks like a network issue. I bet you will have the same problem with plain FTP.
Sherman
Hi,
Can anyone helps me with FtpScanner configuration?
I've a configuration config.properties like this....
scanner.type = ftp
scanner.period = 30000
scanner.cronSchedule = 1 * * * *
scanner.ftpHost = myHost
scanner.ftpUsername = myUser
scanner.ftpPassword = myPwd
scanner.ftpFolder = /home/myUser
scanner.filter = .*.zip
scanner.localBackupFolder = D:/DRG/Zipped
scanner.pipeline = Dbloader
Is the cronSchedule value 1 * * * * the entry for starting the schedule of scanner every hour at minute 1 ? I'm tryng with this configuration but the scanner begins to scan ignorin the schedule setting.
Thanks in advance.
Massimo
Try using latest CVS code. Latest version have a bug with cron scheduling which is fixed in CVS but not yet released.
BTW, you should not use period and cronSchedule options at the same time.
Hi,
I've problems with FtpScanner....
if the ftp server crashes, is the FtpScanner capable to reconnect to the server and restablish connection?
I'm trying with my ftp server under Linux (I rebot the server when from a client I run the FtpScanner).
I think if the FtpScanner throws an exception, it can't reconnect...
Here my logs..
.................
<2004-11-22 00:22:05,106> INFO [Thread-12] : PipelineStage name: dbwriter
<2004-11-22 00:22:08,301> INFO [Thread-6] : PipelineStage name: fWriter3
<2004-11-22 00:22:08,301> INFO [Thread-6] : PipelineStage name: dbwriter
<2004-11-22 00:22:09,313> INFO [Thread-11] : PipelineStage name: fWriter3
<2004-11-22 00:22:09,323> INFO [Thread-11] : PipelineStage name: dbwriter
<2004-11-22 00:22:09,323> INFO [Thread-4] : PipelineStage name: fWriter3
<2004-11-22 00:22:09,453> INFO [Thread-4] : PipelineStage name: dbwriter
<2004-11-22 00:28:27,356> ERROR [scanner] : [ScannerWorker.process] Error scanning...
com.babeldoc.scanner.ScannerException: Error connecting to ftp
at com.babeldoc.scanner.worker.FtpScanner.doScan(FtpScanner.java:171)
at com.babeldoc.scanner.ScannerWorker.process(Unknown Source)
at com.babeldoc.scanner.ScannerThread.run(Unknown Source)
Caused by: com.babeldoc.scanner.ScannerException: Error connecting to ftp
at com.babeldoc.scanner.worker.FtpScanner.processFtpFolder(FtpScanner.java:521)
at com.babeldoc.scanner.worker.FtpScanner.doScan(FtpScanner.java:169)
... 2 more
Caused by: java.lang.NullPointerException
at org.apache.commons.net.telnet.TelnetClient._flushOutputStream(TelnetClient.java:115)
at org.apache.commons.net.telnet.TelnetOutputStream.flush(TelnetOutputStream.java:176)
at java.io.FilterOutputStream.flush(FilterOutputStream.java:123)
at java.io.FilterOutputStream.close(FilterOutputStream.java:140)
at org.apache.commons.net.telnet.TelnetClient.disconnect(TelnetClient.java:161)
at org.apache.commons.net.ftp.FTP.disconnect(FTP.java:401)
at org.apache.commons.net.ftp.FTPClient.disconnect(FTPClient.java:533)
at com.babeldoc.scanner.worker.FtpScanner.connectToHost(FtpScanner.java:267)
at com.babeldoc.scanner.worker.FtpScanner.processFtpFolder(FtpScanner.java:468)
... 3 more
<2004-11-22 00:28:57,359> ERROR [scanner] : [ScannerWorker.process] Error scanning...
com.babeldoc.scanner.ScannerException: Error connecting to ftp
at com.babeldoc.scanner.worker.FtpScanner.doScan(FtpScanner.java:171)
at com.babeldoc.scanner.ScannerWorker.process(Unknown Source)
at com.babeldoc.scanner.ScannerThread.run(Unknown Source)
Caused by: com.babeldoc.scanner.ScannerException: Error connecting to ftp
at com.babeldoc.scanner.worker.FtpScanner.processFtpFolder(FtpScanner.java:521)
at com.babeldoc.scanner.worker.FtpScanner.doScan(FtpScanner.java:169)
... 2 more
Caused by: java.lang.NullPointerException
at org.apache.commons.net.telnet.TelnetClient._flushOutputStream(TelnetClient.java:115)
at org.apache.commons.net.telnet.TelnetOutputStream.flush(TelnetOutputStream.java:176)
at java.io.FilterOutputStream.flush(FilterOutputStream.java:123)
at java.io.FilterOutputStream.close(FilterOutputStream.java:140)
at org.apache.commons.net.telnet.TelnetClient.disconnect(TelnetClient.java:161)
at org.apache.commons.net.ftp.FTP.disconnect(FTP.java:401)
at org.apache.commons.net.ftp.FTPClient.disconnect(FTPClient.java:533)
at com.babeldoc.scanner.worker.FtpScanner.processFtpFolder(FtpScanner.java:467)
... 3 more
<2004-11-22 00:29:27,362> ERROR [scanner] : [ScannerWorker.process] Error scanning...
com.babeldoc.scanner.ScannerException: Error connecting to ftp
..............
Massimo