Menu
▾
▴
[Babeldoc-devel] Re: [Babeldoc-user] Questions. Please Help
|
From: David G. <dg...@co...> - 2003-09-25 21:17:52
|
Hi Qin,
Please see my responses below.
David
On Thursday 25 September 2003 12:18 pm, Qin Ding wrote:
> David: Thank you for the test template, with which I can say I start
> understanding how everything connects.
>
> Now, when I ran the test pipeline, I first got the error stating that the
> directory test/inventory is not accessable. I changed to c:
> \babeltest\test\inventory. The error goes away. So, I guess we need to
> give the full path on my NT system.
>
> And then, I move the file test-file.xml into the inventory fold. I got
> this error, which I don't know how to fix.
>
> <2003-09-25 13:55:44,501> INFO [main] : Initializing workers:
> <2003-09-25 13:55:44,561> INFO [main] : dirscan (directory) configured...
> <2003-09-25 13:55:44,561> INFO [main] : Starting workers...
> <2003-09-25 13:55:44,561> INFO [main] : dirscan started...
> <2003-09-25 13:56:05,081> INFO [Thread-1] : Processing document...
> <2003-09-25 13:56:05,241> ERROR [Thread-1] : [AsynchronousFeeder$1.run]
> com.babeldoc.core.pipeline.PipelineException: PipelineStage: entryStage not
> found
> at
> com.babeldoc.core.pipeline.PipelineStageFactory.getEntryStageName(Unknown
> Source)
> at com.babeldoc.core.pipeline.PipelineFactory.process(Unknown Source)
> at com.babeldoc.core.pipeline.PipelineFactoryFactory.process(Unknown
> Source)
> at
> com.babeldoc.core.pipeline.feeder.SynchronousFeeder.process(Unknown Source)
> at
> com.babeldoc.core.pipeline.feeder.AsynchronousFeeder.actuallyProcess(Unknow
>n Source)
> at com.babeldoc.core.pipeline.feeder.AsynchronousFeeder$1.run(Unknown
> Source)
> at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown
> Source)
> at java.lang.Thread.run(Thread.java:536)
>
> I checked the test-pipeline.properties. I found that entryStage=initialize
> .
>
DG>> The error is clearly saying that it can't find entryStage. I wonder if
it has anything to do with the Unix vs Windows end-of-line character? You
may want to try running the files through a Unix-to-Windows converter to see
if that makes a difference.
> I made the following changes in this file:
>
> email-doc.smtpHost=
> email-doc.smtpTo=
> email-doc.smtpFrom=
>
DG>> I assume you made these changes in order to disable the e-mail stage? I
believe that doing this will generate errors. The better way is to change
the 'insert-transaction-id.nextStage' attribute from email-doc to
get-attributes. This will bypass the email-doc stage.
> I don't know what's this? What's this "test.properties"?
> initialize.script=\
> document.put("properties_file",
> "/home/dglick/test/babeltest/test.properties");\
> document.put("document_id", "D0001WHYNOT");\
> document.put("merchant_id", "M0001THISTOO");
>
> Do you have a guideline for writing this test-pipeline.properties file? I
> see some line are ended with "\" but not the others. What should go in this
> file? I am sure I don't know how to start to write one to call my
> validation routine.
>
> Qin
>
The test.properties file is used simply an example of how you can store a
properties file name into an attribute for later use.
As Bruce mentioned, the '\' character is used to signify that the following
lines are part of the same statement. The reason for this is that the
document.script attribute in your example above only accepts a single line as
input. We use the '\' character to allow us to use multiple lines to improve
the readability of the script.
Other than the Babeldoc User's guide and the included examples, as well as the
documentation pipeline (which is a GREAT example, by the way), there really
isn't any other type of guide to creating a pipeline file. That's why I sent
you the test-pipeline.properties file; I thought it might be a good example
to follow. If it's lacking, feel free to shoot more questions and I'll try
to answer them.
David
> ---------------------------------------------------------------------------
>-------------
>
> This is a PRIVATE message. If you are not the intended recipient, please
> delete without copying and kindly advise us by e-mail of the mistake in
> delivery. NOTE: Regardless of content, this e-mail shall not operate to
> bind CSC to any order or other contract unless pursuant to explicit written
> agreement or government initiative expressly permitting the use of e-mail
> for such purpose.
> ---------------------------------------------------------------------------
>-------------
>
>
>
>
>
> David Glick
> <dglick To: "Qin Ding"
> <qd...@cs...> @cox.net> cc: Babeldoc Developer List
> <bab...@li...> 09/25/03 11:22 Subject:
> Re: [Babeldoc-user] Questions. Please Help AM
>
>
>
>
>
>
> Hi Qin,
>
> I've attached a zip file that contains a template that you can begin
> working
> with. Use it as follows:
>
> 1. Extract the contents. Be sure to preserve the directory structure.
>
> 2. cd into babeltest
>
> 3. Set BABELDOC_USER to point to the config directory. On Unix using ksh
> or
> bash, you can type: export "BABELDOC_USER=`pwd`/config". I'll leave it as
> an exercise for the student for any other platform... :-)
>
> 4. Set BABELDOC_HOME to point to your babeldoc installation.
>
> At this point, you're ready to go. This template allows you to either post
> a
> document directly into babeldoc, or else use a scanner to grab it out of a
> directory.
>
> To post directly into babeldoc:
> babeldoc process -p test-pipeline -f test-file.xml
>
> To use the scanner, first type the following:
> babeldoc scanner
>
> Then, in a separate window, copy the file test-file.xml into the monitored
> directory:
> cp test-file.xml test/inventory
>
> In 10 seconds or less, the file should be picked up and processed.
>
> A couple of points:
>
> 1. I run under Unix. If you use something else, the last stage of the
> pipeline (console-writer) will probably fail. You may want to change it to
> e-mail yourself, write to a real file, or whatever.
>
> 2. Please be sure to cc the Babeldoc list, since this discussion may be of
> interest to others, and also there are others on the list much more
> knowledgable than I who can also help.
>
>
> Good luck!
>
> David
>
> On Wednesday 24 September 2003 12:09 pm, Qin Ding wrote:
> > David: Thank you again.
> >
> > 1. Yes, PLEASE send me the template to set up the typical directory
> > structure
> > and configuration files in order to create a pipeline.
> >
> > 2. I guess I can call TestManager.validate(java.io.File file) method
>
> using
>
> > java.io.File file = new java.io.File(document.get("file_name"));\
> > TestManager.validate(file);
> >
> > But, where should I put these two line of code?
> >
> > 3. I guess I need to modify the sample code you sent. Where should it
>
> go
>
> > afterwards? What's the file name should it be that contains the following
> > lines?
> >
> > get-attributes.stageType=Scripting
> > get-attributes.nextStage=Email
> > get-attributes.language=java
> > get-attributes.script=\
> > java.io.File file = new java.io.File(document.get("file_name"));\
> > var report = TestManager.validate(file);
> > document.setBytes(report.toString().getBytes());\
> > document.setMimeType("text/xml");
> > !!!! How do you switch the report and original document here in order
>
> for
>
> > the next stage to send via email the report and then continue processing
> > the original document say convert to xml?
> >
> > 4. What's format of the this file? Any rules? I see some lines end
>
> with
>
> > \ but some not. I also see some are native java data type; some are just
> > var. Why use var instead of StringBuffer?
> >
> > Once again, as you can see, I am new to this tool. But with your help,
>
> I'm
>
> > sure I can learn quickly. Thank you again.
> >
> > Qin
>
> ---------------------------------------------------------------------------
>
> >-------------
> >
> > This is a PRIVATE message. If you are not the intended recipient, please
> > delete without copying and kindly advise us by e-mail of the mistake in
> > delivery. NOTE: Regardless of content, this e-mail shall not operate to
> > bind CSC to any order or other contract unless pursuant to explicit
>
> written
>
> > agreement or government initiative expressly permitting the use of e-mail
> > for such purpose.
>
> ---------------------------------------------------------------------------
>
> >-------------
> >
> >
> >
> >
> >
> > David Glick
> > <dglick To: "Qin Ding"
> > <qd...@cs...> @cox.net> cc: Babeldoc Developer List
> > <bab...@li...> 09/24/03 01:23 Subject:
> > Re: [Babeldoc-user] Questions. Please Help PM
> >
> >
> >
> >
> >
> >
> > Hi Qin,
> >
> > There aren't any examples in the documentation (yet!) about using the
> > scripting stage, but it's actually very easy to do. Here's an example
>
> that
>
> > I
> > posted yesterday on the list:
> >
> > ######
> > # List all attributes on this document
> > ######
> > get-attributes.stageType=Scripting
> > get-attributes.nextStage=console-writer
> > get-attributes.language=javascript
> > get-attributes.script=\
> > var keyset = document.getAttributes().keySet();\
> > var keyiter = keyset.iterator();\
> > var bufr = new java.lang.StringBuffer();\
> > bufr.append("<attributes>\\n");\
> > while(keyiter.hasNext()) {\
> > var key = keyiter.next();\
> > var cls = document.get(key);\
> > bufr.append(key + "=" + cls + "\\n");\
> > }\
> > bufr.append("</attributes>\\n");\
> > document.setBytes(bufr.toString().getBytes());\
> > document.setMimeType("text/xml");
> >
> > The "get-attributes.script'" attribute defines a script that is embedded
>
> in
>
> > the properties file. An alternative would be to use
> > "get-attributes.scriptFile" in order to point to an external file
> > containing
> > the script program.
> >
> > Beyond the basic Babeldoc configuration no other additional configuration
> > is
> > required to use script files. However, if you reference external
>
> classes,
>
> > those classes will need to be included in the classpath.
> >
> > The document is stored in a PipelineDocument object. You can get at the
> > contents via the toString() method. In order to use your validation
> > routines, you have a couple of options:
> >
> > 1. Feed the document into the pipeline and convert the document String
> > (document.toString()) into a Java stream that can be fed to your
>
> validation
>
> > routine via the TestManager.validate(String docName, java.io.InputStream)
> > method call. The docName can e retrieved from the document attribute
> > "file_name": document.get("file_name");
> >
> > 2. Use the TestManager.validate(java.io.File file) method call via:
> > java.io.File file = new java.io.File(document.get("file_name"));\
> > TestManager.validate(file);
> >
> > I'm not sure if you understand how to set up the typical directory
> > structure
> > and configuration files in order to create a pipeline. If not, let me
>
> know
>
> > and I'll throw together a template for you to use.
> >
> >
> > Hope this helps,
> >
> > David
> >
> > On Wednesday 24 September 2003 8:26 am, you wrote:
> > > David: Thank you for your advise. I'll then try the scripting stage.
>
> I
>
> > > tried to find some document on this particular stage. I did not see
> > > anything. Do you have any? How do I set this up? I assume there must
>
> be
>
> > a
> >
> > > someConfig.properties file to define this stage. Where to put this
>
> file
>
> > > and Would you please walk me through this? After the scanner picks up
> >
> > the
> >
> > > file and delivers to the scriptStage, what kind of document it actually
> > > delivers? It is a java.io.File or a java.io.InputStream? Anyway, my
> > > validation routine has the following signatures:
> > >
> > > TestManager.validate(java.io.File file);
> > > TestManager.validate(String docName, java.io.InputStream);
> > >
> > > Regarding e-mailing the report, your mentioned the following:
> > > a) Call your validation routines via a Scripting stage.
> > > b) Place the original document into an attribute
> > > c) Replace the current document with the report to be e-mailed.
> > > d) Use the SmtpWriter stage to e-mail your report
> > > e) Move the original document from the attribute in which it is stored
> >
> > back
> >
> > > into the current document
> > > f) Continue processing
> > >
> > > Are all these a to f steps can be achieved by setting up in the
> > > config.properties? Or we still have to extend the ScriptingStage to
> > > achieve that goal? How should I start and where do I deploy it?
> > >
> > > Qin
>
> ---------------------------------------------------------------------------
>
> > >-------------
> > >
> > > This is a PRIVATE message. If you are not the intended recipient,
>
> please
>
> > > delete without copying and kindly advise us by e-mail of the mistake in
> > > delivery. NOTE: Regardless of content, this e-mail shall not operate to
> > > bind CSC to any order or other contract unless pursuant to explicit
> >
> > written
> >
> > > agreement or government initiative expressly permitting the use of
>
> e-mail
>
> > > for such purpose.
>
> ---------------------------------------------------------------------------
>
> > >-------------
> > >
> > >
> > >
> > >
> > >
> > > David Glick
> > > <dglick To: Qin
> > > Ding/ASD/CSC@CSC, @cox.net>
> > > bab...@li... cc:
> > > 09/23/03 02:30 Subject: Re:
> >
> > [Babeldoc-user]
> >
> > > Questions. Please Help PM
> > >
> > >
> > >
> > >
> > >
> > >
> > > Hi Qin,
> > >
> > > 1) You should set the environment variable BABELDOC_USER to point at
>
> your
>
> > > project directory.
> > >
> > > 2) While you can certainly subclass the PipelineStage to perform your
> > > validation, an easier approach may be to use the Scripting stage and
>
> just
>
> > > call your validation routines inline. Creating a new stage type
>
> requires
>
> > a
> >
> > > bit more familiarity with the Babeldoc environment.
> > >
> > > 3) Regarding e-mailing your report, you might do something like the
> > > following:
> > >
> > > a) Call your validation routines via a Scripting stage.
> > > b) Place the original document into an attribute
> > > c) Replace the current document with the report to be e-mailed.
> > > d) Use the SmtpWriter stage to e-mail your report
> > > e) Move the original document from the attribute in which it is stored
> >
> > back
> >
> > > into the current document
> > > f) Continue processing
> > >
> > > It sounds like you've got a pretty good handle on everything else.
> > >
> > >
> > > Hope this helps,
> > >
> > > David
> > >
> > > On Tuesday 23 September 2003 11:57 am, Qin Ding wrote:
> > > > I am trying to use this tool. From what I read, this is an excellent
> > >
> > > tool
> > >
> > > > for me. I read the user guide and white paper and usage documents and
> > > > developer guide twice and I am still not sure of how to put this
> >
> > project
> >
> > > > together.
> > > >
> > > > My babeldoc home (BABELDOC_HOME env variable)
> > > > c:\babeldoc
> > > > It has lib and bin directory which has all the scripts and jar files
> >
> > from
> >
> > > > your build/lib and build/bin. In c:\babeldoc\lib, it also has my own
> >
> > jar
> >
> > > > (longlegs-validation.jar) and apache common-validator.jar
> > > >
> > > > My project home
> > > > c:\myproject\longlegs\
> > > > In your documents, at one place, it states that I need to set up
> >
> > USER_DIR
> >
> > > > env variable pointing to this project directory; and at another place
> >
> > in
> >
> > > > the document, it asks to include this directory in the classpath.
>
> Which
>
> > > way
> > >
> > > > should I go?
> > > >
> > > > Under the longlegs folder, I have
> > > > config folder
> > > > config/scanner sub-folder. In here, I have config.properties for the
> > > > scanner like this:
> > > >
> > > > daddy-longlegs.type=directory
> > > > daddy-longlegs.period=10000
> > > > daddy-longlegs.inDirectory=/daddy/longlegs/in
> > > > daddy-longlegs.doneDirectory=/daddy/longlegs/done
> > > > daddy-longlegs.pipeline=longlegs
> > > >
> > > > pipeline folder. In here, I have config.properties like this:
> > > >
> > > > longlegs.type=simple
> > > > longlegs.configFile=pipeline/simple/longlegs
> > > >
> > > > pipeline/simple folder. In here, I have longlegs.properties, and
> > > > flat2xml-convert.xml like this:
> > > >
> > > > entryStage=validate
> > > > validate.stageType=DataValidation
> > > > validate.nextStage=convert
> > > >
> > > > convert.stageType=FlatToXml
> > > > convert.nextStage=transform
> > > > convert.flatToXmlFile=/longlegs/pipeline/simple/flat2xml-convert.xml
> > > >
> > > > transform.stageType=XslTransform
> > > > transform.nextStage=null
> > > >
> > > > flat2xml-convert.xml is omitted here. It is the copy of yours.
> > > >
> > > > journal
> > > > journal/simple
> > > >
> > > > Also
> > > > I have created two directories
> > > > C:\daddy\longlegs\in
> > > > C:\daddy\longlegs\done
> > > >
> > > > I want to drop a flat file (delimited by comma, or tab, or vertical
> >
> > bar)
> >
> > > in
> > >
> > > > c:\daddy\longlegs\in folder. The scanner should pass the file to my
> > > > pipeline stage "DataValidation". I need to write this pipeline
>
> stage.
>
> > > As
> > >
> > > > I understand, all I need to do is to subclass the PipelineStage. In
> >
> > that
> >
> > > > class, I should implement process method to call my validation
>
> logics,
>
> > > > which I have jar-ed and deployed in c:\babeldoc\lib directory along
> >
> > with
> >
> > > > apache common-validator.
> > > >
> > > > Now, where should I deploy my pipelineStage DataValidation class? In
>
> my
>
> > > > longlegs/pipeline directory or some new directory I should add to the
> > > > longlegs/pipeline directory?
> > > >
> > > > A validation report (html) is dynamically generated during the
> >
> > validation
> >
> > > > stage. This report, Not original flat file, needs to be mailed out.
>
> How
>
> > > do
> > >
> > > > I add the report from my DataValidate stage to the mail
>
> pipelineStage?
>
> > > > Also, I believe I can use the FileWriter pipelineStage to write the
> >
> > html
> >
> > > > report out. So, the key is to introduce the report to the mail and
> > > > filewriter stage, not the original flat file. How do I do that?
> > > >
> > > > If I can successfully put all these together quickly, I'd like to
> >
> > present
> >
> > > > this tool to my manager for considering to use in our next project.
> >
> > Your
> >
> > > > help is greatly appreciated. Thanks.
> > > >
> > > > Qin
>
> ---------------------------------------------------------------------------
>
> > > >-------------
> > > >
> > > > This is a PRIVATE message. If you are not the intended recipient,
> >
> > please
> >
> > > > delete without copying and kindly advise us by e-mail of the mistake
>
> in
>
> > > > delivery. NOTE: Regardless of content, this e-mail shall not operate
>
> to
>
> > > > bind CSC to any order or other contract unless pursuant to explicit
> > >
> > > written
> > >
> > > > agreement or government initiative expressly permitting the use of
> >
> > e-mail
> >
> > > > for such purpose.
>
> ---------------------------------------------------------------------------
>
> > > >-------------
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > -------------------------------------------------------
> > > > This sf.net email is sponsored by:ThinkGeek
> > > > Welcome to geek heaven.
> > > > http://thinkgeek.com/sf
> > > > _______________________________________________
> > > > Babeldoc-user mailing list
> > > > Bab...@li...
> > > > https://lists.sourceforge.net/lists/listinfo/babeldoc-user
--
David Glick
Transmit Consulting, Inc
619-475-4052
dg...@tr...
|
