[Flatworm-users] Hey, loyal flatworm fans!

[James T. <tu...@bl...>]

I just did a major check-in of code from James Lawrence over at
Southwest Airlines (I'm a frequent SWA flyer, so nice to know they're
supporting OSS development.)  I haven't had time to look it over in
detail yet, and it'll need doc and unit tests before I can release it as
1.1, but I wanted to at least get it in the source tree.  Here's James'
notes on his additions...
=======================================
Well, I have spent the last month making most of the mods I spoke with
you earlier about.  Unfortunately, I have not had time to participate in
the sourceforge process.  My schedules are tight and when I am done
delivering this to my Customers, I have to move on.

I want to thank you personally for writing Flatworm.  It has been a
learning experience getting under the hood and enhancing (without
hopefully breaking) all that has come before.  It was very nice not to
have to start from scratch.   I hope my contributions will be worthy for
inclusion in future versions as it would be nice to consider Flatworm a
3rd party tool and benefit from future enhancements by others.

With that said, I am sending you this jar file with everything I have
done so far. (basically a tarball) It is not the final production
version, but the API is pretty stable.  All of these changes have been
driven  by Customer requirements.

What I have done.. (at least made a feeble attempt at)

1) Added support for delimited file
2) Added support for writing flatfiles
 - Created examples to demonstrate new functionality
3) Encapsulated much of the main() functionality in your previous
examples into new classes called FileParser, FileCreator
 - This allows the client to use Flatworm with less knowledge of the
internal workings.
4) Delegated type conversion and string manipulation to
ConversionHelper.  Since FileFormat was so involved in parsing and I
needed conversion for
5) Pretty heavily refactored the way type conversions and string
manipulations work.
 - string manipulation routines have moved into the Util class for
re-use by the writing portion of the code
 - since pretty much all converters were calling removePadding(), this
was refactored out.  This method has been replaced by
ConversionHelper.transformString()
   - this will allow others to create converters and not have to worry
about calling string conversion, this is done automatically by
ConversionHelper
6) Renamed the examples, xmlfiles, input file to be self-explaining
7) Moved 'beans' (or what we call domain objects) to a separate folder
  - examples folder was getting a little crowded
  - might consider moving input files and xml files to their own home
too
8) Writing a crude Quickstart guide to explain (on a very high level)
what each example does and which arguments each needs.
9) Changed the way parsing is handled.  Parsing now works similar to
the way the SAX parser operates.  It's event driven.
  - you register a callback to handle each bean type
  - you register an exception handler to deal with lines in the file
that don't parse
    - you need this if you want to do something special with the
offending line, such as write it to a file for human intervention.
(manual cleanup of the record)
10) Added two new exception types to keep client exception handling
simple.
 - FlatwormCreatorException - Used by FileCreator to wrap a myriad of
Exceptions
  - Client is still responsible for handling
 - FlatwormParserException  - Used by FileCreator to wrap a myriad of
Exceptions
11) Probably other stuff I forgot, this is the first time I have tried
to write it all down.

What I still must do..

1) Write some junit tests
2) Update the documentation
3) Update the DTD
  - I added 1 new properties to line
    - delimit- which tells the parser we are not dealing with a fixed
record type.
  - added a substring conversion-option.
4) Record matching needs to consider delimited files

Possible future features...

1)  Add support for returning an object tree Vs. single beans
 - In working with Kodo & Hibernate, this feature seems like it should
be there
 - This would work nice if you had an object that contained list of
other objects. (in our case a ticket contains a list of 4 coupon
records)
   - it would be nice to get back a single object with everything ready
to go.  Currently, the 'handler' would have to construct one and pass it
on to the business layer.
2)  I think there needs to be further refactoring of how a line is
actually parsed.  I have a PERL file parser that can even handle binary
files.
  - Record matching still assumes fixed format.
  - We need to be able to ask the Line about a specific field by it's
index, regardless of how it is actually implemented in the input file.
  - Maybe come up with some sort of plugable architecture that allows
various storage strategies to be considered.

I am sure there will be enhancements/fixes that arise out of my
Customers actually using Flatworm to create real programs.  They have 4
due by the end of this month.

I realize that you might not have the time to digest all that I have
done.  If so, that's ok.  But I would like to give you a chance to
incorporate what I have done into your open source initiative, or at
least comment on it.  I am going to submit myself to peer review here
also. Any feedback you can offer would be greatly appreciated.  I will
be happy to share the final production version of everything Flatworm
should you still have interest in what I have done.

thanks...

James Lawrence
Southwest Airlines - Technology Department
jam...@wn...
W(214) 792-6195
-- 
James M. Turner
Product Review Editor, Linux Journal
Senior Contributing Editor, Linux Planet

tu...@bl...
603-552-2020