babeldoc-devel — Babeldoc developer list

Re: [Babeldoc-devel] Babeldoc development

[Bruce M. <br...@mc...>]

Dejan, All:

I am about to make the first release of the RC1 codebase.  This code
is now completely restructured and is ready for the public.  The finishing 
touches will include crypto and j2ee support and I dont know about the 
j2ee.  The last significant hurdle breached today was how babeldoc finds 
its modules when running in a web container.  I introduced a system 
property that tells babeldoc module handling code to scan one or more 
directories for candidate files.  This sounds like a major hack but its 
actuall quite subtle.  Heres the deal:

a Servlet in Tomcat/catalina needs to have all of its libraries in the 
./shared/lib or in the webapp/<war>/WEB-INF/lib directory.  Now, I opted 
to put all the code in shared lib to avoid very bloated war files  and to 
encourage code sharing.  The catalina script to start the web container 
gets started with the environment variable JAVA_OPTS which has the sytem 
property of the directory to scan, which is the shared lib.  As babeldoc 
starts, it scans this directory and finds all the modules.  There is NO 
other way for 3rd party modules to get stuff into the classpath (the 
catalina script disregards the CLASSPATH).

How do we automate the copying over of the libs and the wars to the 
catalina directory.  This is handled by a script which is copied from the 
web/bin directory to the build/bin directory.  It has a number of tokens 
(of the form @token-name@) which get filled in by the build scripts when 
the copy is performed.  The values for the tokens come from the 
modules/web/build.properties file but can be overriden in a top-level 
file, called locals.properties.  These properties tell the script where to 
find catalina, the location of the shared directory and the actual name of 
the script to start catalina. 

so to build and deploy the babeldoc web stuff do this:

c:\babeldoc>build
c:\babeldoc>build\bin\catalina start

this assumes that catalina is in a directory:
\babeldoc\..\tomcat.v4.1

fair enough.

Bruce.


On Fri, 28 Feb 2003, Dejan Krsmanovic wrote:

> Hi Bruce!
> 
> How new Babeldoc is going? Currently I have not much time for working on it,
> but as soon as I have time I will add crypto module to new Babeldoc
> stucture....
> 
> Have a nice weekend,
> Dejan
> 
> p.s. Have you looked at mail archives on SF for this list? There are a lot
> of doubled or even tripled messages! There were also  3 messages from 1969
> year ;)
> 
> 
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel
> 

-- 

Re: [Babeldoc-devel] Babeldoc development

[Bruce M. <br...@mc...>]

Dejan,

Thanks for the email.  Things are going well, I am now working on the web 
section and things are working well except for the runtime discovery 
mechanism in the web container.  Remember that the runtime module inpsects 
the classpath to search for babeldoc modules...  This cannot be done in a 
web container or even a j2ee container, so I have to either explicitly 
list the modules when the container is launched or tell it where to find 
the module jar files.

About the email, I know nothing!

later man.
Bruce.

PS.  The crypto stuff is ready for you ....
just create a new module directory, write a small build.xml (copy one of 
the others, like sql or conversion) and go from there.


On Fri, 28 Feb 2003, Dejan Krsmanovic wrote:

> Hi Bruce!
> 
> How new Babeldoc is going? Currently I have not much time for working on it,
> but as soon as I have time I will add crypto module to new Babeldoc
> stucture....
> 
> Have a nice weekend,
> Dejan
> 
> p.s. Have you looked at mail archives on SF for this list? There are a lot
> of doubled or even tripled messages! There were also  3 messages from 1969
> year ;)
> 
> 
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel
> 

-- 

[Babeldoc-devel] Babeldoc development

[Dejan K. <dej...@nb...>]

Hi Bruce!

How new Babeldoc is going? Currently I have not much time for working on it,
but as soon as I have time I will add crypto module to new Babeldoc
stucture....

Have a nice weekend,
Dejan

p.s. Have you looked at mail archives on SF for this list? There are a lot
of doubled or even tripled messages! There were also  3 messages from 1969
year ;)

Re: [Babeldoc-devel] One configuration file for all babeldoc settings?

[Dejan K. <dej...@nb...>]

I am not sure about this... Why would one large xml configuration file would
be harder to manage than many small properties files or xml files (in case
of xml pipeline configuration). What about SQL configuration? In my opinion
it is even harder to manage then large xml file.

Anyway, having XMLConfigService would not prevent using LightConfigService.
So if user wants to use LightConfigService he could use it, just like he can
use SqlConfigService now (I guess he can, I haven't tried it!)

Dejan

----- Original Message -----
From: "Bruce McDonald" <br...@mc...>
To: "Dejan Krsmanovic" <dej...@nb...>
Cc: "Babeldoc Developers List" <bab...@li...>
Sent: Tuesday, February 25, 2003 4:45 PM
Subject: Re: [Babeldoc-devel] One configuration file for all babeldoc
settings?


> Having one configuration file for babeldoc is not going to happen for the
> simple reason is that it couples non-coupled code.  As for ant having one
> big configuration file - this is a real problem and lots of projects break
> apart the build.xmls.  There are lots of reasons why this is a problem -
> the most imporant one is that it becomes hard to manage.
>

Re: [Babeldoc-devel] One configuration file for all babeldoc settings?

[Bruce M. <br...@mc...>]

Dejan, All:

You might have noticed but I have a real problem with large, unweildy 
build processes.  I do have some experience here and there is a potential 
problem where babeldoc build and configuration gets out of hand.  I really 
want to make the babeldoc build quite decoupled.  The only coupling that I 
want to support is the dependency relationship between modules of code.  
If you put all the configuration options in a single file, then it becomes 
a really issue to support major forward deltas in the source code.  By 
forward deltas, this means that as functionality changes in the individual 
modules, these changes will probably need to be reflected in the mega 
configuration file.  This means that it is up to the developer to maintain 
the relationship between this mega file and the individual files.  The 
current 1.0 builds manages this fine.  Here is the big problem...

Let us consider the case of the web module.  The web module does not 
participate in the commandline babeldoc relationship.  In order words, the 
web package needs to be somewhat different than the babeldoc-core.jar, 
etc.  These means that the build products are copied into the build 
directory, ready for the babeldoc.bat script file to add to the CLASSPATH.  
Heres what the web module must do...

1.  Build the web code as before.   Produce a babeldoc-web.jar
2.  Build a babeldoc-web.war file which includes the jars in the build 
directory, the server XML files and web documents.  A complete war file to 
support all the servlets, etc is then placed into the build directory.
3.  A script in the bin/ directory which will deploy the war files to a 
particular deployment directory and whatever commands to (possibly) start 
the webserver.  We can configure the script to operate based on properties 
in the module/web/build.properties.  This is basically a filtering copy, 
to the bin directory.

Again the principle is clear - the web module builds based on its 
configuration.  There is a small issue:  How do configurations override 
the web configuration - like a different tomcat installation directory, 
etc, etc.  THis i propose be done another toplevel properties file, a 
locals.properties.  This properties file will be read by the web/build.xml 
and will provide overrides by virtue of its being read AFTER the 
build.properties.  If this file is not present, it is not read.  

what do you think?  I will be working with the web module and the j2ee 
module.  Where are the holes, what are the problems...

Bruce.  

On Tue, 25 Feb 2003, Bruce McDonald wrote:

> Having one configuration file for babeldoc is not going to happen for the 
> simple reason is that it couples non-coupled code.  As for ant having one 
> big configuration file - this is a real problem and lots of projects break 
> apart the build.xmls.  There are lots of reasons why this is a problem - 
> the most imporant one is that it becomes hard to manage.
> 
> 
> 
> On Tue, 25 Feb 2003, Dejan Krsmanovic wrote:
> 
> > What about having one xml file for configuration stuff for all aspects of
> > Babeldoc? So instead of many properties file in many folders we could have
> > one large xml file with all configuration. Just like in ant's build xml.
> > 
> > What should be changes? I guess we should have new class for that implements
> > IConfigService or extends ConfigService class, right? The problem is
> > probably with some module specific functionality. For examples scanner is
> > separeted module from core. Anyway, I would like to have scanner
> > configuration in the same file as other stuff. But what if one day we create
> > some new cool stuff like scanners are? Then we should change shema for xml
> > configuration file and that is not good.
> > 
> > Maybe we could handle this by introducing some kind of plugin api for
> > scanners and other possible extrensions to core babeldoc. Or we could just
> > leave scanner configuration out of the core babeldoc config.
> > Any toughts?!
> > 
> > Best regards,
> > Dejan
> > 
> > 
> > 
> > 
> > -------------------------------------------------------
> > This sf.net email is sponsored by:ThinkGeek
> > Welcome to geek heaven.
> > http://thinkgeek.com/sf
> > _______________________________________________
> > Babeldoc-devel mailing list
> > Bab...@li...
> > https://lists.sourceforge.net/lists/listinfo/babeldoc-devel
> > 
> 
> 

-- 

Re: [Babeldoc-devel] One configuration file for all babeldoc settings?

[Bruce M. <br...@mc...>]

Having one configuration file for babeldoc is not going to happen for the 
simple reason is that it couples non-coupled code.  As for ant having one 
big configuration file - this is a real problem and lots of projects break 
apart the build.xmls.  There are lots of reasons why this is a problem - 
the most imporant one is that it becomes hard to manage.



On Tue, 25 Feb 2003, Dejan Krsmanovic wrote:

> What about having one xml file for configuration stuff for all aspects of
> Babeldoc? So instead of many properties file in many folders we could have
> one large xml file with all configuration. Just like in ant's build xml.
> 
> What should be changes? I guess we should have new class for that implements
> IConfigService or extends ConfigService class, right? The problem is
> probably with some module specific functionality. For examples scanner is
> separeted module from core. Anyway, I would like to have scanner
> configuration in the same file as other stuff. But what if one day we create
> some new cool stuff like scanners are? Then we should change shema for xml
> configuration file and that is not good.
> 
> Maybe we could handle this by introducing some kind of plugin api for
> scanners and other possible extrensions to core babeldoc. Or we could just
> leave scanner configuration out of the core babeldoc config.
> Any toughts?!
> 
> Best regards,
> Dejan
> 
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel
> 

-- 

[Babeldoc-devel] One configuration file for all babeldoc settings?

[Dejan K. <dej...@nb...>]

What about having one xml file for configuration stuff for all aspects of
Babeldoc? So instead of many properties file in many folders we could have
one large xml file with all configuration. Just like in ant's build xml.

What should be changes? I guess we should have new class for that implements
IConfigService or extends ConfigService class, right? The problem is
probably with some module specific functionality. For examples scanner is
separeted module from core. Anyway, I would like to have scanner
configuration in the same file as other stuff. But what if one day we create
some new cool stuff like scanners are? Then we should change shema for xml
configuration file and that is not good.

Maybe we could handle this by introducing some kind of plugin api for
scanners and other possible extrensions to core babeldoc. Or we could just
leave scanner configuration out of the core babeldoc config.
Any toughts?!

Best regards,
Dejan

Re: [Babeldoc-devel] Welcome to the "Babeldoc-devel" mailing list (fwd)

[Bruce M. <br...@mc...>]

Dejan,

Hard to fool you....

The initializer property comes into play only during the runtime 
initialization.  After the BabeldocModule class has identfied each of the 
participant babeldoc modules, it calls the Initializer class on each on.  
Each initializer class has to extend BabeldocModule and provide 
implementation for the abstract method initialize.  This is an optional 
field.  If no initializer is provided, notthing is done.

Bruce.



 On Tue, 25 Feb 2003, Dejan Krsmanovic wrote:

> Hi all,
> 
> > Each of the modules is defined as a directory of ./modules.  The contents
> > of each of these module directories contains at least a build.properties
> > and a build.xml.  The build.properties has only three "special" names:
> > Module, Depends and JarFile.  The Module gives the unique name of the
> > module as it is known to client code and thus to the internal dependancy
> > list.  The Depends name lists those other modules that this module is
> > dependant on and the  JarFile name provides the name of the resultant jar
> > file (although this behaviour is not mandated).
> 
> I see there is Initializer property too. What this property will be used
> for?
> 
> Dejan
> 
> 

-- 

Re: [Babeldoc-devel] Welcome to the "Babeldoc-devel" mailing list (fwd)

[Bruce M. <br...@mc...>]

Sorry

This is continuation of the below...

Anyway these modules exist at two levels:  runtime entities; and modular 
build targets.  

Runtime entities.
The idea here is that merely copy these jar files to your classpath.  As 
babeldoc starts, the environment loader will look throught the classpath 
and check each jar file.  The module will be identified based on the 
presence of special attributes in the jar file manifest.  Again this 
primarily defines the name of the module and the list of dependencies.  
Based on these two data, babeldoc will load configuration data in the 
order from the "least dependant" module to most dependent.  This uses the 
configuration merging ability.  This has is hidden from the client code.  

A good example is the services/query configuration file.  In the core 
module there is only the services defined there particular to that module.  
As more modules get added to the classpath, so do more services as their 
properties get merged.  This means that more commands are possible and 
likewise.  

This same merging is handled for all configuration files.  Later 
name=values get merged later than more general name=values.  Thereby 
specializing functions.  At the moment, most of the modules are simply 
additive (no name=value pairs are overwritten).

Build time merging.
The problem here is similar but relates to code dependencies.  This is 
more tricky since ant does not do resursive build in the way that is 
required.  The build, when it starts DOES not know about all the 
dependencies.  Each of the specific modules gets its own build.xml.  There 
are a number of required targets: clean build deploy.   Each can be called 
at anytime.

Each of the modules is defined as a directory of ./modules.  The contents 
of each of these module directories contains at least a build.properties 
and a build.xml.  The build.properties has only three "special" names: 
Module, Depends and JarFile.  The Module gives the unique name of the 
module as it is known to client code and thus to the internal dependancy 
list.  The Depends name lists those other modules that this module is 
dependant on and the  JarFile name provides the name of the resultant jar 
file (although this behaviour is not mandated).

Each of the targets (clean, build and deploy) get called in dependency 
order from the main build.xml.  This discovers those modules in the 
modules directory by reading the build.properties file.  This is the 
common point.  Just keep the build.properties file up to day and you will 
have fewer headaches.  

The operation of both deploy and clean are more open than the build 
target.  The build target must (if it wants to contribute binary code (and 
hence a module.jar file)) dump all jar files and libraries required for 
this module.  The library jar files are usually copied from a "lib" 
directory in the module directory - /modules/core/lib) to the main ./build 
directory.  The bin/babeldoc scripts just simply add all those jar files in 
the ./build directory to the classpath.  From there its easy - the modules 
arrange themselves and you have a complete system.

more later,

Putting junior to bed.

Bruce.


On Mon, 24 Feb 2003, Bruce McDonald wrote:

> Good Developers!
> 
> I have checked in the first release candidate of the version 1 of 
> babeldoc.  Unfortunately I also checked in the build directory :(
> 
> Anyway there it is...  It builds and seems to run ok - I have done almost 
> no testing on it so use it at your peril.  Tomorrow I will clean out the 
> module directories....
> 
> The big idea...
> The big idea is that babeldoc gets broken into smaller modules that have a 
> clear dependency structure.  The unit of dependency is called a module.  
> This module has a name and a list of modules that it is dependent on.  
> This dependency list determines the build order and the runtime 
> initialization order.  
> 
> Bruce.
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel
> 

-- 

[Babeldoc-devel] Welcome to the "Babeldoc-devel" mailing list (fwd)

[Bruce M. <br...@mc...>]

Good Developers!

I have checked in the first release candidate of the version 1 of 
babeldoc.  Unfortunately I also checked in the build directory :(

Anyway there it is...  It builds and seems to run ok - I have done almost 
no testing on it so use it at your peril.  Tomorrow I will clean out the 
module directories....

The big idea...
The big idea is that babeldoc gets broken into smaller modules that have a 
clear dependency structure.  The unit of dependency is called a module.  
This module has a name and a list of modules that it is dependent on.  
This dependency list determines the build order and the runtime 
initialization order.  

Bruce.

[Babeldoc-devel] Problem with Babeldoc web apps

[Dejan K. <dej...@nb...>]

Bruce,
I have problems starting Babeldoc web applications. Have you tried recently
to deploy war files? Also, I have noticed we have http-feed application but
it is not made when creating zip file for distribution? Is that application
in development phase?

Dejan

Re: [Babeldoc-devel] Log4j vs Commons-logging

[Dejan K. <dej...@nb...>]

I found the root of this problem. It happened only when scanner has been
started from folder different than BABELDOC_HOME, since policy file could
not be found. In this situation, log configuration could not be loaded from
jar since SecurityManager would deny access to it.  I will commit fixed
version on CVS.

I will try to remove

Dejan


----- Original Message -----
From: "Dejan Krsmanovic" <dej...@nb...>
To: <bab...@li...>
Sent: Monday, February 24, 2003 9:06 AM
Subject: Re: [Babeldoc-devel] Log4j vs Commons-logging


> OK I have just found an logging error. I am still not sure what it is but
I
> will try to found out what it is. Anyway I noticed that starting scanner
> with -r option (remote scanner) will  produce an error!
>
> If someone experienced something similiar please let me know!
> Dejan
>
>
> ----- Original Message -----
> From: "Dejan Krsmanovic" <dej...@nb...>
> To: <bab...@li...>
> Sent: Friday, February 21, 2003 10:04 AM
> Subject: [Babeldoc-devel] Log4j vs Commons-logging
>
>
> > Hi all,
> >
> > Welcome to Babeldoc mailing list! I tought it is better to comunicate
this
> > way then using forums on SF. I have already posted this question on SF
but
> > it seems that nobody read develpoer forum ;)
> >
> > OK, this problem is related to logging.
> > We should definitly decide if we should use Log4j or commons-logging. In
> > current Babeldoc version we use both. There is really no reasons for
this.
> > We have switched from log4j to commons, but after some problems, Bruce
> have
> > moved back static initializer for log4j configuration in LogService
class.
> I
> > think if we are using commons-logging we should not use any log4j class
in
> > the code. All configuration is done by commons-logging (LogFactory
class).
> > The only thing it need is log4j.configuration enviroment variable. This
> > variable is set using config file so, user is free to use other logging

> > frameworks without changing the code.
> >
> > I would like to know what problems have been noticed when we didn't have
> > static initializer in LogService. We should solve the problem and remove
> all
> > log4j code if we are going to use commons-logging.
> >
> > However, the other question is if we really should use commons-logging?
> The
> > advantages of it is that we have independance of underlying logging
> > framework, that is - we could use jdk 1.4 logging or LogKit framework
> > without changing the code. The main disadvatage is that using
> > commons-logging we cannot use some advanced log4j specific functionality
> > (which we are not using anyway). Also, commons-logging is wrapper and
> there
> > is probably some overhaed compared to clean log4j.
> >
> > So, I want to discuss about this thing with you guys!
> > Log4j or commons-logging, that is the question!
> >
> > Best regards,
> > Dejan
> >
> >
> >
> >
> >
> >
> >
> > -------------------------------------------------------
> > This SF.net email is sponsored by: SlickEdit Inc. Develop an edge.
> > The most comprehensive and flexible code editor you can use.
> > Code faster. C/C++, C#, Java, HTML, XML, many more. FREE 30-Day Trial.
> > www.slickedit.com/sourceforge
> > _______________________________________________
> > Babeldoc-devel mailing list
> > Bab...@li...
> > https://lists.sourceforge.net/lists/listinfo/babeldoc-devel
>
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: SlickEdit Inc. Develop an edge.
> The most comprehensive and flexible code editor you can use.
> Code faster. C/C++, C#, Java, HTML, XML, many more. FREE 30-Day Trial.
> www.slickedit.com/sourceforge
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel

Re: [Babeldoc-devel] Log4j vs Commons-logging

[Dejan K. <dej...@nb...>]

OK I have just found an logging error. I am still not sure what it is but I
will try to found out what it is. Anyway I noticed that starting scanner
with -r option (remote scanner) will  produce an error!

If someone experienced something similiar please let me know!
Dejan


----- Original Message -----
From: "Dejan Krsmanovic" <dej...@nb...>
To: <bab...@li...>
Sent: Friday, February 21, 2003 10:04 AM
Subject: [Babeldoc-devel] Log4j vs Commons-logging


> Hi all,
>
> Welcome to Babeldoc mailing list! I tought it is better to comunicate this
> way then using forums on SF. I have already posted this question on SF but
> it seems that nobody read develpoer forum ;)
>
> OK, this problem is related to logging.
> We should definitly decide if we should use Log4j or commons-logging. In
> current Babeldoc version we use both. There is really no reasons for this.
> We have switched from log4j to commons, but after some problems, Bruce
have
> moved back static initializer for log4j configuration in LogService class.
I
> think if we are using commons-logging we should not use any log4j class in
> the code. All configuration is done by commons-logging (LogFactory class).
> The only thing it need is log4j.configuration enviroment variable. This
> variable is set using config file so, user is free to use other logging
> frameworks without changing the code.
>
> I would like to know what problems have been noticed when we didn't have
> static initializer in LogService. We should solve the problem and remove
all
> log4j code if we are going to use commons-logging.
>
> However, the other question is if we really should use commons-logging?
The
> advantages of it is that we have independance of underlying logging
> framework, that is - we could use jdk 1.4 logging or LogKit framework
> without changing the code. The main disadvatage is that using
> commons-logging we cannot use some advanced log4j specific functionality
> (which we are not using anyway). Also, commons-logging is wrapper and
there
> is probably some overhaed compared to clean log4j.
>
> So, I want to discuss about this thing with you guys!
> Log4j or commons-logging, that is the question!
>
> Best regards,
> Dejan
>
>
>
>
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: SlickEdit Inc. Develop an edge.
> The most comprehensive and flexible code editor you can use.
> Code faster. C/C++, C#, Java, HTML, XML, many more. FREE 30-Day Trial.
> www.slickedit.com/sourceforge
> _______________________________________________
> Babeldoc-devel mailing list
> Bab...@li...
> https://lists.sourceforge.net/lists/listinfo/babeldoc-devel

[Babeldoc-devel] Log4j vs Commons-logging

[Dejan K. <dej...@nb...>]

Hi all,

Welcome to Babeldoc mailing list! I tought it is better to comunicate this
way then using forums on SF. I have already posted this question on SF but
it seems that nobody read develpoer forum ;)

OK, this problem is related to logging.
We should definitly decide if we should use Log4j or commons-logging. In
current Babeldoc version we use both. There is really no reasons for this.
We have switched from log4j to commons, but after some problems, Bruce have
moved back static initializer for log4j configuration in LogService class. I
think if we are using commons-logging we should not use any log4j class in
the code. All configuration is done by commons-logging (LogFactory class).
The only thing it need is log4j.configuration enviroment variable. This
variable is set using config file so, user is free to use other logging
frameworks without changing the code.

I would like to know what problems have been noticed when we didn't have
static initializer in LogService. We should solve the problem and remove all
log4j code if we are going to use commons-logging.

However, the other question is if we really should use commons-logging? The
advantages of it is that we have independance of underlying logging
framework, that is - we could use jdk 1.4 logging or LogKit framework
without changing the code. The main disadvatage is that using
commons-logging we cannot use some advanced log4j specific functionality
(which we are not using anyway). Also, commons-logging is wrapper and there
is probably some overhaed compared to clean log4j.

So, I want to discuss about this thing with you guys!
Log4j or commons-logging, that is the question!

Best regards,
Dejan