sparrow-devel — Discussion of development of Sparrow

Re: [Sparrow-devel] JDO Implementation Comments

[Joel S. <jo...@ik...>]

> Code Modification
> 
> We did not do code modification, but I had a play with the BCEL library 
> and I think that that would be a candidate for use. There is (was?) a 
> better library from IBM alphaworks that presents an API similar to 
> reflection, but allowing modification of byte-code, but I think that was 
> not under open source. The enhancement would need to be pluggable, to 
> allow us to try out some different implementations.

I've looked at various open source bytecode manipulation APIs and
they're much the same.

I think BCEL will have the largest community and best support probably
(because it's apache) and so I think that would be the best choice.

> We used source-code modification and reflection, ie the jdoXXX calls 
> were in a PersistenceCapableImpl superclass from which all our db 
> classes descended,

This isn't possible. That would require multiple inheritance when the PC
already extends something else.

> and the ClassManagers used reflection to read the 
> actual values. Classes encapsulated all fields with getXXX and setXXX 
> methods and explicitly called jdoMarkDirty() methods on modification. 

With the way the spec is set up, the state manager will know everything
(it knows more than the PC does itself!) so we won't need reflection
anywhere (that I know of).

> Having done this I can see why code modification is necessary - we had a 
> few bugs when a method was added but the jdoMarkDirty() was not :-) I 
> would suggest that an implementation should aim at hand-modified source 
> and worry about code-modification in parallel.

Perhaps, though we have some work already done towards it--and it
appears there is a separate body that Andy knows of that we may be able
to use some code (including an enhancer) from.

Either way, code enhancement is not a big project because it's pretty
much spelled out in the spec and I'm not sure we want to deviate from
that because of compatibility requirements.

> HOLLOW state
> 
> We did not implement the HOLLOW state, which made the code for 1-1 
> relationships nasty. Every relationship getter required a 
> jdoLoad("relationshipName") method to trigger loading of the appropriate 
> object. I would say HOLLOW state is a must.
> 
> Caching
> 
> Our PersistenceManagerImpl maintained 2 caches - a memory sensitive 
> cache like objectbridge, and a dirtyCache which maintained only objects 
> marked as PERSISTENT_DIRTY, PERSISTENT_NEW or PERSISTENT_DELETED in a 
> list. This meant that db checking was only the small number of changed 
> objects. This is a great boon for the JDO.
> 
> savepoint()
> 
> As hinted above, we extended the pm to have a checkpoint() method, which 
> proved extremely useful. This is less so now, as the spec has been 
> modified to allow the user to call setRetainValues(true) so that objects 
> are not made HOLLOW on commit. However there is still a problem 
> potentially with objects that outlive a pm. For example, if you create 
> an object and then want to cache it in a singleton, then changes to the 
> object are not automatically persisted, unless the singleton makes a 
> commit(). It then has to re-associate all objects with a new PM in order 
> to record new changes to the objects. It was much simpler to have a PM 
> associated with the singleton and call checkpoint() periodically. Note 
> that this is on the TO-DO list of the 1.0 spec as savepoint().

I think I understand you. So you're talking outside of transactions?

> The new optional 'optimistic transactions' part of the spec is a good 
> fit for this functionality too, and better in that it doesn't maintain a 
> connection to the database for a long lived pm. This allows usage 
> patterns like creating a PM and storing it in the session of a web 
> transaction, and managing all interactions through this PM. We had to 
> abandon this idea since our implementation maintained an open connection 
> for each open PM, and we rapidly ran out of connections.

Does the spec require one connection per PM? I would think that would be
a bad thing. I would think the db connection pool should be separated
from the PM so the PM could just grab a connection when it needed one.

> Change logging
> 
> One of the interesting things about our implementation was the 
> changelog. Any change to the database was recorded by a Change object, 
> that held the className, fieldNames and the old and new values of 
> changed fields. The fieldName was used so that the Change object could 
> be used to apply changes to a database with a different schema, or where 
> the mapping had changed. This was used to synchronize our web database 
> (MySQL) with our internal database (MS SQL Server). It worked extremely 
> well, and I would heartily recomment this approach. It was implemented 
> by having the PersistenceManagerFactory (nb, not the individual PMs) 
> fire transactionCommitted events with a list of changes. This allows the 
> maximum amount of flexibility in monitoring changes on the database. the 
> changes were logged in an XML format for transfer.

Interesting.

> Queries
> 
> The most we ever did with queries was to filter for objects. It was 
> never necessary to use complex queries, since the relationships 
> maintained by JDO allow you to navigate to the necessary object. I'm 
> sure anyoune who has had to embed queries in code and debug them would 
> agree that its easier to avoid it than work with it. So I don't think 
> complex query support is essential for early versions.
> 
> Having said that, over time it would be nice to support variants like 
> OQL and EJBQL for the implementation, since this would allow an 
> isql-like app that manipulates live object data. Now that would be a 
> killer app.
> 
> IMHO the implementation of querying should be aimed at two uses:
> 
> 1. I have a collection of instances and I want to use a query construct 
> to sort/filter them. So the filter must be applicable to a set of 
> objects without hitting the database.
> 
> 2. I want to fetch data from the database according to a filter. This is 
> easily implemented by navigating an extent and using the same filter as 
> in (1) above, but this is extremely inefficient to the point of being 
> useless in large systems. Thus the system must be able to pass all or 
> part of the query 'down the chain' to a lower level component that can 
> use all or part of it to generate an SQL query. The full filter can then 
> be applied to the smaller set of objects returned. This will be crucial 
> for real world use.

Okay. As you suggested, I envision the querying engine to be one of the
later modules developed as it is rather complicated and because you can
have a very useful piece of software without it (see Ozone for example).

Thanks for your comments!
-- 
Joel Shellman
Comprehensive Internet Solutions -- Building business dreams.
[ web design | database | e-commerce | hosting | marketing ]
iKestrel, Inc.  http://www.ikestrel.com/

[Sparrow-devel] JDO Implementation Comments

[Chris S. <sk...@us...>]

Hi guys, I was pointed to this group by Thomas Mahler of the 
objectbridge group.

I am interested in helping with the sparrow/objectbridge efforts to 
implement JDO in open source. What follows is a description of what we 
did, with some thoughts about the JDO spec as it currently stands, and 
what I think is important to target.

I am moving between countries at the moment, and don't have permanent 
Internet connection, so this is off the top of my head.

At my former employer we implemented a persistence mechanism based on 
the JDO early drafts. It has been in use for over a year now on our 
website, and coped well with a large amount of traffic (30,000 distinct 
page views in a day, with heavy database access, running at a peak of 
25+ pages per second). The library interfaces with a MySQL server on the 
web site, and MS SQL Server on the internal side. Data is synchronized 
in real time through the persistence libraries. I can vouch for the 
suitability of the spec for real world use :-)

Code Modification

We did not do code modification, but I had a play with the BCEL library 
and I think that that would be a candidate for use. There is (was?) a 
better library from IBM alphaworks that presents an API similar to 
reflection, but allowing modification of byte-code, but I think that was 
not under open source. The enhancement would need to be pluggable, to 
allow us to try out some different implementations.

We used source-code modification and reflection, ie the jdoXXX calls 
were in a PersistenceCapableImpl superclass from which all our db 
classes descended, and the ClassManagers used reflection to read the 
actual values. Classes encapsulated all fields with getXXX and setXXX 
methods and explicitly called jdoMarkDirty() methods on modification. 
Having done this I can see why code modification is necessary - we had a 
few bugs when a method was added but the jdoMarkDirty() was not :-) I 
would suggest that an implementation should aim at hand-modified source 
and worry about code-modification in parallel.

HOLLOW state

We did not implement the HOLLOW state, which made the code for 1-1 
relationships nasty. Every relationship getter required a 
jdoLoad("relationshipName") method to trigger loading of the appropriate 
object. I would say HOLLOW state is a must.

Caching

Our PersistenceManagerImpl maintained 2 caches - a memory sensitive 
cache like objectbridge, and a dirtyCache which maintained only objects 
marked as PERSISTENT_DIRTY, PERSISTENT_NEW or PERSISTENT_DELETED in a 
list. This meant that db checking was only the small number of changed 
objects. This is a great boon for the JDO.

savepoint()

As hinted above, we extended the pm to have a checkpoint() method, which 
proved extremely useful. This is less so now, as the spec has been 
modified to allow the user to call setRetainValues(true) so that objects 
are not made HOLLOW on commit. However there is still a problem 
potentially with objects that outlive a pm. For example, if you create 
an object and then want to cache it in a singleton, then changes to the 
object are not automatically persisted, unless the singleton makes a 
commit(). It then has to re-associate all objects with a new PM in order 
to record new changes to the objects. It was much simpler to have a PM 
associated with the singleton and call checkpoint() periodically. Note 
that this is on the TO-DO list of the 1.0 spec as savepoint().

The new optional 'optimistic transactions' part of the spec is a good 
fit for this functionality too, and better in that it doesn't maintain a 
connection to the database for a long lived pm. This allows usage 
patterns like creating a PM and storing it in the session of a web 
transaction, and managing all interactions through this PM. We had to 
abandon this idea since our implementation maintained an open connection 
for each open PM, and we rapidly ran out of connections.

Change logging

One of the interesting things about our implementation was the 
changelog. Any change to the database was recorded by a Change object, 
that held the className, fieldNames and the old and new values of 
changed fields. The fieldName was used so that the Change object could 
be used to apply changes to a database with a different schema, or where 
the mapping had changed. This was used to synchronize our web database 
(MySQL) with our internal database (MS SQL Server). It worked extremely 
well, and I would heartily recomment this approach. It was implemented 
by having the PersistenceManagerFactory (nb, not the individual PMs) 
fire transactionCommitted events with a list of changes. This allows the 
maximum amount of flexibility in monitoring changes on the database. the 
changes were logged in an XML format for transfer.

Queries

The most we ever did with queries was to filter for objects. It was 
never necessary to use complex queries, since the relationships 
maintained by JDO allow you to navigate to the necessary object. I'm 
sure anyoune who has had to embed queries in code and debug them would 
agree that its easier to avoid it than work with it. So I don't think 
complex query support is essential for early versions.

Having said that, over time it would be nice to support variants like 
OQL and EJBQL for the implementation, since this would allow an 
isql-like app that manipulates live object data. Now that would be a 
killer app.

IMHO the implementation of querying should be aimed at two uses:

1. I have a collection of instances and I want to use a query construct 
to sort/filter them. So the filter must be applicable to a set of 
objects without hitting the database.

2. I want to fetch data from the database according to a filter. This is 
easily implemented by navigating an extent and using the same filter as 
in (1) above, but this is extremely inefficient to the point of being 
useless in large systems. Thus the system must be able to pass all or 
part of the query 'down the chain' to a lower level component that can 
use all or part of it to generate an SQL query. The full filter can then 
be applied to the smaller set of objects returned. This will be crucial 
for real world use.

[Sparrow-devel] Collaboration (was: Join Sparrow)

[Joel S. <jo...@ik...>]

On Fri, 2002-04-05 at 11:11, Andy Lewis wrote:
> AS far as collaboration, I see a couple of things here. Ideally there would
> be a single JDO implementation, and the object database is the backend
> instead of an O/R mapping tool. Should be quite doable. However, and that
> point I have to ask if the effort isn't really segmented into three parts:
> 
> JDO Implementation
> ObjectBridge O/R Storage
> Sparrow ODBMS Storage

There may be some additions to that, but in the simplistic sense, that's
generally the idea.

> And if so, how would that be managed?
> Done correctly, one, or at most two jars, and the "drop and go" capability
> should be achievable.

Right, it may involve multiple jars because of desirable modularity.

> While I completely support the open source effort, I found Kodo useful for
> evaluating how the JDO interface and so on actually works.

I'll see about taking a look at it. I was a little concerned about
legality. I was rather surprised by ObjectBridge mucking around the JDO
RI as I thought there would be legal concerns with that--maybe not. I'm
no lawyer. Licenses make things so complicated and confusing.

Forte has a built in JDO support I think with Pointbase or something
like that. I've glanced at how it works. It can work based on running in
a custom classloader--something we discussed on how to do the
enhancement seamlessly.

> The limited developer support would mean some possible recruiting efforts
> to get the manpower, but with a bit of project momentum, that should be
> managable. A lot of people want this.

Right. I think we need to show some significant progress before people
will be willing to jump on board and contribute. I think that's why it
stalled last year.

I have an appreciation for XP and though design is crucial, I think
getting something that compiles and does something is a critical
milestone that should be achieved as quickly as possible--which I had
until my hard drive crashed. Once that is reached (again), I think we'll
be able to start building momentum.
-- 
Joel Shellman
Comprehensive Internet Solutions -- Building business dreams.
[ web design | database | e-commerce | hosting | marketing ]
iKestrel, Inc.  http://www.ikestrel.com/

Re: [Sparrow-devel] Maven

[Joel S. <jo...@ik...>]

On Fri, 2002-04-05 at 10:21, Andy Lewis wrote:
> Maven is comprehensive, well thought out, and very detailed. But Maven is
> also way to heavy for a starting point for a team. Most of the decisions
> there need to be discussed and agreed upon by a community to be effective.
> I would propose using it as a highly regarded reference source, but each
> community needs to work out it operations organicly.

Heavy? Hmm... not what I've read so far. It seems to take care of a lot
of the questions we had been discussing before. It seems rather
straightforward, takes into account many best practices... seems pretty
common sensical to me to use it.

> The first things I see that need to be done for this to move forward are a
> defined decision making process. Voting? If so, who votes, and what
> consitutes a passed vote - majority? 2/3 majority? unanimous? Let's
> establish some basic governance principales that everyone is comfortable
> with first. Then making decisions will be clear, and concise.

At this point, considering that the only contributors besides myself are
not actively working on the project (that I know of), there's not much
basis to divvy out voting rights at this point. Contributors get voting
rights based on the contribution--a vote of current voters gives someone
voting rights.

We discussed this initially and had agreed that a general voting process
with a benevolent dictator (me) breaking things up if a logjam occurs.

Until contributors come forward and claim their voting rights, I'll move
things forward being influenced by any parties who submit their
opinions.

> That is the most basic starting point.
> 
> After that, there needs to be decisions made on how communications are
> handled and recorded, and then there needs to be some agreement on how work
> is assigned/volunteered for/allocated.

There is already a owners document in CVS. Unfortunately, as I
mentioned, no one other than me is actively working on the project yet,
it's out of date.

> Then the project can start moving ahead.
> 
> Bear with me if I seem out of line here, but while my time is limited, and
> I may not be able to contribute much in the way of acutal coding, I do want
> to see this succeed....

No problem. But I thought we already covered much of what you are
discussing.

-- 
Joel Shellman
Comprehensive Internet Solutions -- Building business dreams.
[ web design | database | e-commerce | hosting | marketing ]
iKestrel, Inc.  http://www.ikestrel.com/

Re: [Sparrow-devel] Re: Join Sparrow

[Joel S. <jo...@ik...>]

On Fri, 2002-04-05 at 10:22, Andy Lewis wrote:
> I will need to clear that first - the copyright is own by a Corporation,
> although I am the President of the coorporation, I am not the only party
> with a vested interest.

Understandable, I thought it might be something like that.

> Let me ask a couple of additional questions while I'm at it...
> 
> As a curiosity, what would be your ideal level of colaboration with the
> ObjectBridge group?

I'm not sure. Similar to you, I just want to see this thing done (well,
I want especially to see it done "right") so I can use it as I am sick
of wasting time dealing with RDBMS and O/R. But that includes what I
want--I want an object database behind it, so basically you drop
sparrow.jar in your classpath and boom, you're all done. Maybe some
slight configuration required, but I want it as close to that simple as
possible.

> Have you used/looked at the Kodo product from SolarMetric? They have a 45
> day fully functional demo that is spec compliant. They also have free news
> groups that have a great history of thier support issues, including
> detailed discussions about compliance.

I have heard of it but not looked at it yet. I want an open source
implementation available for people like me to feel free to use however
we like.

> How many people listed as project memebers are actually likly to be able to
> contribute?

At this point, very few I imagine. I don't know if some will jump back
in after some significant progress has started, but I haven't heard from
any of them for a very long time except you and one or two others.

> Is Sparrow staying LGPL, and are you willing to let a community decision
> making process control that? Who will hold the copyright on the Sparrow
> code?

I am willing to consider various options. LGPL seemed fairly safe, but
if there are enough people that want to see something different, I don't
care too much. I want to see efficiency and no wasting of time worrying
about licensing. I'm sick of 1000's of developers writing and rewriting
the same code countless times. So I'm pretty open regarding licensing so
long as it makes life better for people.

-- 
Joel Shellman
Comprehensive Internet Solutions -- Building business dreams.
[ web design | database | e-commerce | hosting | marketing ]
iKestrel, Inc.  http://www.ikestrel.com/

Re: [Sparrow-devel] List Question....

[Joel S. <jo...@ik...>]

On Fri, 2002-04-05 at 10:23, Andy Lewis wrote:
> Why does this list not put the list manager itself as the Reply-To in
> messages?

1) It's the default for mailing lists on sourceforge. Use the "reply to
all" or "reply to list" command in your email client.

2) See:
http://sourceforge.net/docman/display_doc.php?docid=6693&group_id=1

-- 
Joel Shellman
Comprehensive Internet Solutions -- Building business dreams.
[ web design | database | e-commerce | hosting | marketing ]
iKestrel, Inc.  http://www.ikestrel.com/

[Sparrow-devel] List Question....

[Andy L. <aj...@as...>]

Why does this list not put the list manager itself as the Reply-To in
messages?

"The heights of genius are only measurable by the depths of stupidity."

Re: [Sparrow-devel] Maven

[Andy L. <aj...@as...>]

Maven is comprehensive, well thought out, and very detailed. But Maven is
also way to heavy for a starting point for a team. Most of the decisions
there need to be discussed and agreed upon by a community to be effective.
I would propose using it as a highly regarded reference source, but each
community needs to work out it operations organicly.

The first things I see that need to be done for this to move forward are a
defined decision making process. Voting? If so, who votes, and what
consitutes a passed vote - majority? 2/3 majority? unanimous? Let's
establish some basic governance principales that everyone is comfortable
with first. Then making decisions will be clear, and concise.

That is the most basic starting point.

After that, there needs to be decisions made on how communications are
handled and recorded, and then there needs to be some agreement on how work
is assigned/volunteered for/allocated.

Then the project can start moving ahead.

Bear with me if I seem out of line here, but while my time is limited, and
I may not be able to contribute much in the way of acutal coding, I do want
to see this succeed....

> I would like to propose we follow the project guidelines as outlined by
> apache's Maven:
>
> http://jakarta.apache.org/turbine/maven
>
> That seems like it will make a lot of mundane project structure
> decisions for us based on those involved in similar projects. As Maven
> is adopted more, it will make various projects easier to use and
> integrate.
>
> Any comments?
> --
> Joel Shellman
> Comprehensive Internet Solutions -- Building business dreams.
> [ web design | database | e-commerce | hosting | marketing ]
> iKestrel, Inc.  http://www.ikestrel.com/
>
> _______________________________________________
> Sparrow-devel mailing list
> Spa...@li...
> https://lists.sourceforge.net/lists/listinfo/sparrow-devel


"The heights of genius are only measurable by the depths of stupidity."

[Sparrow-devel] Maven

[Joel S. <jo...@ik...>]

I would like to propose we follow the project guidelines as outlined by
apache's Maven:

http://jakarta.apache.org/turbine/maven

That seems like it will make a lot of mundane project structure
decisions for us based on those involved in similar projects. As Maven
is adopted more, it will make various projects easier to use and
integrate.

Any comments?
-- 
Joel Shellman
Comprehensive Internet Solutions -- Building business dreams.
[ web design | database | e-commerce | hosting | marketing ]
iKestrel, Inc.  http://www.ikestrel.com/

Re: [Sparrow-devel] bytecode manipulator

[Joel S. <jo...@ik...>]

On Fri, 2002-04-05 at 09:20, Andy Lewis wrote:
> I just subscribed on ObjectBridge as well.
> 
> I haven't heard back, but the conversation just stareted yesterday, so I
> have hopes...

Okay, I'll hold off for a little bit on the work on the enhancer then.

> On another note - are you aware of the number of other projects on
> SourceForge that deal with Java object persistence?

Yes. All the ones that I saw yesterday about JDO have only one member
and absolutely nothing happening that is visible on Sourceforge except
for ObjectBridge and Lyophilizer.

Do you know of anything else?

-joel

Re: [Sparrow-devel] bytecode manipulator

[Andy L. <aj...@as...>]

+1 here.

BCEL is the way to go. I am having an offline thread with someone who in
the world of Academia right now and is developing a full JDO implementation
(though not and Object/Relational mapping system) and has already built,
among other things, an enhancer using BCEL that sounds like it may already
be fully compliant.

Also- any worlkd back from the ObjectBridge group? I see a great deal of
potential for collaboration there...

> If no one else has any further comment on the subject, I think it would
> be best to use Apache's BCEL for bytecode manipulation.
>
> It seems to me that it will have the largest community and support
> behind it.
>
> The previous work I had done was with a Maurice's bytecode library he
> had written himself. But now that I have lost my work on that, I think
> I'll go back and finish it using BCEL so we can build off Apache's
> work.
>
> Any comments?
> --
> Joel Shellman
>
> _______________________________________________
> Sparrow-devel mailing list
> Spa...@li...
> https://lists.sourceforge.net/lists/listinfo/sparrow-devel


"The heights of genius are only measurable by the depths of stupidity."

[Sparrow-devel] bytecode manipulator

[Joel S. <jo...@ik...>]

If no one else has any further comment on the subject, I think it would
be best to use Apache's BCEL for bytecode manipulation.

It seems to me that it will have the largest community and support
behind it.

The previous work I had done was with a Maurice's bytecode library he
had written himself. But now that I have lost my work on that, I think
I'll go back and finish it using BCEL so we can build off Apache's work.

Any comments?
-- 
Joel Shellman

Re: [Sparrow-devel] Re: Join Sparrow

[Andy L. <aj...@as...>]

The ObjectBridge group already has a fully functional product with and ODMG
3.0 interface, and a second slightly cleaner interface. They are looking to
add JDO to this. The also seem to have a decent active community going. I
think collaboration would be an excellent idea, and a great path to a
working solution...


> Zak has been added as a developer.
>
> I have just noticed the OJB project on sourceforge.
> http://sourceforge.net/projects/objectbridge
>
> They are an O/R bridge that is now looking into supporting the JDO api.
> I have emailed them and suggested collaboration.
>
> I'm going to try to go through CVS and get it all updated and get
> moving again.
>
> -joel
>
>
> On Thu, 2002-04-04 at 07:16, Zakarya Mandhro wrote:
>> Joel,
>>
>> I would like to make open source JDO a reality. I feel that an RDBMS
>> persistence engine would be most practical. I understand that is an
>> enormous undertaking, but if we can gather up enough talent, it is
>> definitely possible.
>>
>> How much time do you and the rest of the developers have to commit to
>> this?
>>
>> Let me join as a developer and we can discuss this in the forum.
>>
>> - Zak
>> --- Joel Shellman <jo...@ik...> wrote:
>> > Thank you for your interest. I had it to the point where it would
>> > persist an object (just very basic functionality) and had the
>> > enhancer done and then my hard drive crashed. I thought I had a
>> > backup but I can't find it.
>> >
>> > So we're back to where we were. We're mostly just starting still,
>> > please look through the mailing archives. I'm mostly focused on a
>> > file store  implementation (similar to Ozone) for speed and ease of
>> > use. But obviously a relational mapping is important to create a
>> > complete enterprise product.
>> >
>> > I definitely welcome your help. Is there a specific aspect you would
>> > most be interested in working on?
>> >
>> > Should I add you as a developer to the sourceforge project?
>> >
>> > Thanks,
>> >
>> > -joel
>> >
>> >
>> > On Wed, 2002-04-03 at 06:57, Zak Mandhro wrote:
>> > > Joel,
>> > >
>> > > I have been working with persistent Java objects for over
>> > > two years now. I have released two large scale ($2.5
>> > > million+) projects using JDO-like models. I started with my  own
>> > > persistence engine and later used jRelational
>> > > Framework, CMP, and Castor. I am currently working on a
>> > > framework that extends Struts and Castor to provide a
>> > > complete end-to-end web application framework designed for
>> > > business functions (not e-commerce).
>> > >
>> > > I would like to participate, as a developer, on an open
>> > > source JDO implementation. I would like to helpout with
>> > > Sparrow. Let me know the status of the project and if I can  help
>> > > out.
>> > >
>> > > Thanks,
>> > > - Zak
>> >
>>
>>
>> __________________________________________________
>> Do You Yahoo!?
>> Yahoo! Tax Center - online filing with TurboTax
>> http://taxes.yahoo.com/
>>
> --
> Joel Shellman
> Comprehensive Internet Solutions -- Building business dreams.
> [ web design | database | e-commerce | hosting | marketing ]
> iKestrel, Inc.  http://www.ikestrel.com/
>
> _______________________________________________
> Sparrow-devel mailing list
> Spa...@li...
> https://lists.sourceforge.net/lists/listinfo/sparrow-devel


"The heights of genius are only measurable by the depths of stupidity."

[Sparrow-devel] Re: Join Sparrow

[Joel S. <jo...@ik...>]

Zak has been added as a developer.

I have just noticed the OJB project on sourceforge.
http://sourceforge.net/projects/objectbridge

They are an O/R bridge that is now looking into supporting the JDO api.
I have emailed them and suggested collaboration.

I'm going to try to go through CVS and get it all updated and get moving
again.

-joel


On Thu, 2002-04-04 at 07:16, Zakarya Mandhro wrote:
> Joel,
> 
> I would like to make open source JDO a reality. I feel that an RDBMS
> persistence engine would be most practical. I understand that is an enormous
> undertaking, but if we can gather up enough talent, it is definitely possible.
> 
> How much time do you and the rest of the developers have to commit to this?
> 
> Let me join as a developer and we can discuss this in the forum.
> 
> - Zak
> --- Joel Shellman <jo...@ik...> wrote:
> > Thank you for your interest. I had it to the point where it would
> > persist an object (just very basic functionality) and had the enhancer
> > done and then my hard drive crashed. I thought I had a backup but I
> > can't find it.
> > 
> > So we're back to where we were. We're mostly just starting still, please
> > look through the mailing archives. I'm mostly focused on a file store 
> > implementation (similar to Ozone) for speed and ease of use. But
> > obviously a relational mapping is important to create a complete
> > enterprise product.
> > 
> > I definitely welcome your help. Is there a specific aspect you would
> > most be interested in working on?
> > 
> > Should I add you as a developer to the sourceforge project?
> > 
> > Thanks,
> > 
> > -joel
> > 
> > 
> > On Wed, 2002-04-03 at 06:57, Zak Mandhro wrote:
> > > Joel,
> > > 
> > > I have been working with persistent Java objects for over 
> > > two years now. I have released two large scale ($2.5 
> > > million+) projects using JDO-like models. I started with my 
> > > own persistence engine and later used jRelational 
> > > Framework, CMP, and Castor. I am currently working on a 
> > > framework that extends Struts and Castor to provide a 
> > > complete end-to-end web application framework designed for 
> > > business functions (not e-commerce).
> > > 
> > > I would like to participate, as a developer, on an open 
> > > source JDO implementation. I would like to helpout with 
> > > Sparrow. Let me know the status of the project and if I can 
> > > help out.
> > > 
> > > Thanks,
> > > - Zak
> > 
> 
> 
> __________________________________________________
> Do You Yahoo!?
> Yahoo! Tax Center - online filing with TurboTax
> http://taxes.yahoo.com/
> 
-- 
Joel Shellman
Comprehensive Internet Solutions -- Building business dreams.
[ web design | database | e-commerce | hosting | marketing ]
iKestrel, Inc.  http://www.ikestrel.com/

[Sparrow-devel] Statemanager -> PersistenceCapable multiplicity

[Joel S. <jo...@ik...>]

Okay...

JDO was architected in such a way that you can have arbitrary
multiplicity between state managers and PersistenceCapable instances.

The advantage of having one to many is that you have far fewer state
manager objects running around.

One disadvantage is that then you have to associate the state some other
way--whether through a hashmap or something. Which makes me think that
for highest performance, one state manager per PersistenceCapable
instance is probably the best way. I don't like doing it that way,
though. I would much rather have a lot of state information inside the
PersistenceCapable class. But as I was looking through the reference
enhancement information in the spec, if I go by that, then I can't 
really put more stuff in the PersistenceCapable class. Because the PC
keeps delegating almost everything to the state manager.

Anyway, so right now, I think one statemanager per PC
instance--primarily for simplicity and performance. I think the overhead
of that will be far less than having to do a lookup on every call into
SM from PC because you have to get the state object for that PC anyway.


-- 
Joel Shellman
Comprehensive Internet Solutions -- Building business dreams.
[ web design | database | e-commerce | hosting | marketing ]
iKestrel, Inc.  http://www.ikestrel.com/

Re: [Sparrow-devel] Technologies

[Joel S. <jo...@ik...>]

On Tue, 2001-12-04 at 09:48, Joel Shellman wrote:
> I think I'll keep these docs in CVS instead of sourceforge so we can
> have version tracking for them. I realize it's a little harder to get
> at, but it will be better for the project I think. We can upload it to
> sourceforge occasionally.

When I said "on sourceforge" I just meant their doc manager. I'll still
use the CVS repository on sourceforge as we are currently doing.

-joel

[Sparrow-devel] JDK 1.4

[Joel S. <jo...@ik...>]

I'd like to develop against JDK 1.4 so we can take advantage of the new
logging API, the asynchronous IO, and if applicable the regex package,
along with anything else that comes up.

I know it's not final yet, but it should be fairly soon, and I've heard
that it's pretty solid already.
-- 
Joel Shellman
iKestrel, Inc.  http://www.ikestrel.com/

[Sparrow-devel] License

[Joel S. <jo...@ik...>]

This is probably going to be discussed for a while, but I have a
concern. Something brought up in another mailing list made me wonder. I
know there's a huge warfare about licenses and whatever, but one of the
things I like about copylefting is that it protects the initial
developers from a company "stealing" their "open source" code and
bundling it with a proprietary product and selling it. I think this is
unfair to the developers (unless of course they get a cut which could be
accomplished via dual licensing).

Anyway, so I'm leaning toward GPL, though someone mentioned they look at
that strictly esp. as far as Java is concerned, so maybe there needs to
be a Java GPL that takes that into account.

I saw someone on a Java project say they said they considered the GPL
applies to anything running in the same virtual machine had to be GPL
along with it--which is absoluately absurd, and it means you could
legally only run it on a GPLed VM (is there one?) because all the
standard libraries are not GPLed.

Anyway...

As far as philosophy is concerned, GPL's "why not to LGPL" document to
me justifies using LGPL, but I'm concerned about the developer
communities efforts being "stolen" by a proprietary company.

Anyway, I'll keep an open mind and we'll press forward with an
understanding that we'll try to do what's best for the developer
community.
-- 
Joel Shellman
iKestrel, Inc.  http://www.ikestrel.com/

[Sparrow-devel] Technologies

[Joel S. <jo...@ik...>]

Okay, I'm sorry it's been quiet--what with Thanksgiving, being sick for
a week, etc., things kind of slowed down.

Anyway, I'm going to start a document regarding what technologies we're
planning on using.

Also, need to start on some architecture documents.

I think I'll keep these docs in CVS instead of sourceforge so we can
have version tracking for them. I realize it's a little harder to get
at, but it will be better for the project I think. We can upload it to
sourceforge occasionally.

Anyone who wants CVS write access to the project who doesn't already
have it, please request it.

Thanks,
-- 
Joel Shellman
iKestrel, Inc.  http://www.ikestrel.com/

[Sparrow-devel] BCEL

[Joel S. <jo...@ik...>]

Now I remember--I did see BCEL before, but discarded it because it
wasn't released yet. But apparently it has a release candidate now (just
a few days ago), so that's probably good enough for our needs. And if we
find bugs we can fix them (isn't open source great!).

-- 
Joel Shellman
iKestrel, Inc.  http://www.ikestrel.com/

Re: [Sparrow-devel] Byte Code Libraries...

[Andy L. <aj...@as...>]

My thoughts exactly...

Joel Shellman wrote:

>On Mon, 2001-11-19 at 08:43, Andy Lewis wrote:
>
>>Sorry to mention this after all the previous conversations, but was the 
>>Apache BCEL project on the list? It is thier Byte Code Engineering 
>>Library. It offers an abstracted model for class file manipulation, and 
>>appears to even include a corresponding classloader.
>>
>
>Thanks for the heads up. I just took a look at it. That might be our
>best bet. It's got some really nice features and as part of Apache,
>probably will have good ongoing support.
>

Re: [Sparrow-devel] Byte Code Libraries...

[Joel S. <jo...@ik...>]

On Mon, 2001-11-19 at 08:43, Andy Lewis wrote:
> Sorry to mention this after all the previous conversations, but was the 
> Apache BCEL project on the list? It is thier Byte Code Engineering 
> Library. It offers an abstracted model for class file manipulation, and 
> appears to even include a corresponding classloader.

Thanks for the heads up. I just took a look at it. That might be our
best bet. It's got some really nice features and as part of Apache,
probably will have good ongoing support.
-- 
Joel Shellman
iKestrel, Inc.  http://www.ikestrel.com/

[Sparrow-devel] Byte Code Libraries...

[Andy L. <aj...@as...>]

Sorry to mention this after all the previous conversations, but was the 
Apache BCEL project on the list? It is thier Byte Code Engineering 
Library. It offers an abstracted model for class file manipulation, and 
appears to even include a corresponding classloader.

Re: [Sparrow-devel] A JDO Query framework for Sparrow

[Geir O. G. <gr...@on...>]

* Joel Shellman
| >  2. Is there a computer readable form of the JDO Query BNF available
| >     anywhere?
| 
| Not sure, but you can download it from the JDO page. If it's in PDF
| format, it should be convertable to another format. What format do you
| need it in?

I've been thinking about using ANTLR[1]. A colleague of mine is using
it. He likes it. Unfortunately I'm not really familiar with it myself,
but on the positive side it's always interesting to learn something
new. Does any of you have any experience with it - or know any better
suited parser generators?

Now I have a more manageble representation of the JDO Query BNF. See
below. Quote: "The grammar notation is taken from the Java Language
Specification", so I guess it is not directly useable by ANTLR without
some tweaking.

Next on the agenda is to experiment with ANTLR and map the Java
Specification grammar notation[2][3] to something suitable for ANTLR.

Have a nice weekend!

Geir O.

[1] http://www.antlr.org
[2] http://java.sun.com/docs/books/jls/second_edition/html/grammars.doc.html#40485
[3] http://java.sun.com/docs/books/jls/second_edition/html/lexical.doc.html#48198

======================================================================
JDO Query BNF
======================================================================

----------------------------------------------------------------------
* Grammar Notation 

The grammar notation is taken from the Java Language Specification:
Terminal symbols are shown in bold in the productions of the lexical
and syntactic grammars, and throughout this specification whenever the
text is directly referring to such a terminal symbol. These are to
appear in a program exactly as written. Nonterminal symbols are shown
in italic type. The definition of a nonterminal is introduced by the
name of the nonterminal being defined followed by a colon. One or more
alternative right-hand sides for the nonterminal then follow on
succeeding lines. The suffix opt , which may appear after a terminal
or nonterminal, indicates an optional symbol. The alternative
containing the optional symbol actually specifies two right-hand
sides, one that omits the optional element and one that includes
it. When the words one of follow the colon in a grammar definition,
they signify that each of the terminal symbols on the following line
or lines is an alternative definition.

----------------------------------------------------------------------
* Parameter Declaration

This section describes the syntax of the declareParameters argument. 

DeclareParameters: 
  Parameters , opt 

Parameters: 
  Parameter
  Parameters , Parameter 

Parameter:
  Type Identifier

----------------------------------------------------------------------
* Variable Declaration 

This section describes the syntax of the declareVariables argument.

DeclareVariables:
  Variables ;opt

Variables:
  Variable
  Variables ; Variable 

Variable:
  Type Identifier

----------------------------------------------------------------------
* Import Declaration 

This section describes the syntax of the declareImports argument. 

DeclareImports:
  ImportDeclarations ;opt 

ImportDeclarations:
  ImportDeclaration
  ImportDeclarations ; ImportDeclaration 

ImportDeclaration:
  import Name

----------------------------------------------------------------------
* Order Specification 

This section describes the syntax of the setOrdering argument. 

SetOrdering:
  OrderSpecifications ,opt 

OrderSpecifications:
  OrderSpecification
  OrderSpecifications , OrderSpecification 

OrderSpecification: 
  Identifier ascending 
  Identifier descending

----------------------------------------------------------------------
* Filter Expression 

This section describes the syntax of the setFilter
argument.

Basically, the query filter expression is a Java boolean expression,
where some of the Java expression are not permitted. The description
follows the structure of the grammar for of Java expression in chapter
19.12 of the Java Language Specification. The description is
bottom-up, i.e. the last rule Expression is the root of the filter
expression syntax.

Please note, the grammar allows arbritary method calls
(MethodInvocation), where JDO only permits calls to the methods
contains(), isEmpty() , and a number of String methods. This
restriction cannot be expressed in terms of the syntax and has to be
ensured by a semantic check.

Primary:
  Literal
  this
  ( Expression )
  FieldAccess
  MethodInvocation

ArgumentList:
  Expression
  ArgumentList , Expression

FieldAccess:
  Primary . Identifier

MethodInvocation:
  Name ( ArgumentList opt )
  Primary . Identifier ( ArgumentList opt )

PostfixExpression:
  Primary
  Name 

UnaryExpression:
  + UnaryExpression
  - UnaryExpression
  UnaryExpressionNotPlusMinus

UnaryExpressionNotPlusMinus:
  PostfixExpression
  ~ UnaryExpression
  ! UnaryExpression
  CastExpression

CastExpression: 
  ( Type ) UnaryExpression

MultiplicativeExpression:
  UnaryExpression
  MultiplicativeExpression * UnaryExpression
  MultiplicativeExpression / UnaryExpression
  MultiplicativeExpression % UnaryExpression

AdditiveExpression:
  MultiplicativeExpression
  AdditiveExpression + MultiplicativeExpression
  AdditiveExpression - MultiplicativeExpression

RelationalExpression: 
  AdditiveExpression 
  RelationalExpression < AdditiveExpression 
  RelationalExpression > AdditiveExpression
  RelationalExpression <= AdditiveExpression 
  RelationalExpression >= AdditiveExpression

EqualityExpression: 
  RelationalExpression 
  EqualityExpression == RelationalExpression 
  EqualityExpression != RelationalExpression

AndExpression: 
  EqualityExpression 
  AndExpression & EqualityExpression

ExclusiveOrExpression: 
  AndExpression 
  ExclusiveOrExpression ^ AndExpression

InclusiveOrExpression: 
  ExclusiveOrExpression 
  InclusiveOrExpression | ExclusiveOrExpression

ConditionalAndExpression: 
  InclusiveOrExpression 
  ConditionalAndExpression && InclusiveOrExpression

ConditionalOrExpression: 
  ConditionalAndExpression 
  ConditionalOrExpression || ConditionalAndExpression

Expression: 
  ConditionalOrExpression

----------------------------------------------------------------------
* Types 

This section decribes a type specification, used in a parameter or
variable declaration or in a cast expression.

Type 
  PrimitiveType 
  Name 

PrimitiveType: 
  NumericType
  boolean

NumericType: 
  IntegralType
  FloatingPointType

IntegralType: one of 
  byte
  short
  int
  long
  char

FloatingPointType: one of 
  float
  double

----------------------------------------------------------------------
* Literals 

A literal is the source code representation of a value of a primitive
type, the String type, or the null type. Please refer to the Java
Language Specification for the lexical structure of IntegerLiterals,
FloatingPointLiterals, CharacterLiterals and StringLiterals.

IntegerLiteral: ... 

FloatingPointLiteral: ... 

BooleanLiteral: one of
  true
  false

CharacterLiteral: ... 

StringLiteral: ...

NullLiteral:
  null

Literal:
  IntegerLiteral
  FloatingPointLiteral
  BooleanLiteral
  CharacterLiteral
  StringLiteral
  NullLiteral

----------------------------------------------------------------------
* Names

Name:
  Identifier
  QualifiedName

QualifiedName: 
  Name . Identifier

Re: [Sparrow-devel] A JDO Query framework for Sparrow

[Andy L. <aj...@as...>]

no garuntees on quality - but this is the section from the PFD in ASCII 
- probably needs cleaned up.



Joel Shellman wrote:

>On Wed, 2001-11-14 at 02:56, Geir Ove GrXnmo wrote:
>
>>Exactly. :)
>>
>>Two questions:
>>
>> 1. Has anybody started on the design of the JDO query support?
>>
>
>Not yet. You're welcome to do it.
>
>> 2. Is there a computer readable form of the JDO Query BNF available
>>    anywhere?
>>
>
>Not sure, but you can download it from the JDO page. If it's in PDF
>format, it should be convertable to another format. What format do you
>need it in?
>