jdbm

On a related note: I hate warnings and I work with JDK1.6 by default.
Would anyone object against introducing generics into the codebase?

On Mon, Jun 9, 2008 at 4:24 PM, Cees de Groot <cdegroot@...> wrote:
> For starters, I have started a conversion to Maven2+SVN - I have a
> couple of failing unit tests, will try to fix them during my train
> commute back home tonight, and then commit to SF SVN (I really don't
> care whether SF or CH plays SVN service, so if anyone cares to take a
> vote on it, I'll be happy to follow the crowd ;-)).
>
> When that baseline is there, it's probably much easier for people to
> work on the codebase and we can get a couple of bugs fixed.
>
> I grabbed the extensible serializer source code and added it to the
> source tree - the original project doesn't seem to exist anymore so I
> thought this was the quickest way to get rid of a binary-only
> dependency.
>
> On Mon, Jun 9, 2008 at 2:18 PM, Mark Proctor <mproctor@...> wrote:
>> Tatu Saloranta wrote:
>>
>> I wasn't even aware of such dependency. :-/
>> If this is the case I agree that it should be gotten rid of. I wouldn't want
>> a mandatory dependency to Xstream either (although I like xstream and use
>> it), because not only is 'automatic' object serialization not need for all
>> use cases, different apps probably want to use different backend
>> serializers.
>>
>>
>> Yes, didn't mean to couple it to XStream. More meant please make it
>> pluggable/optional and out of interest why extser(which is unmaintained for
>> several years) instead of XStream?
>>
>> This change isn't something new it was done in 3rd of May 2006 by
>> thompsonbry. The 1.0 release was 2005, so no release has incorporated it. I
>> don't believe thompsonbry is an active maintainer any more? So maybe
>> everyone isn't aware of this.
>>
>> JDBM seems to be a nice little project,  a few more regular releases would
>> really help people's confidences, even if it only had minor fixes or
>> changes. People tend to worry about downloading a jar last released 3+ years
>> ago. I've read on the ApacheDS mailing list that they are thinking about
>> forking the code base, which would be a shame to see happen. Although I
>> notice it was moved to codehaus and then ignored - codehaus is a far better
>> development environment than sourceforge, so moving back to codehaus is
>> something I would definitely recommend. I've read of a number of bugs since
>> 2005, with people saying they have patches, someone else also said they had
>> an XA Resource adapter, would be nice to see that in the project too.
>>
>> Mark
>>
>> -+ Tatu +-
>>
>> --- On Sun, 6/8/08, Mark Proctor <mproctor@...> wrote:
>>
>>
>>
>> From: Mark Proctor <mproctor@...>
>> Subject: [Jdbm-general] extensible serializer
>> To: jdbm-general@...
>> Date: Sunday, June 8, 2008, 8:16 PM
>> I'm going over the jdbm code in cvs and concerned to see
>> that JDBM is
>> now tightly coupled to an LGPL like library, Cognitive
>> Web's Extensible
>> Serializer library. As Serializer now extends extser it
>> means that
>> library must always be present, also increasing the
>> depencnies for any
>> derivitive apps. My app does not need it's objects
>> serialized, they are
>> not user objects and all turned into byte[] optimally
>> before being asked
>> to store in JDBM.
>>
>> I hear that 1.1 is out soon, could this be de-coupled, so
>> that it's
>> user's choice if they wish to use this.
>>
>> Also why use extser rather than say xtream which is also
>> extensible,
>> mature and well maintained and provides total control of
>> it's format,
>> and also already provides a  binary format.
>>
>> Mark
>>
>>
>> -------------------------------------------------------------------------
>> Check out the new SourceForge.net Marketplace.
>> It's the best place to buy or sell services for
>> just about anything Open Source.
>> http://sourceforge.net/services/buy/index.php
>> _______________________________________________
>> Jdbm-general mailing list
>> Jdbm-general@...
>> https://lists.sourceforge.net/lists/listinfo/jdbm-general
>>
>>
>>
>>
>>
>>
>> -------------------------------------------------------------------------
>> Check out the new SourceForge.net Marketplace.
>> It's the best place to buy or sell services for
>> just about anything Open Source.
>> http://sourceforge.net/services/buy/index.php
>> _______________________________________________
>> Jdbm-general mailing list
>> Jdbm-general@...
>> https://lists.sourceforge.net/lists/listinfo/jdbm-general
>>
>>
>
>
>
> --
> "Human beings make life so interesting. Do you know, that in a
> universe so full of wonders, they have managed to invent boredom. " -
> Death, in "The Hogfather"
>



-- 
"Human beings make life so interesting. Do you know, that in a
universe so full of wonders, they have managed to invent boredom. " -
Death, in "The Hogfather"

Re: [Jdbm-general] extensible serializer From: Cees de Groot <cdegroot@gm...> - 2008-06-09 14:38 On Mon, Jun 9, 2008 at 4:24 PM, Cees de Groot <cdegroot@...> wrote: > I grabbed the extensible serializer source code and added it to the > source tree - the original project doesn't seem to exist anymore so I > thought this was the quickest way to get rid of a binary-only > dependency. On second thought, I agree it's not good to have JDBM depend on a single serializer. So I'm removing the dependency (rolling back to the pre-extser tag in CVS and checking what happened after that) -- "Human beings make life so interesting. Do you know, that in a universe so full of wonders, they have managed to invent boredom. " - Death, in "The Hogfather"

Re: [Jdbm-general] extensible serializer From: Cees de Groot <cdegroot@gm...> - 2008-06-09 14:49 Committed code from PRE_EXTSER in CVS to Subversion. CVS is now for historical reasons only :) (still need to transfer examples and performance tests)

Re: [Jdbm-general] extensible serializer From: Mark Proctor <mproctor@co...> - 2008-06-09 14:53 Attachments: Message as HTML      Cees de Groot wrote: > For starters, I have started a conversion to Maven2+SVN - I have a > couple of failing unit tests, will try to fix them during my train > commute back home tonight, and then commit to SF SVN (I really don't > care whether SF or CH plays SVN service, so if anyone cares to take a > vote on it, I'll be happy to follow the crowd ;-)). > Advantage with CH is you get fisheye, maven repositories, confluence wiki, jira etc. Basically a much better infrastructure in which to run your project. I believe Jason will resync up the CH project for you again if you like. > When that baseline is there, it's probably much easier for people to > work on the codebase and we can get a couple of bugs fixed. > > I grabbed the extensible serializer source code and added it to the > source tree - the original project doesn't seem to exist anymore so I > thought this was the quickest way to get rid of a binary-only > dependency. > the library is under an LGPL like license, so without the authors permission to re-license under BSD, you can't do that. As it would cause the whole of JBDM to come under this license. > On Mon, Jun 9, 2008 at 2:18 PM, Mark Proctor <mproctor@...> wrote: > >> Tatu Saloranta wrote: >> >> I wasn't even aware of such dependency. :-/ >> If this is the case I agree that it should be gotten rid of. I wouldn't want >> a mandatory dependency to Xstream either (although I like xstream and use >> it), because not only is 'automatic' object serialization not need for all >> use cases, different apps probably want to use different backend >> serializers. >> >> >> Yes, didn't mean to couple it to XStream. More meant please make it >> pluggable/optional and out of interest why extser(which is unmaintained for >> several years) instead of XStream? >> >> This change isn't something new it was done in 3rd of May 2006 by >> thompsonbry. The 1.0 release was 2005, so no release has incorporated it. I >> don't believe thompsonbry is an active maintainer any more? So maybe >> everyone isn't aware of this. >> >> JDBM seems to be a nice little project, a few more regular releases would >> really help people's confidences, even if it only had minor fixes or >> changes. People tend to worry about downloading a jar last released 3+ years >> ago. I've read on the ApacheDS mailing list that they are thinking about >> forking the code base, which would be a shame to see happen. Although I >> notice it was moved to codehaus and then ignored - codehaus is a far better >> development environment than sourceforge, so moving back to codehaus is >> something I would definitely recommend. I've read of a number of bugs since >> 2005, with people saying they have patches, someone else also said they had >> an XA Resource adapter, would be nice to see that in the project too. >> >> Mark >> >> -+ Tatu +- >> >> --- On Sun, 6/8/08, Mark Proctor <mproctor@...> wrote: >> >> >> >> From: Mark Proctor <mproctor@...> >> Subject: [Jdbm-general] extensible serializer >> To: jdbm-general@... >> Date: Sunday, June 8, 2008, 8:16 PM >> I'm going over the jdbm code in cvs and concerned to see >> that JDBM is >> now tightly coupled to an LGPL like library, Cognitive >> Web's Extensible >> Serializer library. As Serializer now extends extser it >> means that >> library must always be present, also increasing the >> depencnies for any >> derivitive apps. My app does not need it's objects >> serialized, they are >> not user objects and all turned into byte[] optimally >> before being asked >> to store in JDBM. >> >> I hear that 1.1 is out soon, could this be de-coupled, so >> that it's >> user's choice if they wish to use this. >> >> Also why use extser rather than say xtream which is also >> extensible, >> mature and well maintained and provides total control of >> it's format, >> and also already provides a binary format. >> >> Mark >> >> >> ------------------------------------------------------------------------- >> Check out the new SourceForge.net Marketplace. >> It's the best place to buy or sell services for >> just about anything Open Source. >> http://sourceforge.net/services/buy/index.php >> _______________________________________________ >> Jdbm-general mailing list >> Jdbm-general@... >> https://lists.sourceforge.net/lists/listinfo/jdbm-general >> >> >> >> >> >> >> ------------------------------------------------------------------------- >> Check out the new SourceForge.net Marketplace. >> It's the best place to buy or sell services for >> just about anything Open Source. >> http://sourceforge.net/services/buy/index.php >> _______________________________________________ >> Jdbm-general mailing list >> Jdbm-general@... >> https://lists.sourceforge.net/lists/listinfo/jdbm-general >> >> >> > > > >

Re: [Jdbm-general] extensible serializer From: Mark Proctor <mproctor@co...> - 2008-06-09 14:56 Attachments: Message as HTML      Cees de Groot wrote: > On a related note: I hate warnings and I work with JDK1.6 by default. > Would anyone object against introducing generics into the codebase? > I think it's acceptible for a project to be JDK1.5+ now. We dropped JDK1.4 on Drools this year. > On Mon, Jun 9, 2008 at 4:24 PM, Cees de Groot <cdegroot@...> wrote: > >> For starters, I have started a conversion to Maven2+SVN - I have a >> couple of failing unit tests, will try to fix them during my train >> commute back home tonight, and then commit to SF SVN (I really don't >> care whether SF or CH plays SVN service, so if anyone cares to take a >> vote on it, I'll be happy to follow the crowd ;-)). >> >> When that baseline is there, it's probably much easier for people to >> work on the codebase and we can get a couple of bugs fixed. >> >> I grabbed the extensible serializer source code and added it to the >> source tree - the original project doesn't seem to exist anymore so I >> thought this was the quickest way to get rid of a binary-only >> dependency. >> >> On Mon, Jun 9, 2008 at 2:18 PM, Mark Proctor <mproctor@...> wrote: >> >>> Tatu Saloranta wrote: >>> >>> I wasn't even aware of such dependency. :-/ >>> If this is the case I agree that it should be gotten rid of. I wouldn't want >>> a mandatory dependency to Xstream either (although I like xstream and use >>> it), because not only is 'automatic' object serialization not need for all >>> use cases, different apps probably want to use different backend >>> serializers. >>> >>> >>> Yes, didn't mean to couple it to XStream. More meant please make it >>> pluggable/optional and out of interest why extser(which is unmaintained for >>> several years) instead of XStream? >>> >>> This change isn't something new it was done in 3rd of May 2006 by >>> thompsonbry. The 1.0 release was 2005, so no release has incorporated it. I >>> don't believe thompsonbry is an active maintainer any more? So maybe >>> everyone isn't aware of this. >>> >>> JDBM seems to be a nice little project, a few more regular releases would >>> really help people's confidences, even if it only had minor fixes or >>> changes. People tend to worry about downloading a jar last released 3+ years >>> ago. I've read on the ApacheDS mailing list that they are thinking about >>> forking the code base, which would be a shame to see happen. Although I >>> notice it was moved to codehaus and then ignored - codehaus is a far better >>> development environment than sourceforge, so moving back to codehaus is >>> something I would definitely recommend. I've read of a number of bugs since >>> 2005, with people saying they have patches, someone else also said they had >>> an XA Resource adapter, would be nice to see that in the project too. >>> >>> Mark >>> >>> -+ Tatu +- >>> >>> --- On Sun, 6/8/08, Mark Proctor <mproctor@...> wrote: >>> >>> >>> >>> From: Mark Proctor <mproctor@...> >>> Subject: [Jdbm-general] extensible serializer >>> To: jdbm-general@... >>> Date: Sunday, June 8, 2008, 8:16 PM >>> I'm going over the jdbm code in cvs and concerned to see >>> that JDBM is >>> now tightly coupled to an LGPL like library, Cognitive >>> Web's Extensible >>> Serializer library. As Serializer now extends extser it >>> means that >>> library must always be present, also increasing the >>> depencnies for any >>> derivitive apps. My app does not need it's objects >>> serialized, they are >>> not user objects and all turned into byte[] optimally >>> before being asked >>> to store in JDBM. >>> >>> I hear that 1.1 is out soon, could this be de-coupled, so >>> that it's >>> user's choice if they wish to use this. >>> >>> Also why use extser rather than say xtream which is also >>> extensible, >>> mature and well maintained and provides total control of >>> it's format, >>> and also already provides a binary format. >>> >>> Mark >>> >>> >>> ------------------------------------------------------------------------- >>> Check out the new SourceForge.net Marketplace. >>> It's the best place to buy or sell services for >>> just about anything Open Source. >>> http://sourceforge.net/services/buy/index.php >>> _______________________________________________ >>> Jdbm-general mailing list >>> Jdbm-general@... >>> https://lists.sourceforge.net/lists/listinfo/jdbm-general >>> >>> >>> >>> >>> >>> >>> ------------------------------------------------------------------------- >>> Check out the new SourceForge.net Marketplace. >>> It's the best place to buy or sell services for >>> just about anything Open Source. >>> http://sourceforge.net/services/buy/index.php >>> _______________________________________________ >>> Jdbm-general mailing list >>> Jdbm-general@... >>> https://lists.sourceforge.net/lists/listinfo/jdbm-general >>> >>> >>> >> >> -- >> "Human beings make life so interesting. Do you know, that in a >> universe so full of wonders, they have managed to invent boredom. " - >> Death, in "The Hogfather" >> >> > > > >

Re: [Jdbm-general] extensible serializer From: Mark Proctor <mproctor@co...> - 2008-06-09 14:58 Attachments: Message as HTML      Cees de Groot wrote: > On Mon, Jun 9, 2008 at 4:24 PM, Cees de Groot <cdegroot@...> wrote: > >> I grabbed the extensible serializer source code and added it to the >> source tree - the original project doesn't seem to exist anymore so I >> thought this was the quickest way to get rid of a binary-only >> dependency. >> Great, I'l update and look over it. For me I just want JDBM to write my byte[], I don't want it to go anywhere near a serialistion method call, I'm currently just trying to find out if that is possible. > > On second thought, I agree it's not good to have JDBM depend on a > single serializer. So I'm removing the dependency (rolling back to the > pre-extser tag in CVS and checking what happened after that) > >

Re: [Jdbm-general] extensible serializer From: Cees de Groot <cdegroot@gm...> - 2008-06-09 15:09 As I said, I'm more than happy to move over (and as I use the Atlassian toolchain at work as well, so much the better :-)). Anyone objecting? If not, we'll need to grab the last couple of sources from CVS, synch over, and close down the sourceforge project. (and publish an artefact on ibiblio.org so that we prove that we are still alive ;-)) On Mon, Jun 9, 2008 at 4:52 PM, Mark Proctor <mproctor@...> wrote: > Cees de Groot wrote: > > For starters, I have started a conversion to Maven2+SVN - I have a > couple of failing unit tests, will try to fix them during my train > commute back home tonight, and then commit to SF SVN (I really don't > care whether SF or CH plays SVN service, so if anyone cares to take a > vote on it, I'll be happy to follow the crowd ;-)). > > > Advantage with CH is you get fisheye, maven repositories, confluence wiki, > jira etc. Basically a much better > infrastructure in which to run your project. I believe Jason will resync up > the CH project for you again if you like. > > When that baseline is there, it's probably much easier for people to > work on the codebase and we can get a couple of bugs fixed. > > I grabbed the extensible serializer source code and added it to the > source tree - the original project doesn't seem to exist anymore so I > thought this was the quickest way to get rid of a binary-only > dependency. > > > the library is under an LGPL like license, so without the authors permission > to re-license under BSD, you can't do that. > As it would cause the whole of JBDM to come under this license. > > On Mon, Jun 9, 2008 at 2:18 PM, Mark Proctor <mproctor@...> wrote: > > > Tatu Saloranta wrote: > > I wasn't even aware of such dependency. :-/ > If this is the case I agree that it should be gotten rid of. I wouldn't want > a mandatory dependency to Xstream either (although I like xstream and use > it), because not only is 'automatic' object serialization not need for all > use cases, different apps probably want to use different backend > serializers. > > > Yes, didn't mean to couple it to XStream. More meant please make it > pluggable/optional and out of interest why extser(which is unmaintained for > several years) instead of XStream? > > This change isn't something new it was done in 3rd of May 2006 by > thompsonbry. The 1.0 release was 2005, so no release has incorporated it. I > don't believe thompsonbry is an active maintainer any more? So maybe > everyone isn't aware of this. > > JDBM seems to be a nice little project, a few more regular releases would > really help people's confidences, even if it only had minor fixes or > changes. People tend to worry about downloading a jar last released 3+ years > ago. I've read on the ApacheDS mailing list that they are thinking about > forking the code base, which would be a shame to see happen. Although I > notice it was moved to codehaus and then ignored - codehaus is a far better > development environment than sourceforge, so moving back to codehaus is > something I would definitely recommend. I've read of a number of bugs since > 2005, with people saying they have patches, someone else also said they had > an XA Resource adapter, would be nice to see that in the project too. > > Mark > > -+ Tatu +- > > --- On Sun, 6/8/08, Mark Proctor <mproctor@...> wrote: > > > > From: Mark Proctor <mproctor@...> > Subject: [Jdbm-general] extensible serializer > To: jdbm-general@... > Date: Sunday, June 8, 2008, 8:16 PM > I'm going over the jdbm code in cvs and concerned to see > that JDBM is > now tightly coupled to an LGPL like library, Cognitive > Web's Extensible > Serializer library. As Serializer now extends extser it > means that > library must always be present, also increasing the > depencnies for any > derivitive apps. My app does not need it's objects > serialized, they are > not user objects and all turned into byte[] optimally > before being asked > to store in JDBM. > > I hear that 1.1 is out soon, could this be de-coupled, so > that it's > user's choice if they wish to use this. > > Also why use extser rather than say xtream which is also > extensible, > mature and well maintained and provides total control of > it's format, > and also already provides a binary format. > > Mark > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Jdbm-general mailing list > Jdbm-general@... > https://lists.sourceforge.net/lists/listinfo/jdbm-general > > > > > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Jdbm-general mailing list > Jdbm-general@... > https://lists.sourceforge.net/lists/listinfo/jdbm-general > > > > > > -- "Human beings make life so interesting. Do you know, that in a universe so full of wonders, they have managed to invent boredom. " - Death, in "The Hogfather"

Re: [Jdbm-general] extensible serializer From: Mark Proctor <mproctor@co...> - 2008-06-09 15:21 Attachments: Message as HTML      Cees de Groot wrote: > As I said, I'm more than happy to move over (and as I use the > Atlassian toolchain at work as well, so much the better :-)). > > Anyone objecting? If not, we'll need to grab the last couple of > sources from CVS, synch over, and close down the sourceforge project. > > (and publish an artefact on ibiblio.org so that we prove that we are > still alive ;-)) > Getting a jar out would be a good thing, the apache ds people either have or are on the verge of creating a fork, so let them know you are going to do some soon. > On Mon, Jun 9, 2008 at 4:52 PM, Mark Proctor <mproctor@...> wrote: > >> Cees de Groot wrote: >> >> For starters, I have started a conversion to Maven2+SVN - I have a >> couple of failing unit tests, will try to fix them during my train >> commute back home tonight, and then commit to SF SVN (I really don't >> care whether SF or CH plays SVN service, so if anyone cares to take a >> vote on it, I'll be happy to follow the crowd ;-)). >> >> >> Advantage with CH is you get fisheye, maven repositories, confluence wiki, >> jira etc. Basically a much better >> infrastructure in which to run your project. I believe Jason will resync up >> the CH project for you again if you like. >> >> When that baseline is there, it's probably much easier for people to >> work on the codebase and we can get a couple of bugs fixed. >> >> I grabbed the extensible serializer source code and added it to the >> source tree - the original project doesn't seem to exist anymore so I >> thought this was the quickest way to get rid of a binary-only >> dependency. >> >> >> the library is under an LGPL like license, so without the authors permission >> to re-license under BSD, you can't do that. >> As it would cause the whole of JBDM to come under this license. >> >> On Mon, Jun 9, 2008 at 2:18 PM, Mark Proctor <mproctor@...> wrote: >> >> >> Tatu Saloranta wrote: >> >> I wasn't even aware of such dependency. :-/ >> If this is the case I agree that it should be gotten rid of. I wouldn't want >> a mandatory dependency to Xstream either (although I like xstream and use >> it), because not only is 'automatic' object serialization not need for all >> use cases, different apps probably want to use different backend >> serializers. >> >> >> Yes, didn't mean to couple it to XStream. More meant please make it >> pluggable/optional and out of interest why extser(which is unmaintained for >> several years) instead of XStream? >> >> This change isn't something new it was done in 3rd of May 2006 by >> thompsonbry. The 1.0 release was 2005, so no release has incorporated it. I >> don't believe thompsonbry is an active maintainer any more? So maybe >> everyone isn't aware of this. >> >> JDBM seems to be a nice little project, a few more regular releases would >> really help people's confidences, even if it only had minor fixes or >> changes. People tend to worry about downloading a jar last released 3+ years >> ago. I've read on the ApacheDS mailing list that they are thinking about >> forking the code base, which would be a shame to see happen. Although I >> notice it was moved to codehaus and then ignored - codehaus is a far better >> development environment than sourceforge, so moving back to codehaus is >> something I would definitely recommend. I've read of a number of bugs since >> 2005, with people saying they have patches, someone else also said they had >> an XA Resource adapter, would be nice to see that in the project too. >> >> Mark >> >> -+ Tatu +- >> >> --- On Sun, 6/8/08, Mark Proctor <mproctor@...> wrote: >> >> >> >> From: Mark Proctor <mproctor@...> >> Subject: [Jdbm-general] extensible serializer >> To: jdbm-general@... >> Date: Sunday, June 8, 2008, 8:16 PM >> I'm going over the jdbm code in cvs and concerned to see >> that JDBM is >> now tightly coupled to an LGPL like library, Cognitive >> Web's Extensible >> Serializer library. As Serializer now extends extser it >> means that >> library must always be present, also increasing the >> depencnies for any >> derivitive apps. My app does not need it's objects >> serialized, they are >> not user objects and all turned into byte[] optimally >> before being asked >> to store in JDBM. >> >> I hear that 1.1 is out soon, could this be de-coupled, so >> that it's >> user's choice if they wish to use this. >> >> Also why use extser rather than say xtream which is also >> extensible, >> mature and well maintained and provides total control of >> it's format, >> and also already provides a binary format. >> >> Mark >> >> >> ------------------------------------------------------------------------- >> Check out the new SourceForge.net Marketplace. >> It's the best place to buy or sell services for >> just about anything Open Source. >> http://sourceforge.net/services/buy/index.php >> _______________________________________________ >> Jdbm-general mailing list >> Jdbm-general@... >> https://lists.sourceforge.net/lists/listinfo/jdbm-general >> >> >> >> >> >> >> ------------------------------------------------------------------------- >> Check out the new SourceForge.net Marketplace. >> It's the best place to buy or sell services for >> just about anything Open Source. >> http://sourceforge.net/services/buy/index.php >> _______________________________________________ >> Jdbm-general mailing list >> Jdbm-general@... >> https://lists.sourceforge.net/lists/listinfo/jdbm-general >> >> >> >> >> >> >> > > > >

Re: [Jdbm-general] extensible serializer From: Cees de Groot <cdegroot@gm...> - 2008-06-09 17:51 I don't have any contact with the DS people - could anyone give them a ping? On Mon, Jun 9, 2008 at 5:20 PM, Mark Proctor <mproctor@...> wrote: > Cees de Groot wrote: > > As I said, I'm more than happy to move over (and as I use the > Atlassian toolchain at work as well, so much the better :-)). > > Anyone objecting? If not, we'll need to grab the last couple of > sources from CVS, synch over, and close down the sourceforge project. > > (and publish an artefact on ibiblio.org so that we prove that we are > still alive ;-)) > > > Getting a jar out would be a good thing, the apache ds people either have or > are on the verge of creating a fork, so let them know you are going to do > some soon. > > On Mon, Jun 9, 2008 at 4:52 PM, Mark Proctor <mproctor@...> wrote: > > > Cees de Groot wrote: > > For starters, I have started a conversion to Maven2+SVN - I have a > couple of failing unit tests, will try to fix them during my train > commute back home tonight, and then commit to SF SVN (I really don't > care whether SF or CH plays SVN service, so if anyone cares to take a > vote on it, I'll be happy to follow the crowd ;-)). > > > Advantage with CH is you get fisheye, maven repositories, confluence wiki, > jira etc. Basically a much better > infrastructure in which to run your project. I believe Jason will resync up > the CH project for you again if you like. > > When that baseline is there, it's probably much easier for people to > work on the codebase and we can get a couple of bugs fixed. > > I grabbed the extensible serializer source code and added it to the > source tree - the original project doesn't seem to exist anymore so I > thought this was the quickest way to get rid of a binary-only > dependency. > > > the library is under an LGPL like license, so without the authors permission > to re-license under BSD, you can't do that. > As it would cause the whole of JBDM to come under this license. > > On Mon, Jun 9, 2008 at 2:18 PM, Mark Proctor <mproctor@...> wrote: > > > Tatu Saloranta wrote: > > I wasn't even aware of such dependency. :-/ > If this is the case I agree that it should be gotten rid of. I wouldn't want > a mandatory dependency to Xstream either (although I like xstream and use > it), because not only is 'automatic' object serialization not need for all > use cases, different apps probably want to use different backend > serializers. > > > Yes, didn't mean to couple it to XStream. More meant please make it > pluggable/optional and out of interest why extser(which is unmaintained for > several years) instead of XStream? > > This change isn't something new it was done in 3rd of May 2006 by > thompsonbry. The 1.0 release was 2005, so no release has incorporated it. I > don't believe thompsonbry is an active maintainer any more? So maybe > everyone isn't aware of this. > > JDBM seems to be a nice little project, a few more regular releases would > really help people's confidences, even if it only had minor fixes or > changes. People tend to worry about downloading a jar last released 3+ years > ago. I've read on the ApacheDS mailing list that they are thinking about > forking the code base, which would be a shame to see happen. Although I > notice it was moved to codehaus and then ignored - codehaus is a far better > development environment than sourceforge, so moving back to codehaus is > something I would definitely recommend. I've read of a number of bugs since > 2005, with people saying they have patches, someone else also said they had > an XA Resource adapter, would be nice to see that in the project too. > > Mark > > -+ Tatu +- > > --- On Sun, 6/8/08, Mark Proctor <mproctor@...> wrote: > > > > From: Mark Proctor <mproctor@...> > Subject: [Jdbm-general] extensible serializer > To: jdbm-general@... > Date: Sunday, June 8, 2008, 8:16 PM > I'm going over the jdbm code in cvs and concerned to see > that JDBM is > now tightly coupled to an LGPL like library, Cognitive > Web's Extensible > Serializer library. As Serializer now extends extser it > means that > library must always be present, also increasing the > depencnies for any > derivitive apps. My app does not need it's objects > serialized, they are > not user objects and all turned into byte[] optimally > before being asked > to store in JDBM. > > I hear that 1.1 is out soon, could this be de-coupled, so > that it's > user's choice if they wish to use this. > > Also why use extser rather than say xtream which is also > extensible, > mature and well maintained and provides total control of > it's format, > and also already provides a binary format. > > Mark > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Jdbm-general mailing list > Jdbm-general@... > https://lists.sourceforge.net/lists/listinfo/jdbm-general > > > > > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Jdbm-general mailing list > Jdbm-general@... > https://lists.sourceforge.net/lists/listinfo/jdbm-general > > > > > > > > > > -- "Human beings make life so interesting. Do you know, that in a universe so full of wonders, they have managed to invent boredom. " - Death, in "The Hogfather"

Re: [Jdbm-general] extensible serializer From: Alex Boisvert <boisvert@in...> - 2008-06-09 18:03 Attachments: Message as HTML      I've discussed with Emmanuel Lecharny <elecharny@...>, who sent me this patch about 2 months ago to add a method to the BTree class: public void setValueSerializer( Serializer valueSerializer ) { _valueSerializer = valueSerializer; } Mea culpa, I just haven't gotten around to applying it :-( alex On Mon, Jun 9, 2008 at 10:51 AM, Cees de Groot <cdegroot@...> wrote: > I don't have any contact with the DS people - could anyone give them a > ping? > > On Mon, Jun 9, 2008 at 5:20 PM, Mark Proctor <mproctor@...> > wrote: > > Cees de Groot wrote: > > > > As I said, I'm more than happy to move over (and as I use the > > Atlassian toolchain at work as well, so much the better :-)). > > > > Anyone objecting? If not, we'll need to grab the last couple of > > sources from CVS, synch over, and close down the sourceforge project. > > > > (and publish an artefact on ibiblio.org so that we prove that we are > > still alive ;-)) > > > > > > Getting a jar out would be a good thing, the apache ds people either have > or > > are on the verge of creating a fork, so let them know you are going to do > > some soon. > > > > On Mon, Jun 9, 2008 at 4:52 PM, Mark Proctor <mproctor@...> > wrote: > > > > > > Cees de Groot wrote: > > > > For starters, I have started a conversion to Maven2+SVN - I have a > > couple of failing unit tests, will try to fix them during my train > > commute back home tonight, and then commit to SF SVN (I really don't > > care whether SF or CH plays SVN service, so if anyone cares to take a > > vote on it, I'll be happy to follow the crowd ;-)). > > > > > > Advantage with CH is you get fisheye, maven repositories, confluence > wiki, > > jira etc. Basically a much better > > infrastructure in which to run your project. I believe Jason will resync > up > > the CH project for you again if you like. > > > > When that baseline is there, it's probably much easier for people to > > work on the codebase and we can get a couple of bugs fixed. > > > > I grabbed the extensible serializer source code and added it to the > > source tree - the original project doesn't seem to exist anymore so I > > thought this was the quickest way to get rid of a binary-only > > dependency. > > > > > > the library is under an LGPL like license, so without the authors > permission > > to re-license under BSD, you can't do that. > > As it would cause the whole of JBDM to come under this license. > > > > On Mon, Jun 9, 2008 at 2:18 PM, Mark Proctor <mproctor@...> > wrote: > > > > > > Tatu Saloranta wrote: > > > > I wasn't even aware of such dependency. :-/ > > If this is the case I agree that it should be gotten rid of. I wouldn't > want > > a mandatory dependency to Xstream either (although I like xstream and use > > it), because not only is 'automatic' object serialization not need for > all > > use cases, different apps probably want to use different backend > > serializers. > > > > > > Yes, didn't mean to couple it to XStream. More meant please make it > > pluggable/optional and out of interest why extser(which is unmaintained > for > > several years) instead of XStream? > > > > This change isn't something new it was done in 3rd of May 2006 by > > thompsonbry. The 1.0 release was 2005, so no release has incorporated it. > I > > don't believe thompsonbry is an active maintainer any more? So maybe > > everyone isn't aware of this. > > > > JDBM seems to be a nice little project, a few more regular releases > would > > really help people's confidences, even if it only had minor fixes or > > changes. People tend to worry about downloading a jar last released 3+ > years > > ago. I've read on the ApacheDS mailing list that they are thinking about > > forking the code base, which would be a shame to see happen. Although I > > notice it was moved to codehaus and then ignored - codehaus is a far > better > > development environment than sourceforge, so moving back to codehaus is > > something I would definitely recommend. I've read of a number of bugs > since > > 2005, with people saying they have patches, someone else also said they > had > > an XA Resource adapter, would be nice to see that in the project too. > > > > Mark > > > > -+ Tatu +- > > > > --- On Sun, 6/8/08, Mark Proctor <mproctor@...> wrote: > > > > > > > > From: Mark Proctor <mproctor@...> > > Subject: [Jdbm-general] extensible serializer > > To: jdbm-general@... > > Date: Sunday, June 8, 2008, 8:16 PM > > I'm going over the jdbm code in cvs and concerned to see > > that JDBM is > > now tightly coupled to an LGPL like library, Cognitive > > Web's Extensible > > Serializer library. As Serializer now extends extser it > > means that > > library must always be present, also increasing the > > depencnies for any > > derivitive apps. My app does not need it's objects > > serialized, they are > > not user objects and all turned into byte[] optimally > > before being asked > > to store in JDBM. > > > > I hear that 1.1 is out soon, could this be de-coupled, so > > that it's > > user's choice if they wish to use this. > > > > Also why use extser rather than say xtream which is also > > extensible, > > mature and well maintained and provides total control of > > it's format, > > and also already provides a binary format. > > > > Mark > > > > > > ------------------------------------------------------------------------- > > Check out the new SourceForge.net Marketplace. > > It's the best place to buy or sell services for > > just about anything Open Source. > > http://sourceforge.net/services/buy/index.php > > _______________________________________________ > > Jdbm-general mailing list > > Jdbm-general@... > > https://lists.sourceforge.net/lists/listinfo/jdbm-general > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > Check out the new SourceForge.net Marketplace. > > It's the best place to buy or sell services for > > just about anything Open Source. > > http://sourceforge.net/services/buy/index.php > > _______________________________________________ > > Jdbm-general mailing list > > Jdbm-general@... > > https://lists.sourceforge.net/lists/listinfo/jdbm-general > > > > > > > > > > > > > > > > > > > > > > > > -- > "Human beings make life so interesting. Do you know, that in a > universe so full of wonders, they have managed to invent boredom. " - > Death, in "The Hogfather" > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Jdbm-general mailing list > Jdbm-general@... > https://lists.sourceforge.net/lists/listinfo/jdbm-general >

Re: [Jdbm-general] extensible serializer From: Mark Proctor <mproctor@co...> - 2008-06-10 16:55 Attachments: Message as HTML      Bryan Thompson wrote: > Well, easy come, easy go - but you might want to see who's using it > before you drop it out. > > There is zero overhead when extser is not enabled. it's not so much the overhead, its the extra dependency. Even if you don't use it you are forced to include it, because Serialiser extends it. So if we are to use it, it needs to be "plugged" in, so that it's optional. The LGPL is a wierd issue, basically Apache takes a stand against LGPL and will not allow any of it's projects to depend on an LGPL dependency. This would force Apache DS to have to fork JDBM to maintain a version without that LGPL dependency. It's not that they think that LGPL is causing any wierd violation, they just don't like some of the ambiguity, and thus fall on the side of caution. > > -b > > ------------------------------------------------------------------------ > *From:* jdbm-general-bounces@... > [mailto:jdbm-general-bounces@...] *On Behalf Of > *Mark Proctor > *Sent:* Monday, June 09, 2008 10:58 AM > *To:* Cees de Groot > *Cc:* jdbm-general@...; cowtowncoder@... > *Subject:* Re: [Jdbm-general] extensible serializer > > Cees de Groot wrote: >> On Mon, Jun 9, 2008 at 4:24 PM, Cees de Groot <cdegroot@...> wrote: >> >>> I grabbed the extensible serializer source code and added it to the >>> source tree - the original project doesn't seem to exist anymore so I >>> thought this was the quickest way to get rid of a binary-only >>> dependency. >>> > Great, I'l update and look over it. For me I just want JDBM to > write my byte[], I don't want it to go anywhere near a > serialistion method call, I'm currently just trying to find out if > that is possible. >> >> On second thought, I agree it's not good to have JDBM depend on a >> single serializer. So I'm removing the dependency (rolling back to the >> pre-extser tag in CVS and checking what happened after that) >> >> >

Re: [Jdbm-general] extensible serializer From: Bryan Thompson <bryan@sy...> - 2008-06-10 17:11 Attachments: Message as HTML      Interesting. I had to do some significant work in order to get jdbm to persist the serializer state. I would suggest that you look at the code more carefully rather than just rolling it back. Assuming that xstream uses a stateful serializer, you are going to want to preserve the integration points. By "stateful" serializer, I mean one that maintains persistent state NOT recorded in the individual serialized records. extser factors out what serializers are declared, the class ids (int's) assigned to each class for which there is a registered serializer, and the corresponding serializer version(s) and puts that all into a persistent record accessed off of one of the named roots for the store. This makes it extremely compact when serializing object graphs. The shared state is all factored out. In order to support that I had to put in a bunch of hooks that you will want to keep around. Another issue with versioned serializers is that they basically have to be inner classes in order to access the various fields (unless you want the overhead of reflection during serialization!). One of the changes that I introduced with the extser integration was transparently versioning for the btree nodes and leaves for stores that choose to enable extser. If that forward versioning is important then you are going to wind up with something that's tightly coupled regardless. If the broader issue is the dependency, then Cees already imported extser and I "authorize" its relicensing under the license for the jdbm project. -bryan _____ From: jdbm-general-bounces@... [mailto:jdbm-general-bounces@...] On Behalf Of Mark Proctor Sent: Tuesday, June 10, 2008 12:54 PM To: Bryan Thompson Cc: 'Cees de Groot'; jdbm-general@...; cowtowncoder@... Subject: Re: [Jdbm-general] extensible serializer Bryan Thompson wrote: Well, easy come, easy go - but you might want to see who's using it before you drop it out. There is zero overhead when extser is not enabled. it's not so much the overhead, its the extra dependency. Even if you don't use it you are forced to include it, because Serialiser extends it. So if we are to use it, it needs to be "plugged" in, so that it's optional. The LGPL is a wierd issue, basically Apache takes a stand against LGPL and will not allow any of it's projects to depend on an LGPL dependency. This would force Apache DS to have to fork JDBM to maintain a version without that LGPL dependency. It's not that they think that LGPL is causing any wierd violation, they just don't like some of the ambiguity, and thus fall on the side of caution. -b _____ From: jdbm-general-bounces@... [mailto:jdbm-general-bounces@...] On Behalf Of Mark Proctor Sent: Monday, June 09, 2008 10:58 AM To: Cees de Groot Cc: jdbm-general@...; cowtowncoder@... Subject: Re: [Jdbm-general] extensible serializer Cees de Groot wrote: On Mon, Jun 9, 2008 at 4:24 PM, Cees de Groot <mailto:cdegroot@...> <cdegroot@...> wrote: I grabbed the extensible serializer source code and added it to the source tree - the original project doesn't seem to exist anymore so I thought this was the quickest way to get rid of a binary-only dependency. Great, I'l update and look over it. For me I just want JDBM to write my byte[], I don't want it to go anywhere near a serialistion method call, I'm currently just trying to find out if that is possible. On second thought, I agree it's not good to have JDBM depend on a single serializer. So I'm removing the dependency (rolling back to the pre-extser tag in CVS and checking what happened after that)

Re: [Jdbm-general] extensible serializer From: Cees de Groot <cdegroot@gm...> - 2008-06-10 18:34 If the license issue is sorted, I'm more than happy to keep it in. The original JDBM interfaces are still there, and I'll just move it to jdbm.extser or similar. Any objections? I mean, upside, no license problems, and through code inclusion no dependencies issues. Sounds like ok to me... On Tue, Jun 10, 2008 at 7:11 PM, Bryan Thompson <bryan@...> wrote: > Interesting. > > I had to do some significant work in order to get jdbm to persist the > serializer state. I would suggest that you look at the code more carefully > rather than just rolling it back. Assuming that xstream uses a stateful > serializer, you are going to want to preserve the integration points. > > By "stateful" serializer, I mean one that maintains persistent state NOT > recorded in the individual serialized records. extser factors out what > serializers are declared, the class ids (int's) assigned to each class for > which there is a registered serializer, and the corresponding serializer > version(s) and puts that all into a persistent record accessed off of one of > the named roots for the store. This makes it extremely compact when > serializing object graphs. The shared state is all factored out. In order > to support that I had to put in a bunch of hooks that you will want to keep > around. > > Another issue with versioned serializers is that they basically have to be > inner classes in order to access the various fields (unless you want the > overhead of reflection during serialization!). One of the changes that I > introduced with the extser integration was transparently versioning for the > btree nodes and leaves for stores that choose to enable extser. If that > forward versioning is important then you are going to wind up with > something that's tightly coupled regardless. > > If the broader issue is the dependency, then Cees already imported extser > and I "authorize" its relicensing under the license for the jdbm project. > > -bryan > > ________________________________ > From: jdbm-general-bounces@... > [mailto:jdbm-general-bounces@...] On Behalf Of Mark > Proctor > Sent: Tuesday, June 10, 2008 12:54 PM > To: Bryan Thompson > Cc: 'Cees de Groot'; jdbm-general@...; > cowtowncoder@... > Subject: Re: [Jdbm-general] extensible serializer > > Bryan Thompson wrote: > > Well, easy come, easy go - but you might want to see who's using it before > you drop it out. > > There is zero overhead when extser is not enabled. > > it's not so much the overhead, its the extra dependency. Even if you don't > use it you are forced to include it, because Serialiser extends it. So if we > are to use it, it needs to be "plugged" in, so that it's optional. > > The LGPL is a wierd issue, basically Apache takes a stand against LGPL and > will not allow any of it's projects to depend on an LGPL dependency. This > would force Apache DS to have to fork JDBM to maintain a version without > that LGPL dependency. It's not that they think that LGPL is causing any > wierd violation, they just don't like some of the ambiguity, and thus fall > on the side of caution. > > > -b > > ________________________________ > From: jdbm-general-bounces@... > [mailto:jdbm-general-bounces@...] On Behalf Of Mark > Proctor > Sent: Monday, June 09, 2008 10:58 AM > To: Cees de Groot > Cc: jdbm-general@...; cowtowncoder@... > Subject: Re: [Jdbm-general] extensible serializer > > Cees de Groot wrote: > > On Mon, Jun 9, 2008 at 4:24 PM, Cees de Groot <cdegroot@...> wrote: > > > I grabbed the extensible serializer source code and added it to the > source tree - the original project doesn't seem to exist anymore so I > thought this was the quickest way to get rid of a binary-only > dependency. > > > Great, I'l update and look over it. For me I just want JDBM to write my > byte[], I don't want it to go anywhere near a serialistion method call, I'm > currently just trying to find out if that is possible. > > On second thought, I agree it's not good to have JDBM depend on a > single serializer. So I'm removing the dependency (rolling back to the > pre-extser tag in CVS and checking what happened after that) > > > > -- "Human beings make life so interesting. Do you know, that in a universe so full of wonders, they have managed to invent boredom. " - Death, in "The Hogfather"

Re: [Jdbm-general] extensible serializer From: Alex Boisvert <boisvert@in...> - 2008-06-10 19:24 Attachments: Message as HTML      Ok with me. alex On Tue, Jun 10, 2008 at 11:34 AM, Cees de Groot <cdegroot@...> wrote: > If the license issue is sorted, I'm more than happy to keep it in. The > original JDBM interfaces are still there, and I'll just move it to > jdbm.extser or similar. > > Any objections? I mean, upside, no license problems, and through code > inclusion no dependencies issues. Sounds like ok to me... > > On Tue, Jun 10, 2008 at 7:11 PM, Bryan Thompson <bryan@...> wrote: > > Interesting. > > > > I had to do some significant work in order to get jdbm to persist the > > serializer state. I would suggest that you look at the code more > carefully > > rather than just rolling it back. Assuming that xstream uses a stateful > > serializer, you are going to want to preserve the integration points. > > > > By "stateful" serializer, I mean one that maintains persistent state NOT > > recorded in the individual serialized records. extser factors out what > > serializers are declared, the class ids (int's) assigned to each class > for > > which there is a registered serializer, and the corresponding serializer > > version(s) and puts that all into a persistent record accessed off of one > of > > the named roots for the store. This makes it extremely compact when > > serializing object graphs. The shared state is all factored out. In > order > > to support that I had to put in a bunch of hooks that you will want to > keep > > around. > > > > Another issue with versioned serializers is that they basically have to > be > > inner classes in order to access the various fields (unless you want the > > overhead of reflection during serialization!). One of the changes that I > > introduced with the extser integration was transparently versioning for > the > > btree nodes and leaves for stores that choose to enable extser. If that > > forward versioning is important then you are going to wind up with > > something that's tightly coupled regardless. > > > > If the broader issue is the dependency, then Cees already imported extser > > and I "authorize" its relicensing under the license for the jdbm project. > > > > -bryan > > > > ________________________________ > > From: jdbm-general-bounces@... > > [mailto:jdbm-general-bounces@...] On Behalf Of Mark > > Proctor > > Sent: Tuesday, June 10, 2008 12:54 PM > > To: Bryan Thompson > > Cc: 'Cees de Groot'; jdbm-general@...; > > cowtowncoder@... > > Subject: Re: [Jdbm-general] extensible serializer > > > > Bryan Thompson wrote: > > > > Well, easy come, easy go - but you might want to see who's using it > before > > you drop it out. > > > > There is zero overhead when extser is not enabled. > > > > it's not so much the overhead, its the extra dependency. Even if you > don't > > use it you are forced to include it, because Serialiser extends it. So if > we > > are to use it, it needs to be "plugged" in, so that it's optional. > > > > The LGPL is a wierd issue, basically Apache takes a stand against LGPL > and > > will not allow any of it's projects to depend on an LGPL dependency. This > > would force Apache DS to have to fork JDBM to maintain a version without > > that LGPL dependency. It's not that they think that LGPL is causing any > > wierd violation, they just don't like some of the ambiguity, and thus > fall > > on the side of caution. > > > > > > -b > > > > ________________________________ > > From: jdbm-general-bounces@... > > [mailto:jdbm-general-bounces@...] On Behalf Of Mark > > Proctor > > Sent: Monday, June 09, 2008 10:58 AM > > To: Cees de Groot > > Cc: jdbm-general@...; cowtowncoder@... > > Subject: Re: [Jdbm-general] extensible serializer > > > > Cees de Groot wrote: > > > > On Mon, Jun 9, 2008 at 4:24 PM, Cees de Groot <cdegroot@...> > wrote: > > > > > > I grabbed the extensible serializer source code and added it to the > > source tree - the original project doesn't seem to exist anymore so I > > thought this was the quickest way to get rid of a binary-only > > dependency. > > > > > > Great, I'l update and look over it. For me I just want JDBM to write my > > byte[], I don't want it to go anywhere near a serialistion method call, > I'm > > currently just trying to find out if that is possible. > > > > On second thought, I agree it's not good to have JDBM depend on a > > single serializer. So I'm removing the dependency (rolling back to the > > pre-extser tag in CVS and checking what happened after that) > > > > > > > > > > > > -- > "Human beings make life so interesting. Do you know, that in a > universe so full of wonders, they have managed to invent boredom. " - > Death, in "The Hogfather" > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Jdbm-general mailing list > Jdbm-general@... > https://lists.sourceforge.net/lists/listinfo/jdbm-general >

Re: [Jdbm-general] extensible serializer From: Cees de Groot <cdegroot@gm...> - 2008-06-11 05:55 If no-one objects, i'll try to bake a 1.0rc1 today that has: - CVS HEAD - The Big ApacheDS Patch[tm] :-) Another issue is package naming - we use "jdbm." atm but we need a real domain name to release on ibiblio. So that either means we shove it under the Apache umbrella fast, or we get our own domain name (vastly in favor of the first option, I'm already the proud owner of more domain names than I want). On Tue, Jun 10, 2008 at 9:24 PM, Alex Boisvert <boisvert@...> wrote: > Ok with me. > > alex > > On Tue, Jun 10, 2008 at 11:34 AM, Cees de Groot <cdegroot@...> wrote: >> >> If the license issue is sorted, I'm more than happy to keep it in. The >> original JDBM interfaces are still there, and I'll just move it to >> jdbm.extser or similar. >> >> Any objections? I mean, upside, no license problems, and through code >> inclusion no dependencies issues. Sounds like ok to me... >> >> On Tue, Jun 10, 2008 at 7:11 PM, Bryan Thompson <bryan@...> wrote: >> > Interesting. >> > >> > I had to do some significant work in order to get jdbm to persist the >> > serializer state. I would suggest that you look at the code more >> > carefully >> > rather than just rolling it back. Assuming that xstream uses a stateful >> > serializer, you are going to want to preserve the integration points. >> > >> > By "stateful" serializer, I mean one that maintains persistent state NOT >> > recorded in the individual serialized records. extser factors out what >> > serializers are declared, the class ids (int's) assigned to each class >> > for >> > which there is a registered serializer, and the corresponding serializer >> > version(s) and puts that all into a persistent record accessed off of >> > one of >> > the named roots for the store. This makes it extremely compact when >> > serializing object graphs. The shared state is all factored out. In >> > order >> > to support that I had to put in a bunch of hooks that you will want to >> > keep >> > around. >> > >> > Another issue with versioned serializers is that they basically have to >> > be >> > inner classes in order to access the various fields (unless you want the >> > overhead of reflection during serialization!). One of the changes that >> > I >> > introduced with the extser integration was transparently versioning for >> > the >> > btree nodes and leaves for stores that choose to enable extser. If that >> > forward versioning is important then you are going to wind up with >> > something that's tightly coupled regardless. >> > >> > If the broader issue is the dependency, then Cees already imported >> > extser >> > and I "authorize" its relicensing under the license for the jdbm >> > project. >> > >> > -bryan >> > >> > ________________________________ >> > From: jdbm-general-bounces@... >> > [mailto:jdbm-general-bounces@...] On Behalf Of Mark >> > Proctor >> > Sent: Tuesday, June 10, 2008 12:54 PM >> > To: Bryan Thompson >> > Cc: 'Cees de Groot'; jdbm-general@...; >> > cowtowncoder@... >> > Subject: Re: [Jdbm-general] extensible serializer >> > >> > Bryan Thompson wrote: >> > >> > Well, easy come, easy go - but you might want to see who's using it >> > before >> > you drop it out. >> > >> > There is zero overhead when extser is not enabled. >> > >> > it's not so much the overhead, its the extra dependency. Even if you >> > don't >> > use it you are forced to include it, because Serialiser extends it. So >> > if we >> > are to use it, it needs to be "plugged" in, so that it's optional. >> > >> > The LGPL is a wierd issue, basically Apache takes a stand against LGPL >> > and >> > will not allow any of it's projects to depend on an LGPL dependency. >> > This >> > would force Apache DS to have to fork JDBM to maintain a version without >> > that LGPL dependency. It's not that they think that LGPL is causing any >> > wierd violation, they just don't like some of the ambiguity, and thus >> > fall >> > on the side of caution. >> > >> > >> > -b >> > >> > ________________________________ >> > From: jdbm-general-bounces@... >> > [mailto:jdbm-general-bounces@...] On Behalf Of Mark >> > Proctor >> > Sent: Monday, June 09, 2008 10:58 AM >> > To: Cees de Groot >> > Cc: jdbm-general@...; cowtowncoder@... >> > Subject: Re: [Jdbm-general] extensible serializer >> > >> > Cees de Groot wrote: >> > >> > On Mon, Jun 9, 2008 at 4:24 PM, Cees de Groot <cdegroot@...> >> > wrote: >> > >> > >> > I grabbed the extensible serializer source code and added it to the >> > source tree - the original project doesn't seem to exist anymore so I >> > thought this was the quickest way to get rid of a binary-only >> > dependency. >> > >> > >> > Great, I'l update and look over it. For me I just want JDBM to write my >> > byte[], I don't want it to go anywhere near a serialistion method call, >> > I'm >> > currently just trying to find out if that is possible. >> > >> > On second thought, I agree it's not good to have JDBM depend on a >> > single serializer. So I'm removing the dependency (rolling back to the >> > pre-extser tag in CVS and checking what happened after that) >> > >> > >> > >> > >> >> >> >> -- >> "Human beings make life so interesting. Do you know, that in a >> universe so full of wonders, they have managed to invent boredom. " - >> Death, in "The Hogfather" >> >> ------------------------------------------------------------------------- >> Check out the new SourceForge.net Marketplace. >> It's the best place to buy or sell services for >> just about anything Open Source. >> http://sourceforge.net/services/buy/index.php >> _______________________________________________ >> Jdbm-general mailing list >> Jdbm-general@... >> https://lists.sourceforge.net/lists/listinfo/jdbm-general > > -- "Human beings make life so interesting. Do you know, that in a universe so full of wonders, they have managed to invent boredom. " - Death, in "The Hogfather"

Re: [Jdbm-general] extensible serializer From: Tatu Saloranta <cowtowncoder@ya...> - 2008-06-12 04:52 --- On Tue, 6/10/08, Cees de Groot <cdegroot@...> wrote: > From: Cees de Groot <cdegroot@...> > Subject: Re: [Jdbm-general] extensible serializer > To: "Bryan Thompson" <bryan@...> > Cc: "Mark Proctor" <mproctor@...>, jdbm-general@..., cowtowncoder@... > Date: Tuesday, June 10, 2008, 12:34 PM > If the license issue is sorted, I'm more than happy to > keep it in. The > original JDBM interfaces are still there, and I'll just > move it to > jdbm.extser or similar. > > Any objections? I mean, upside, no license problems, and > through code > inclusion no dependencies issues. Sounds like ok to me... As long as number of classes is limited and they are on different package, sounds good to me. It should be possible to then also compile "core" jdbm without these parts, but default version can include them? -+ Tatu +-

Re: [Jdbm-general] extensible serializer From: Tatu Saloranta <cowtowncoder@ya...> - 2008-06-12 05:00 --- On Mon, 6/9/08, Cees de Groot <cdegroot@...> wrote: ... > As I said, I'm more than happy to move over (and as I use the > Atlassian toolchain at work as well, so much the better > :-)). I assume this has already started, so this is irrelevant, but I think it is a code idea to move to Codehaus & get artifacts more easily integratable via Maven repo. -+ Tatu +-

Re: [Jdbm-general] extensible serializer From: Cees de Groot <cdegroot@gm...> - 2008-06-12 05:56 Yup. I wanted to create a release candidate yesterday, but not all unit tests ran. Maven (well, Surefire) automagically creates a test suite so it probably is including some tests that jdbm.AllTests is skipping. I'm going to use my train commute today to hash this out. Whether we move to Codehaus or find shelter with an Apache project is something still to be decided :) On Thu, Jun 12, 2008 at 6:59 AM, Tatu Saloranta <cowtowncoder@...> wrote: > > --- On Mon, 6/9/08, Cees de Groot <cdegroot@...> wrote: > > ... >> As I said, I'm more than happy to move over (and as I use the >> Atlassian toolchain at work as well, so much the better >> :-)). > > I assume this has already started, so this is irrelevant, but I think it is a code idea to move to Codehaus & get artifacts more easily integratable via Maven repo. > > -+ Tatu +- > > > > > > -- "Human beings make life so interesting. Do you know, that in a universe so full of wonders, they have managed to invent boredom. " - Death, in "The Hogfather"

Re: [Jdbm-general] extensible serializer From: Kevin Day <kevin@tr...> - 2008-06-16 21:48 Cees- I have no objections. I actually did a pretty thorough code review of Bryan's work and found his changes to be very well though through. Haven't used it in projection yet, but probably will. Also, as he points out, it's easy to drop back to the old serialization mechanism (or even the *shudder* Java serialzation mechanism). It would be nice to be able to factor things so alternative serializers could be specified, but as Bryan points out, a *lot* of tweaks and hooks were required to add this one other serialization mechanism. I suspect that a serious refactoring of the entire jdbm codebase would be required to make it truly pluggable (and there are, I believe, more desirable goals for future development - like true transaction isolation and rollback support). - K Kevin Day Trumpet, Inc. http://www.trumpetinc.com kevin@... 480-961-6003 x1002 ----------------------- Original Message ----------------------- From: "Cees de Groot" <cdegroot@...> To: "Bryan Thompson" <bryan@...> Cc: Mark Proctor <mproctor@...>, jdbm-general@..., cowtowncoder@... Date: Tue, 10 Jun 2008 20:34:25 +0200 Subject: Re: [Jdbm-general] extensible serializer If the license issue is sorted, I'm more than happy to keep it in. The original JDBM interfaces are still there, and I'll just move it to jdbm.extser or similar. Any objections? I mean, upside, no license problems, and through code inclusion no dependencies issues. Sounds like ok to me... On Tue, Jun 10, 2008 at 7:11 PM, Bryan Thompson <bryan@...> wrote: > Interesting. > > I had to do some significant work in order to get jdbm to persist the > serializer state. I would suggest that you look at the code more carefully > rather than just rolling it back. Assuming that xstream uses a stateful > serializer, you are going to want to preserve the integration points. > > By "stateful" serializer, I mean one that maintains persistent state NOT > recorded in the individual serialized records. extser factors out what > serializers are declared, the class ids (int's) assigned to each class for > which there is a registered serializer, and the corresponding serializer > version(s) and puts that all into a persistent record accessed off of one of > the named roots for the store. This makes it extremely compact when > serializing object graphs. The shared state is all factored out. In order > to support that I had to put in a bunch of hooks that you will want to keep > around. > > Another issue with versioned serializers is that they basically have to be > inner classes in order to access the various fields (unless you want the > overhead of reflection during serialization!). One of the changes that I > introduced with the extser integration was transparently versioning for the > btree nodes and leaves for stores that choose to enable extser. If that > forward versioning is important then you are going to wind up with > something that's tightly coupled regardless. > > If the broader issue is the dependency, then Cees already imported extser > and I "authorize" its relicensing under the license for the jdbm project. > > -bryan > > ________________________________ > From: jdbm-general-bounces@... > [mailto:jdbm-general-bounces@...] On Behalf Of Mark > Proctor > Sent: Tuesday, June 10, 2008 12:54 PM > To: Bryan Thompson > Cc: 'Cees de Groot'; jdbm-general@...; > cowtowncoder@... > Subject: Re: [Jdbm-general] extensible serializer > > Bryan Thompson wrote: > > Well, easy come, easy go - but you might want to see who's using it before > you drop it out. > > There is zero overhead when extser is not enabled. > > it's not so much the overhead, its the extra dependency. Even if you don't > use it you are forced to include it, because Serialiser extends it. So if we > are to use it, it needs to be "plugged" in, so that it's optional. > > The LGPL is a wierd issue, basically Apache takes a stand against LGPL and > will not allow any of it's projects to depend on an LGPL dependency. This > would force Apache DS to have to fork JDBM to maintain a version without > that LGPL dependency. It's not that they think that LGPL is causing any > wierd violation, they just don't like some of the ambiguity, and thus fall > on the side of caution. > > > -b > > ________________________________ > From: jdbm-general-bounces@... > [mailto:jdbm-general-bounces@...] On Behalf Of Mark > Proctor > Sent: Monday, June 09, 2008 10:58 AM > To: Cees de Groot > Cc: jdbm-general@...; cowtowncoder@... > Subject: Re: [Jdbm-general] extensible serializer > > Cees de Groot wrote: > > On Mon, Jun 9, 2008 at 4:24 PM, Cees de Groot <cdegroot@...> wrote: > > > I grabbed the extensible serializer source code and added it to the > source tree - the original project doesn't seem to exist anymore so I > thought this was the quickest way to get rid of a binary-only > dependency. > > > Great, I'l update and look over it. For me I just want JDBM to write my > byte[], I don't want it to go anywhere near a serialistion method call, I'm > currently just trying to find out if that is possible. > > On second thought, I agree it's not good to have JDBM depend on a > single serializer. So I'm removing the dependency (rolling back to the > pre-extser tag in CVS and checking what happened after that) > > > > -- "Human beings make life so interesting. Do you know, that in a universe so full of wonders, they have managed to invent boredom. " - Death, in "The Hogfather" ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php _______________________________________________ Jdbm-general mailing list Jdbm-general@... https://lists.sourceforge.net/lists/listinfo/jdbm-general

Re: [Jdbm-general] extensible serializer From: Mark Proctor <mproctor@co...> - 2008-06-17 07:03 Attachments: Message as HTML      Kevin Day wrote: > Cees- > > I have no objections. I actually did a pretty thorough code review of Bryan's work and found his changes to be very well though through. Haven't used it in projection yet, but probably will. Also, as he points out, it's easy to drop back to the old serialization mechanism (or even the *shudder* Java serialzation mechanism). > for me I just want to be able to efficiently write my own byte[], without having to go through any serialisation mechanism. When I last looked the Serializer interface allowed this, so should be fine. > It would be nice to be able to factor things so alternative serializers could be specified, but as Bryan points out, a *lot* of tweaks and hooks were required to add this one other serialization mechanism. I suspect that a serious refactoring of the entire jdbm codebase would be required to make it truly pluggable (and there are, I believe, more desirable goals for future development - like true transaction isolation and rollback support). > Getting JDBM to play in JTA transactions is important, someone said in an old posting they had created some code for this and submitted it - anyone know what happened to that? Interestingly the other day I was talking to someone who built a key/value store for journal logging, they use to use BDB, but moved away sa the btree was overkill. They built a system that created 10 x 10mb files (configurable). You would insert the byte[] and it would return a long handle, the byte[] would be written to the head location, which currently pointed to a specific file, if there wasn't enough space it would point to the next file, they do not allow spanning and that free space would not be written to later as they do not seek, they only ever write to the head location. Whlie you can remove entries the indivual entry filespace is not reclaimed, only if all entries for that file are marked as removed would it delete the entire file. They claim this approach gives blistering fast speed, as there is no seek at write time, and only seek once at read time to find the start position, because they do not fragment their files to fit in gaps. Most journalling systems have entries only possibly numbering in their hundreds and won't exist forever, so you get something that is less efficient with space but much faster. I was wondering if the RecordManager in JDBM could be extended to do something similar, as another possible backend? Talking of which would it be possible for someone to write some docs on how RecordManager and JDBM works in general? > - K > > Kevin Day > Trumpet, Inc. > http://www.trumpetinc.com > kevin@... > 480-961-6003 x1002 > > > ----------------------- Original Message ----------------------- > > From: "Cees de Groot" <cdegroot@...> > To: "Bryan Thompson" <bryan@...> > Cc: Mark Proctor <mproctor@...>, jdbm-general@..., cowtowncoder@... > Date: Tue, 10 Jun 2008 20:34:25 +0200 > Subject: Re: [Jdbm-general] extensible serializer > > If the license issue is sorted, I'm more than happy to keep it in. The > original JDBM interfaces are still there, and I'll just move it to > jdbm.extser or similar. > > Any objections? I mean, upside, no license problems, and through code > inclusion no dependencies issues. Sounds like ok to me... > > On Tue, Jun 10, 2008 at 7:11 PM, Bryan Thompson <bryan@...> wrote: > >> Interesting. >> >> I had to do some significant work in order to get jdbm to persist the >> serializer state. I would suggest that you look at the code more carefully >> rather than just rolling it back. Assuming that xstream uses a stateful >> serializer, you are going to want to preserve the integration points. >> >> By "stateful" serializer, I mean one that maintains persistent state NOT >> recorded in the individual serialized records. extser factors out what >> serializers are declared, the class ids (int's) assigned to each class for >> which there is a registered serializer, and the corresponding serializer >> version(s) and puts that all into a persistent record accessed off of one of >> the named roots for the store. This makes it extremely compact when >> serializing object graphs. The shared state is all factored out. In order >> to support that I had to put in a bunch of hooks that you will want to keep >> around. >> >> Another issue with versioned serializers is that they basically have to be >> inner classes in order to access the various fields (unless you want the >> overhead of reflection during serialization!). One of the changes that I >> introduced with the extser integration was transparently versioning for the >> btree nodes and leaves for stores that choose to enable extser. If that >> forward versioning is important then you are going to wind up with >> something that's tightly coupled regardless. >> >> If the broader issue is the dependency, then Cees already imported extser >> and I "authorize" its relicensing under the license for the jdbm project. >> >> -bryan >> >> ________________________________ >> From: jdbm-general-bounces@... >> [mailto:jdbm-general-bounces@...] On Behalf Of Mark >> Proctor >> Sent: Tuesday, June 10, 2008 12:54 PM >> To: Bryan Thompson >> Cc: 'Cees de Groot'; jdbm-general@...; >> cowtowncoder@... >> Subject: Re: [Jdbm-general] extensible serializer >> >> Bryan Thompson wrote: >> >> Well, easy come, easy go - but you might want to see who's using it before >> you drop it out. >> >> There is zero overhead when extser is not enabled. >> >> it's not so much the overhead, its the extra dependency. Even if you don't >> use it you are forced to include it, because Serialiser extends it. So if we >> are to use it, it needs to be "plugged" in, so that it's optional. >> >> The LGPL is a wierd issue, basically Apache takes a stand against LGPL and >> will not allow any of it's projects to depend on an LGPL dependency. This >> would force Apache DS to have to fork JDBM to maintain a version without >> that LGPL dependency. It's not that they think that LGPL is causing any >> wierd violation, they just don't like some of the ambiguity, and thus fall >> on the side of caution. >> >> >> -b >> >> ________________________________ >> From: jdbm-general-bounces@... >> [mailto:jdbm-general-bounces@...] On Behalf Of Mark >> Proctor >> Sent: Monday, June 09, 2008 10:58 AM >> To: Cees de Groot >> Cc: jdbm-general@...; cowtowncoder@... >> Subject: Re: [Jdbm-general] extensible serializer >> >> Cees de Groot wrote: >> >> On Mon, Jun 9, 2008 at 4:24 PM, Cees de Groot <cdegroot@...> wrote: >> >> >> I grabbed the extensible serializer source code and added it to the >> source tree - the original project doesn't seem to exist anymore so I >> thought this was the quickest way to get rid of a binary-only >> dependency. >> >> >> Great, I'l update and look over it. For me I just want JDBM to write my >> byte[], I don't want it to go anywhere near a serialistion method call, I'm >> currently just trying to find out if that is possible. >> >> On second thought, I agree it's not good to have JDBM depend on a >> single serializer. So I'm removing the dependency (rolling back to the >> pre-extser tag in CVS and checking what happened after that) >> >> >> >> >> > > > >

[Jdbm-general] how JDBM works (was extensible serializer) From: Mark Proctor <mproctor@co...> - 2008-06-17 18:02 Attachments: Message as HTML      Mark Proctor wrote: > Kevin Day wrote: >> Cees- >> >> I have no objections. I actually did a pretty thorough code review of Bryan's work and found his changes to be very well though through. Haven't used it in projection yet, but probably will. Also, as he points out, it's easy to drop back to the old serialization mechanism (or even the *shudder* Java serialzation mechanism). >> > for me I just want to be able to efficiently write my own byte[], > without having to go through any serialisation mechanism. When I last > looked the Serializer interface allowed this, so should be fine. >> It would be nice to be able to factor things so alternative serializers could be specified, but as Bryan points out, a *lot* of tweaks and hooks were required to add this one other serialization mechanism. I suspect that a serious refactoring of the entire jdbm codebase would be required to make it truly pluggable (and there are, I believe, more desirable goals for future development - like true transaction isolation and rollback support). >> > Getting JDBM to play in JTA transactions is important, someone said in > an old posting they had created some code for this and submitted it - > anyone know what happened to that? > > Interestingly the other day I was talking to someone who built a > key/value store for journal logging, they use to use BDB, but moved > away sa the btree was overkill. They built a system that created 10 x > 10mb files (configurable). You would insert the byte[] and it would > return a long handle, the byte[] would be written to the head > location, which currently pointed to a specific file, if there wasn't > enough space it would point to the next file, they do not allow > spanning and that free space would not be written to later as they do > not seek, they only ever write to the head location. Whlie you can > remove entries the indivual entry filespace is not reclaimed, only if > all entries for that file are marked as removed would it delete the > entire file. They claim this approach gives blistering fast speed, as > there is no seek at write time, and only seek once at read time to > find the start position, because they do not fragment their files to > fit in gaps. Most journalling systems have entries only possibly > numbering in their hundreds and won't exist forever, so you get > something that is less efficient with space but much faster. I was > wondering if the RecordManager in JDBM could be extended to do > something similar, as another possible backend? > > Talking of which would it be possible for someone to write some docs > on how RecordManager and JDBM works in general? Chatting to my collegue about his approach, his stuff is here http://anonsvn.jboss.org/repos/messaging/trunk/src/main/org/jboss/messaging/core/journal/. The files are made small enough to fit into a disk cylinder and each one is append only, this maximises throughput. This is geared up more for journalling though, but it does do long/value add, remove and update. They have record types, so the TX info can be appended to the same log file, to avoid the seek between the .db and .log. Anyway thought the idea of a single appending log only idea might be a good "optimisation" backend for JDBM. I've been going through the JDBM code and it's quite well written, so I'm able to understand it. Interesting I can see that RecordManager can be used directly without BTree, for a basic store using a long key, so thats interesting. There seems to be no repeated disk seeking on write, as the location is determined in memory and then it's a continuous write. A delete again is an in memory lookup and a single write to mark the location free, it doesn't have to actually free all bytes on the disk. This looks pretty optimial to me. Buffering is optional, out of interest when does this pay off? I can't imagine the logic and physical mapping has any measurable cost. Down side is that byte[] size needs to be known ahead of time so async streaming for writes won't work. I saw that in the docs it mentions replacing some of the code with DBCache, but I couldn't find the mailing list discussion on this - any details? There is obviously the issue of multithreading. With the move to JDK1.5, does that help some? JDBM should probably atleast allow multiple reads, regardless of write, sorta like concurrent hashmap. I'm guessing at this point it might be preferable to split up dbs, to allow concurrent writes too, via striping? One of the use cases I'd like to support is the idea of in memory meta-data for each long, without having to iterate over the entire .db reading in all files. I will probably do this as a second db, that holds only meta data - although then I need to get transactions to span across both dbs. This would hold the long id to the read data. I'd then do a permanent in memory cache of data, as we'd only ever have a few hundred items anyway. I'm wondering if the idea of "meta" data for records could be built into the main .db. And it should be possible to startup a record manager and load in all meta data. This way the meta data and record can be written continously together. Mark >> - K >> >> Kevin Day >> Trumpet, Inc. >> http://www.trumpetinc.com >> kevin@... >> 480-961-6003 x1002 >> >> >> ----------------------- Original Message ----------------------- >> >> From: "Cees de Groot" <cdegroot@...> >> To: "Bryan Thompson" <bryan@...> >> Cc: Mark Proctor <mproctor@...>, jdbm-general@..., cowtowncoder@... >> Date: Tue, 10 Jun 2008 20:34:25 +0200 >> Subject: Re: [Jdbm-general] extensible serializer >> >> If the license issue is sorted, I'm more than happy to keep it in. The >> original JDBM interfaces are still there, and I'll just move it to >> jdbm.extser or similar. >> >> Any objections? I mean, upside, no license problems, and through code >> inclusion no dependencies issues. Sounds like ok to me... >> >> On Tue, Jun 10, 2008 at 7:11 PM, Bryan Thompson <bryan@...> wrote: >> >>> Interesting. >>> >>> I had to do some significant work in order to get jdbm to persist the >>> serializer state. I would suggest that you look at the code more carefully >>> rather than just rolling it back. Assuming that xstream uses a stateful >>> serializer, you are going to want to preserve the integration points. >>> >>> By "stateful" serializer, I mean one that maintains persistent state NOT >>> recorded in the individual serialized records. extser factors out what >>> serializers are declared, the class ids (int's) assigned to each class for >>> which there is a registered serializer, and the corresponding serializer >>> version(s) and puts that all into a persistent record accessed off of one of >>> the named roots for the store. This makes it extremely compact when >>> serializing object graphs. The shared state is all factored out. In order >>> to support that I had to put in a bunch of hooks that you will want to keep >>> around. >>> >>> Another issue with versioned serializers is that they basically have to be >>> inner classes in order to access the various fields (unless you want the >>> overhead of reflection during serialization!). One of the changes that I >>> introduced with the extser integration was transparently versioning for the >>> btree nodes and leaves for stores that choose to enable extser. If that >>> forward versioning is important then you are going to wind up with >>> something that's tightly coupled regardless. >>> >>> If the broader issue is the dependency, then Cees already imported extser >>> and I "authorize" its relicensing under the license for the jdbm project. >>> >>> -bryan >>> >>> ________________________________ >>> From: jdbm-general-bounces@... >>> [mailto:jdbm-general-bounces@...] On Behalf Of Mark >>> Proctor >>> Sent: Tuesday, June 10, 2008 12:54 PM >>> To: Bryan Thompson >>> Cc: 'Cees de Groot'; jdbm-general@...; >>> cowtowncoder@... >>> Subject: Re: [Jdbm-general] extensible serializer >>> >>> Bryan Thompson wrote: >>> >>> Well, easy come, easy go - but you might want to see who's using it before >>> you drop it out. >>> >>> There is zero overhead when extser is not enabled. >>> >>> it's not so much the overhead, its the extra dependency. Even if you don't >>> use it you are forced to include it, because Serialiser extends it. So if we >>> are to use it, it needs to be "plugged" in, so that it's optional. >>> >>> The LGPL is a wierd issue, basically Apache takes a stand against LGPL and >>> will not allow any of it's projects to depend on an LGPL dependency. This >>> would force Apache DS to have to fork JDBM to maintain a version without >>> that LGPL dependency. It's not that they think that LGPL is causing any >>> wierd violation, they just don't like some of the ambiguity, and thus fall >>> on the side of caution. >>> >>> >>> -b >>> >>> ________________________________ >>> From: jdbm-general-bounces@... >>> [mailto:jdbm-general-bounces@...] On Behalf Of Mark >>> Proctor >>> Sent: Monday, June 09, 2008 10:58 AM >>> To: Cees de Groot >>> Cc: jdbm-general@...; cowtowncoder@... >>> Subject: Re: [Jdbm-general] extensible serializer >>> >>> Cees de Groot wrote: >>> >>> On Mon, Jun 9, 2008 at 4:24 PM, Cees de Groot <cdegroot@...> wrote: >>> >>> >>> I grabbed the extensible serializer source code and added it to the >>> source tree - the original project doesn't seem to exist anymore so I >>> thought this was the quickest way to get rid of a binary-only >>> dependency. >>> >>> >>> Great, I'l update and look over it. For me I just want JDBM to write my >>> byte[], I don't want it to go anywhere near a serialistion method call, I'm >>> currently just trying to find out if that is possible. >>> >>> On second thought, I agree it's not good to have JDBM depend on a >>> single serializer. So I'm removing the dependency (rolling back to the >>> pre-extser tag in CVS and checking what happened after that) >>> >>> >>> >>> >>> >> >> >> >> > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > ------------------------------------------------------------------------ > > _______________________________________________ > Jdbm-general mailing list > Jdbm-general@... > https://lists.sourceforge.net/lists/listinfo/jdbm-general >

Re: [Jdbm-general] extensible serializer From: Tatu Saloranta <cowtowncoder@ya...> - 2008-06-17 18:43 --- On Tue, 6/17/08, Mark Proctor <mproctor@...> wrote: ... > for me I just want to be able to efficiently write my own > byte[], > without having to go through any serialisation mechanism. Hear hear. That's kind of my view too -- I am not against layering additional stuff on top of this low-level layer, but for my use cases they are not essential. Conversely, when layer are defined properly (and hooks are in place -- which was an important part of the patches) it should be possible to add more advanced but still integrated object persistence systems. ... > > Interestingly the other day I was talking to someone who > built a > key/value store for journal logging, they use to use BDB, > but moved away > sa the btree was overkill. They built a system that created > 10 x 10mb > files (configurable). You would insert the byte[] and it > would return a ... > they only ever > write to the head location. Whlie you can remove entries Isn't this similar to what BDB-JE does too? (not the traditional BDB). Basically allowing fast writers, good concurrency, but quite a bit more storage space overhead. -+ Tatu +-

Re: [Jdbm-general] extensible serializer From: Kevin Day <kevin@tr...> - 2008-06-17 19:03 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> <HTML><HEAD> <STYLE type=text/css> P, UL, OL, DL, DIR, MENU, PRE { margin: 0 auto;}</STYLE> <META content="MSHTML 6.00.2900.3059" name=GENERATOR></HEAD> <BODY leftMargin=1 topMargin=1 rightMargin=1><FONT face=Arial size=2> <DIV>I think something that may be getting missed here is that the BTree implementation is, itself, a higher level construct (which serializes itself into a byte array that is then stored in the record manager).&nbsp; Simple storage of byte arrays is supported directly in record manager (that's pretty much the point of RecordManager).</DIV> <DIV>&nbsp;</DIV> <DIV>If you wish to store into a BTree, then serialization is required (the tree has to have *some* mechanism for serializing itself, after all - so a nested serialization is called for on both the key and value side of the tuple).&nbsp; If all you care about is storing byte arrays, then use the provided ByteArraySerializer.INSTANCE object when you initialize the BTree, and you are done.</DIV> <DIV>&nbsp;</DIV> <DIV>&nbsp;</DIV> <DIV>FYI - If your keys are relatively well ordered, and you are doing a lot of storage, I definitely recommend that you structure your key byte arrays so the common values are left oriented, then turn on the the key compressor feature in the BTree.&nbsp; This significantly reduces disk usage and in most cases improves speed considerably.</DIV> <DIV>&nbsp;</DIV> <DIV>- K</DIV> <DIV>&nbsp;</DIV> <DIV>&nbsp;</DIV> <DIV>&nbsp;</DIV> <DIV>&nbsp;</DIV></FONT> <DIV style="FONT-SIZE: x-small; FONT-FAMILY: Tahoma"> <DIV>----------------------- <B>Original Message</B> -----------------------</DIV> <DIV>&nbsp;&nbsp;</DIV> <DIV><B>From:</B>&nbsp;Tatu Saloranta <A href="mailto:cowtowncoder@... color=#0000ff>&lt;cowtowncoder@...> <DIV><B>To:</B>&nbsp;<A href="mailto:jdbm-general@... color=#0000ff>"jdbm-general@...> &gt;&gt; JDBM General listserv" <A href="mailto:jdbm-general@... color=#0000ff>&lt;jdbm-general@...> Mark Proctor <A href="mailto:mproctor@... color=#0000ff>&lt;mproctor@...> <DIV><B>Cc:</B>&nbsp;</DIV> <DIV><B>Date:</B>&nbsp;Tue, 17 Jun 2008 11:42:59 -0700 (PDT)</DIV> <DIV><B>Subject:&nbsp;<U>Re: [Jdbm-general] extensible serializer</U></B></DIV> <DIV>&nbsp;&nbsp;</DIV></DIV><FONT face=Tahoma size=2> <DIV>--- On Tue, 6/17/08, Mark Proctor <A href="mailto:mproctor@... color=#0000ff>&lt;mproctor@...> wrote:<BR><BR>...<BR>&gt; for me I just want to be able to efficiently write my own<BR>&gt; byte[], <BR>&gt; without having to go through any serialisation mechanism.<BR><BR>Hear hear. That's kind of my view too -- I am not against layering<BR>additional stuff on top of this low-level layer, but for my use<BR>cases they are not essential. Conversely, when layer are defined<BR>properly (and hooks are in place -- which was an important part of<BR>the patches) it should be possible to add more advanced but still<BR>integrated object persistence systems.<BR><BR>...<BR>&gt; <BR>&gt; Interestingly the other day I was talking to someone who<BR>&gt; built a <BR>&gt; key/value store for journal logging, they use to use BDB,<BR>&gt; but moved away <BR>&gt; sa the btree was overkill. They built a system that created<BR>&gt; 10 x 10mb <BR>&gt; files ( configurable). You would insert the byte[] and it<BR>&gt; would return a <BR>...<BR>&gt; they only ever <BR>&gt; write to the head location. Whlie you can remove entries<BR><BR>Isn't this similar to what BDB-JE does too? (not the traditional BDB).<BR>Basically allowing fast writers, good concurrency, but quite a bit more storage space overhead.<BR><BR>-+ Tatu +-<BR><BR><BR><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<BR><BR>-------------------------------------------------------------------------<BR>Check out the new SourceForge.net Marketplace.<BR>It's the best place to buy or sell services for<BR>just about anything Open Source.<BR><A href="http://sourceforge.net/services/buy/index.php"><FONT color=#0000ff>http://sourceforge.net/services/buy/index.php</FONT></A><BR>_______________________________________________<BR>Jdbm-general mailing list<BR><A href="mailto:Jdbm-general@... color=#0000ff>Jdbm-general@... href="https://lis ts.sourceforge.net/lists/listinfo/jdbm-general"><FONT color=#0000ff>https://lists.sourceforge.net/lists/listinfo/jdbm-general</FONT></A><BR><BR></DIV></FONT></BODY></HTML>

Re: [Jdbm-general] how JDBM works (was extensible serializer) From: Kevin Day <kevin@tr...> - 2008-06-17 19:11 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> <HTML><HEAD> <META http-equiv=Content-Type content=text/html;charset=ISO-8859-1> <META content="MSHTML 6.00.2900.3059" name=GENERATOR></HEAD> <BODY text=#000000 bgColor=#ffffff leftMargin=1 topMargin=1 rightMargin=1><FONT face=Arial size=2> <DIV>There were discussions awhile back about adding another container type object to jdbm - I can't remember what Bryan called them -&nbsp;but the idea was to provide list storage using a linked list of paged elements.&nbsp; It's basically a degenerate B+Tree.&nbsp; This kind of thing would be fairly simple to implement, and would be similar to what you associate described (with the exception that all of the pages of the list would be in a single jdbm database file).</DIV> <DIV>&nbsp;</DIV> <DIV>The idea of using linear, write-only strategies for journaling was much discussed (and is still, in my opinion, a very valid option) for some future jdbm log file implementation.&nbsp; I believe that there are a ton of emails about this in the developers listserv history.&nbsp; I know that Bryan had some fascinating research articles on the subject.</DIV> <DIV>&nbsp;</DIV> <DIV>&nbsp;</DIV> <DIV>In terms of meta data, in one of our apps, we do capture in-memory meta data models of records as the records are read from disk.&nbsp; This is done in a layer above jdbm.&nbsp; In our case, meta data is constructed entirely from the record contents, so there's no need to store it, per se.&nbsp; We actually use BTrees to capture indexes of records stored directly into the record manager, so the meta data is necessary for updating the indexes when records change.</DIV> <DIV>&nbsp;</DIV> <DIV>- K<BR></DIV> <DIV>&nbsp;</DIV></FONT> <DIV style="FONT-SIZE: x-small; FONT-FAMILY: Tahoma"> <DIV>----------------------- <B>Original Message</B> -----------------------</DIV> <DIV>&nbsp;&nbsp;</DIV> <DIV><B>From:</B>&nbsp;Mark Proctor <A href="mailto:mproctor@... color=#0000ff>&lt;mproctor@...> <DIV><B>To:</B>&nbsp;<A href="mailto:jdbm-general@... color=#0000ff>"jdbm-general@...> &gt;&gt; JDBM General listserv" <A href="mailto:jdbm-general@... color=#0000ff>&lt;jdbm-general@...> <DIV><B>Cc:</B>&nbsp;</DIV> <DIV><B>Date:</B>&nbsp;Tue, 17 Jun 2008 19:01:25 +0100</DIV> <DIV><B>Subject:&nbsp;<U>[Jdbm-general] how JDBM works (was extensible serializer)</U></B></DIV> <DIV>&nbsp;&nbsp;</DIV></DIV>Mark Proctor wrote: <BLOCKQUOTE cite=mid:4857619A.2070105@... type="cite">Kevin Day wrote: <BLOCKQUOTE cite=mid:RERPQVMzRShWVmBVJi0+MjA3MjA0OTQ4NA@... type="cite"><PRE wrap="">Cees- I have no objections. I actually did a pretty thorough code review of Bryan's work and found his changes to be very well though through. Haven't used it in projection yet, but probably will. Also, as he points out, it's easy to drop back to the old serialization mechanism (or even the *shudder* Java serialzation mechanism). </PRE></BLOCKQUOTE>for me I just want to be able to efficiently write my own byte[], without having to go through any serialisation mechanism. When I last looked the Serializer interface allowed this, so should be fine. <BR> <BLOCKQUOTE cite=mid:RERPQVMzRShWVmBVJi0+MjA3MjA0OTQ4NA@... type="cite"><PRE wrap="">It would be nice to be able to factor things so alternative serializers could be specified, but as Bryan points out, a *lot* of tweaks and hooks were required to add this one other serialization mechanism. I suspect that a serious refactoring of the entire jdbm codebase would be required to make it truly pluggable (and there are, I believe, more desirable goals for future development - like true transaction isolation and rollback support). </PRE></BLOCKQUOTE>Getting JDBM to play in JTA transactions is important, someone said in an old posting they had created some code for this and submitted it - anyone know what happened to that?<BR><BR>Interestingly the other day I was talking to someone who built a key/value store for journal logging, they use to use BDB, but moved away sa the btree was overkill. They built a system that created 10 x 10mb files (configurable). You would insert the byte[] and it would return a long handle, the byte[] would be written to the head location, which currently pointed to a specific file, if there wasn't enough space it would point to the next file, they do not allow spanning and that free space would not be written to later as they do not seek, they only ever write to the head location. Whlie you can remove entries the indivual entry filespace is not reclaimed, only if all entries for that file are marked as removed would it delete the entire file. They claim this approach gives blistering fast speed, as there is no seek at write time, and only seek once at read time to find the start position, because they do not fragment their files to fit in gaps. Most journalling systems have entries only possibly numbering in their hundreds and won't exist forever, so you get something that is less efficient with space but much faster. I was wondering if the RecordManager in JDBM could be extended to do something similar, as another possible backend?<BR><BR>Talking of which would it be possible for someone to write some docs on how RecordManager and JDBM works in general?<BR></BLOCKQUOTE>Chatting to my collegue about his approach, his stuff is here <A class=moz-txt-link-freetext href="http://anonsvn.jboss.org/repos/messaging/trunk/src/main/org/jboss/messaging/core/journal/">http://anonsvn.jboss.org/repos/messaging/trunk/src/main/org/jboss/messaging/core/journal/</A>. The files are made small enough to fit into a disk cylinder and each one is append only, this m aximises throughput. This is geared up more for journalling though, but it does do long/value add, remove and update. They have record types, so the TX info can be appended to the same log file, to avoid the seek between the .db and .log. Anyway thought the idea of a single appending log only idea might be a good "optimisation" backend for JDBM.<BR><BR>I've been going through the JDBM code and it's quite well written, so I'm able to understand it. Interesting I can see that RecordManager can be used directly without BTree, for a basic store using a long key, so thats interesting. There seems to be no repeated disk seeking on write, as the location is determined in memory and then it's a continuous write. A delete again is an in memory lookup and a single write to mark the location free, it doesn't have to actually free all bytes on the disk. This looks pretty optimial to me. Buffering is optional, out of interest when does this pay off? I can't imagine the logic and physical mapping has any measurable cost. Down side is that byte[] size needs to be known ahead of time so async streaming for writes won't work. I saw that in the docs it mentions replacing some of the code with DBCache, but I couldn't find the mailing list discussion on this - any details?<BR><BR>There is obviously the issue of multithreading. With the move to JDK1.5, does that help some? JDBM should probably atleast allow multiple reads, regardless of write, sorta like concurrent hashmap. I'm guessing at this point it might be preferable to split up dbs, to allow concurrent writes too, via striping?<BR><BR>One of the use cases I'd like to support is the idea of in memory meta-data for each long, without having to iterate over the entire .db reading in all files. I will probably do this as a second db, that holds only meta data - although then I need to get transactions to span across both dbs. This would hold the long id to the read data. I'd then do a permanent in memory cache o f data, as we'd only ever have a few hundred items anyway. I'm wondering if the idea of "meta" data for records could be built into the main .db. And it should be possible to startup a record manager and load in all meta data. This way the meta data and record can be written continously together.<BR><BR>Mark<BR> <BLOCKQUOTE cite=mid:4857619A.2070105@... type="cite"> <BLOCKQUOTE cite=mid:RERPQVMzRShWVmBVJi0+MjA3MjA0OTQ4NA@... type="cite"><PRE wrap="">- K Kevin Day Trumpet, Inc. <A class=moz-txt-link-abbreviated href="http://www.trumpetinc.com" moz-do-not-send="true">www.trumpetinc.com</A> <A class=moz-txt-link-abbreviated href="mailto:kevin@..." moz-do-not-send="true">kevin@...> 480-961-6003 x1002 ----------------------- Original Message ----------------------- From: "Cees de Groot" <A class=moz-txt-link-rfc2396E href="mailto:cdegroot@..." moz-do-not-send="true">&lt;cdegroot@...> To: "Bryan Thompson" <A class=moz-txt-link-rfc2396E href="mailto:bryan@..." moz-do-not-send="true">&lt;bryan@...> Cc: Mark Proctor <A class=moz-txt-link-rfc2396E href="mailto:mproctor@..." moz-do-not-send="true">&lt;mproctor@...>, <A class=moz-txt-link-abbreviated href="mailto:jdbm-general@..." moz-do-not-send="true">jdbm-general@...>, <A class=moz-txt-link-abbreviated href="mailto:cowtowncoder@..." moz-do-not-send="true">cowtowncoder@...> Date: Tue, 10 Jun 2008 20:34:25 +0200 Subject: Re: [Jdbm-general] extensible serializer If the license issue is sorted, I'm more than happy to keep it in. The original JDBM interfaces are still there, and I'll just move it to jdbm.extser or similar. Any objections? I mean, upside, no license problems, and through code inclusion no dependencies issues. Sounds like ok to me... On Tue, Jun 10, 2008 at 7:11 PM, Bryan Thompson <A class=moz-txt-link-rfc2396E href="mailto:bryan@..." moz-do-not-send="true">&lt;bryan@...> wrote: </PRE> <BLOCKQUOTE type="cite"><PRE wrap="">Interesting. I had to do some significant work in order to get jdbm to persist the serializer state. I would suggest that you look at the code more carefully rather than just rolling it back. Assuming that xstream uses a stateful serializer, you are going to want to preserve the integration points. By "stateful" serializer, I mean one that maintains persistent state NOT recorded in the individual serialized records. extser factors out what serializers are declared, the class ids (int's) assigned to each class for which there is a registered serializer, and the corresponding serializer version(s) and puts that all into a persistent record accessed off of one of the named roots for the store. This makes it extremely compact when serializing object graphs. The shared state is all factored out. In order to support that I had to put in a bunch of hooks that you will want to keep around. Another issue with versioned serializers is that they basically have to be inner classes in order to access the various fields (unless you want the overhead of reflection during serialization!). One of the changes that I introduced with the extser integration was transparently versioning for the btree nodes and leaves for stores that choose to enable extser. If that forward versioning is important then you are going to wind up with something that's tightly coupled regardless. If the broader issue is the dependency, then Cees already imported extser and I "authorize" its relicensing under the license for the jdbm project. -bryan ________________________________ From: <A class=moz-txt-link-abbreviated href="mailto:jdbm-general-bounces@..." moz-do-not-send="true">jdbm-general-bounces@...> [<A class=moz-txt-link-freetext href="mailto:jdbm-general-bounces@..." moz-do-not-send="true">mailto:jdbm-general-bounces@...>] On Behalf Of Mark Proctor Sent: Tuesday, June 10, 2008 12:54 PM To: Bryan Thompson Cc: 'Cees de Groot'; <A class=moz-txt-link-abbreviated href="mailto:jdbm-general@..." moz-do-not-send="true">jdbm-general@...>; <A class=moz-txt-link-abbreviated href="mailto:cowtowncoder@..." moz-do-not-send="true">cowtowncoder@...> Subject: Re: [Jdbm-general] extensible serializer Bryan Thompson wrote: Well, easy come, easy go - but you might want to see who's using it before you drop it out. There is zero overhead when extser is not enabled. it's not so much the overhead, its the extra dependency. Even if you don't use it you are forced to include it, because Serialiser extends it. So if we are to use it, it needs to be "plugged" in, so that it's optional. The LGPL is a wierd issue, basically Apache takes a stand against LGPL and will not allow any of it's projects to depend on an LGPL dependency. This would force Apache DS to have to fork JDBM to maintain a version without that LGPL dependency. It's not that they think that LGPL is causing any wierd violation, they just don't like some of the ambiguity, and thus fall on the side of caution. -b ________________________________ From: <A class=moz-txt-link-abbreviated href="mailto:jdbm-general-bounces@..." moz-do-not-send="true">jdbm-general-bounces@...> [<A class=moz-txt-link-freetext href="mailto:jdbm-general-bounces@..." moz-do-not-send="true">mailto:jdbm-general-bounces@...>] On Behalf Of Mark Proctor Sent: Monday, June 09, 2008 10:58 AM To: Cees de Groot Cc: <A class=moz-txt-link-abbreviated href="mailto:jdbm-general@..." moz-do-not-send="true">jdbm-general@...>; <A class=moz-txt-link-abbreviated href="mailto:cowtowncoder@..." moz-do-not-send="true">cowtowncoder@...> Subject: Re: [Jdbm-general] extensible serializer Cees de Groot wrote: On Mon, Jun 9, 2008 at 4:24 PM, Cees de Groot <A class=moz-txt-link-rfc2396E href="mailto:cdegroot@..." moz-do-not-send="true">&lt;cdegroot@...> wrote: I grabbed the extensible serializer source code and added it to the source tree - the original project doesn't seem to exist anymore so I thought this was the quickest way to get rid of a binary-only dependency. Great, I'l update and look over it. For me I just want JDBM to write my byte[], I don't want it to go anywhere near a serialistion method call, I'm currently just trying to find out if that is possible. On second thought, I agree it's not good to have JDBM depend on a single serializer. So I'm removing the dependency (rolling back to the pre-extser tag in CVS and checking what happened after that) </PRE></BLOCKQUOTE><PRE wrap=""><!----> </PRE></BLOCKQUOTE><BR><PRE wrap=""><HR width="90%" SIZE=4> ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. <A class=moz-txt-link-freetext href="http://sourceforge.net/services/buy/index.php">http://sourceforge.net/services/buy/index.php</A></PRE><PRE wrap=""><HR width="90%" SIZE=4> _______________________________________________ Jdbm-general mailing list <A class=moz-txt-link-abbreviated href="mailto:Jdbm-general@...> <A class=moz-txt-link-freetext href="https://lists.sourceforge.net/lists/listinfo/jdbm-general">https://lists.sourceforge.net/lists/listinfo/jdbm-general</A> </PRE></BLOCKQUOTE><BR> <STYLE type=text/css> P, UL, OL, DL, DIR, MENU, PRE { margin: 0 auto;}</STYLE> <FONT face=Tahoma size=2> <DIV>-------------------------------------------------------------------------<BR>Check out the new SourceForge.net Marketplace.<BR>It's the best place to buy or sell services for<BR>just about anything Open Source.<BR>http://sourceforge.net/services/buy/index.php<BR><BR></DIV></FONT> <STYLE type=text/css> P, UL, OL, DL, DIR, MENU, PRE { margin: 0 auto;}</STYLE> <FONT face=Tahoma size=2> <DIV>_______________________________________________<BR>Jdbm-general mailing list<BR>Jdbm-general@... href="https://lists.sourceforge.net/lists/listinfo/jdbm-general<BR><BR><BR></DIV></FONT></BODY></HTML" target="_new">https://lists.sourceforge.net/lists/listinfo/jdbm-general<BR><BR><BR></DIV></FONT></BODY></HTML>

Re: [Jdbm-general] extensible serializer From: Tatu Saloranta <cowtowncoder@ya...> - 2008-06-17 21:39 --- On Tue, 6/17/08, Kevin Day <kevin@...> wrote: ... > <div>I think something that may be getting missed > here is that the BTree implementation is, itself, a higher > level construct (which serializes itself into a byte array > that is then stored in the record manager). Simple > storage of byte arrays is supported directly in record > manager (that's pretty much the point of > RecordManager).</div> Perfect. And apologies for confusion -- I should have checked the details (it has been a while since I read through source code, but I do remember division between low-level block/record handling and higher level constructs). -+ Tatu +-

2 messages have been excluded from this view by a project administrator.

Next Messages