Menu
▾
▴
Re: [Jdbm-developer] Need help on .db file optimisation | JDBM
|
From: Alex B. <boi...@in...> - 2008-01-28 14:40:04
|
Hi Naveen,
The size of the database file will not shrink when you delete objects. The
space is released in the database file and will be reused when you add new
objects, so the database size will not grow beyond a certain size if you
keep deleting and adding similar objects.
alex
On 1/28/08, NAVEEN UPADHYAY <nav...@gm...> wrote:
>
>
> Hi all,
> In my code i am using the jdbm-0.12.jar and i am cleaning up
> records from btree using a pseudocode like
> if (_btree.size() >= threshold){//NaveenBEG
> //Going for bulk purging!Please wait.....
> int count =0;
> Tuple tuple;
> TupleBrowser browser;
> browser = _btree.browse();
> tuple = new Tuple();
> while(count <threshold/2 &&browser.getNext(tuple)){
> _btree.remove(tuple.getKey());
> count++ ;
> }
>
> and there is indeed a recman commit to follow...Still i am not seeing any
> decrease in the .db file size.
> Is there still a problem with my code or something wrong with the *
> jdbm-0.12.jar.
>
> Naveen
> *Alex i didnt use the timestamp method as u suggested ,since as rightly
> pointed out by Kevin lot many transaction were hitting the same timestamp :(
> On Jan 25, 2008 2:44 AM, NAVEEN UPADHYAY <nav...@gm...>
> wrote:
>
> > Kevin,
> > Regarding your point, if I do BTree.remove, then I don't need not bother
> > separately for the record manager..
> >
> > Naveen
> >
> > On Jan 24, 2008 11:52 PM, NAVEEN UPADHYAY <nav...@gm...>
> > wrote:
> >
> > > Just to confirm to Alex's comment..
> > > We need *another *Btree(*one more)*..??this time its value being the
> > > key to the first one...
> > >
> > >
> > >
> > > On Jan 24, 2008 3:37 AM, Kevin Day < ke...@tr...> wrote:
> > >
> > > > Just make sure you delete the records from the BTree when you
> > > > delete them from the record manager :-)
> > > >
> > > > For these situations, we actually store the key in the btree as:
> > > >
> > > > [timestamp][recid]
> > > >
> > > > and the value as a null value. that way we don't have to worry
> > > > about key collision if two transactions hit at the exact same time stamp.
> > > >
> > > > I have a class around here somehwere that allows you to easily
> > > > implement a multi-keyed index like the above using longs or strings. If
> > > > it's of interest, let me know and I'll send it to you directly.
> > > >
> > > > - K
> > > >
> > > > ----------------------- *Original Message* -----------------------
> > > >
> > > > *From:* "Alex Boisvert" <boi...@in...><boi...@in...>
> > > > *To:* "NAVEEN UPADHYAY" <nav...@gm...><nav...@gm...>
> > > > *Cc:* jdb...@li...,<jdb...@li...>Kevin Day
> > > > <ke...@tr...> <ke...@tr...>
> > > > *Date:* Wed, 23 Jan 2008 05:32:38 -0800
> > > > *Subject: Re: [Jdbm-developer] Need help on .db file optimisation |
> > > > JDBM*
> > > >
> > > > Naveen,
> > > >
> > > > You can delete/remove whatever you are putting into the database.
> > > > It's up to your application code to do that.
> > > >
> > > > For example, if you are recording events, simply tag the events with
> > > > a timestamp, create an index (btree) on the timestamp and delete old events
> > > > when you reach a given threshold. You can collect statistics on a sample
> > > > set of records to determine your average space requirement and then
> > > > extrapolate to obtain your upper limit based on your file system limitations
> > > > (2G), minus some conservative margin.
> > > >
> > > > regards,
> > > > alex
> > > >
> > > >
> > > >
> > > > On 1/22/08, NAVEEN UPADHYAY <nav...@gm...> wrote:
> > > > >
> > > > > Thanks Alex,Kevin for your responses ;^)
> > > > > The proposed solutions seem to be quite good.
> > > > > Only thing i have is my main concern is to purge old data with
> > > > > these .db files and* NOT *their corruption beyond 2gb.
> > > > > My only aim is to keep the size of the file always within a limit.
> > > > >
> > > > > For my driver only the recent data is of use..
> > > > > So all that i am looking for is a way to purge old data
> > > > >
> > > > > Will throwing IOExceptions work?
> > > > > Please suggest some solutions...
> > > > > Thanks in advance,
> > > > > Naveen
> > > > >
> > > > > On Jan 23, 2008 5:02 AM, Alex Boisvert < boi...@in...>
> > > > > wrote:
> > > > >
> > > > > > On 1/22/08, Kevin Day <ke...@tr...> wrote:
> > > > > >
> > > > > > > Another option would be to have an OOSProvider that could be
> > > > > > > configured with an instrumentable version for testing (this would be more
> > > > > > > akin to dependency injection)... It feels weird to add code to the runtime
> > > > > > > implementation of a class that forces it to fail...
> > > > > > >
> > > > > > > Any preference? (although I love Brian's idea of using a thumb
> > > > > > > drive - or maybe even a RAM disk - it's just hard to enforce that stuff in a
> > > > > > > test suite :-) )
> > > > > > >
> > > > > >
> > > > > > I understand the weird feeling and I don't have a strong
> > > > > > preference. I think the OOSProvider solution is going to involve more code
> > > > > > to achieve the same goal. If the abstraction isn't needed in the first
> > > > > > place, I don't think it's worth the added complexity, meaning adding 4-5
> > > > > > more classes for this test alone.
> > > > > >
> > > > > > alex
> > > > > >
> > > > > >
> > > > > >
> > > > > > -------------------------------------------------------------------------
> > > > > > This SF.net email is sponsored by: Microsoft
> > > > > > Defy all challenges. Microsoft(R) Visual Studio 2008.
> > > > > > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> > > > > > _______________________________________________
> > > > > > Jdbm-developer mailing list
> > > > > > Jdb...@li...
> > > > > > <Jdb...@li...urcefor%0A+ge.net>
> > > > > > https://lists.sourceforge.net/lists/listinfo/jdbm-developer
> > > > > >
> > > > > >
> > > > >
> > > >
> > > > -------------------------------------------------------------------------
> > > > This SF.net email is sponsored by: Microsoft
> > > > Defy all challenges. Microsoft(R) Visual Studio 2008.
> > > > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> > > >
> > > > _______________________________________________
> > > > Jdbm-developer mailing list
> > > > Jdb...@li...
> > > > https://lists.sourceforge.net/lists/listinfo/jdbm-developer
> > > >
> > > >
> > > >
> > > >
> > > > -------------------------------------------------------------------------
> > > > This SF.net email is sponsored by: Microsoft
> > > > Defy all challenges. Microsoft(R) Visual Studio 2008.
> > > > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> > > > _______________________________________________
> > > > Jdbm-developer mailing list
> > > > Jdb...@li...
> > > > https://lists.sourceforge.net/lists/listinfo/jdbm-developer
> > > >
> > > >
> > >
> >
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Jdbm-developer mailing list
> Jdb...@li...
> https://lists.sourceforge.net/lists/listinfo/jdbm-developer
>
>
|