I am trying to figure out why jdbm runs out of memory on a large insert operation. I thought that there was an option (jdbm.disableTransactions) to turn off the in-memory transaction buffer, but that does not seem to solve the problem. I have not looked into this deeply (yet), but I was wondering if other folks had already explored this problem in their applications and what tradeoffs are involved as the size of the transaction grows very large.
Thanks,
-bryan
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
We're definately not hoarding. All references are recids. The application itself is layered. We have a different persistence layer that we also run over which does not have these problems. I ran the profiler earlier, but I have not delved into the data yet to see where all the memory is going. I was hoping that there was an easy answer with respect to transaction buffers.
One thing that I do not understand is the tradeoff when the in-memory transaction buffers are disabled. Could you expand on that?
Thanks,
-bryan
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I spoke too soon. There is a graph of listeners that are coded as pointers and not recids....oops!
I was thinking in terms of the application, which is the same over jdbm and the other store. However the intermediate layer for jdbm is different -- and that is where we are hoarding. My bad.
Thanks,
-bryan
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hello,
I am trying to figure out why jdbm runs out of memory on a large insert operation. I thought that there was an option (jdbm.disableTransactions) to turn off the in-memory transaction buffer, but that does not seem to solve the problem. I have not looked into this deeply (yet), but I was wondering if other folks had already explored this problem in their applications and what tradeoffs are involved as the size of the transaction grows very large.
Thanks,
-bryan
Have you made sure that your application is not "hoarding" a large object graph?
Beyond the LRU cache and the transaction buffers, JDBM does not retain any object in memory.
Maybe it's time to launch your memory profiler to see what objects stay in memory and how they are being referenced...
alex
Alex,
We're definately not hoarding. All references are recids. The application itself is layered. We have a different persistence layer that we also run over which does not have these problems. I ran the profiler earlier, but I have not delved into the data yet to see where all the memory is going. I was hoping that there was an easy answer with respect to transaction buffers.
One thing that I do not understand is the tradeoff when the in-memory transaction buffers are disabled. Could you expand on that?
Thanks,
-bryan
Alex,
I spoke too soon. There is a graph of listeners that are coded as pointers and not recids....oops!
I was thinking in terms of the application, which is the same over jdbm and the other store. However the intermediate layer for jdbm is different -- and that is where we are hoarding. My bad.
Thanks,
-bryan
Good news; I'm glad you resolved your problem.
alex