wh, thanks for the additions to the server. see the changelog for the server for comments about fixes, changes, etc.
Basically, SOAP methods w/ bool returns used to be true, or throw a soap fault. Now they are true/false, and only throw soap faults in dire cases.
made conainer_exists just a bit faster by avoiding call into dbxml, and system call, disk i/o by checking internal map of containers.
delete_container proved a bit more tricky than anticipated, as you need to drop the XmlContainer from the internal map to be able to call XmlManager::removeContainer(). However, another thread could be either putting/getting a doc from same container, or having a long running query against the container, in which case the XmlManager::removeContainer() would fail, and it is required to reinstate the container into the internal map. whatever, it should work now...
i also kept your check_container( ) SOAP method, though i just route it to my container_exists( ) method...
-tom
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Please also grep the code - I put a TODO in there for you.
Do we absoluetely HAVE to have that global static map of container names? Seems to be a point of failure to me, surely we can just get this from DbXML:: somewhere?
For my duplicated method - rather delete it than duplicate it, just remove all the container_exists stuff and update the wsdl. I really want this code base to be squeaky clean. container_exists() is probably a better name...
I'll check out the code now, and see how far the JUnit tests go. They were failing earlier because I was expecting correct true/false results from the method calls, but if that's fixed, we will be close to a release in just a few weeks.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
wh, (you may know some/all of this so forgive my diatribe)
the m_containers is an instance variable privately scoped (well protected really) to the server object, and is a std::map of container names to XmlContainer objects. The XmlContainer class uses a handle idiom, and is meant to be shared across threads. However, when the last reference to the container goes out of scope, the container is closed. Hence, i maintain this map to be shared across threads, keeping containers open and ready for use. This is needed for get_document, put_document, list_documents, query, and likely others. XmlManager supports an auto-open but that will only help in the case of query, and not put_doc, get_doc, list_docs, etc which all require an XmlContainer object. The alternative would be to have each thread maintain individual maps on thread specific storage, but then create_container and delete_container would require all threads to be signaled to update each map, or delete ref's to the underlying container when a delete was needed, etc. and this would be entirely complex and prohibitive from a scalability issue.
The alternative is to open/close each container as needed on each client request. This seems appallingly slow and unneccessary. I would like to do some benchmarks with normal access paterns (what in the hell that is, i don't know...) and see at what level do the threads start blocking based on contention for the m_containers map, and compare that to individual open/closes.
In the long run, i had more invisioned having a multi-tier server w/ a manager process that pre-forks multi-threaded worker daemons. Should a multi-threaded worker fall over, the manager catches the sigchld, and replaces them. This allows us to cap the threads per server to limit contention for the m_containers, while supporting further scaling on bigger iron. It also makes it very reliable. I've had very good experiences w/ similar designs for some of my clients.
I however am open to any new ideas...so pls don't hold your tongue!
I also would really like to see this project get some wind in its sails, and that means being rock solid, packaged, and doc'd. We've got a way to go, but i think this may have a purpose out there in the world!
So, please keep complaining, keep breaking my code, find my weaknesses, and challenge everything...you aren't going to hurt my feelings.
how long before we make a list announce for dbxml? That's what i'd like, is to see some of the sleepycat people getting on, and trying it out.
oh well, i'll shut up now...slight wind here today, but 2 small children and a wife kept me land bound!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
wh, thanks for the additions to the server. see the changelog for the server for comments about fixes, changes, etc.
Basically, SOAP methods w/ bool returns used to be true, or throw a soap fault. Now they are true/false, and only throw soap faults in dire cases.
made conainer_exists just a bit faster by avoiding call into dbxml, and system call, disk i/o by checking internal map of containers.
delete_container proved a bit more tricky than anticipated, as you need to drop the XmlContainer from the internal map to be able to call XmlManager::removeContainer(). However, another thread could be either putting/getting a doc from same container, or having a long running query against the container, in which case the XmlManager::removeContainer() would fail, and it is required to reinstate the container into the internal map. whatever, it should work now...
i also kept your check_container( ) SOAP method, though i just route it to my container_exists( ) method...
-tom
Please also grep the code - I put a TODO in there for you.
Do we absoluetely HAVE to have that global static map of container names? Seems to be a point of failure to me, surely we can just get this from DbXML:: somewhere?
For my duplicated method - rather delete it than duplicate it, just remove all the container_exists stuff and update the wsdl. I really want this code base to be squeaky clean. container_exists() is probably a better name...
I'll check out the code now, and see how far the JUnit tests go. They were failing earlier because I was expecting correct true/false results from the method calls, but if that's fixed, we will be close to a release in just a few weeks.
wh, (you may know some/all of this so forgive my diatribe)
the m_containers is an instance variable privately scoped (well protected really) to the server object, and is a std::map of container names to XmlContainer objects. The XmlContainer class uses a handle idiom, and is meant to be shared across threads. However, when the last reference to the container goes out of scope, the container is closed. Hence, i maintain this map to be shared across threads, keeping containers open and ready for use. This is needed for get_document, put_document, list_documents, query, and likely others. XmlManager supports an auto-open but that will only help in the case of query, and not put_doc, get_doc, list_docs, etc which all require an XmlContainer object. The alternative would be to have each thread maintain individual maps on thread specific storage, but then create_container and delete_container would require all threads to be signaled to update each map, or delete ref's to the underlying container when a delete was needed, etc. and this would be entirely complex and prohibitive from a scalability issue.
The alternative is to open/close each container as needed on each client request. This seems appallingly slow and unneccessary. I would like to do some benchmarks with normal access paterns (what in the hell that is, i don't know...) and see at what level do the threads start blocking based on contention for the m_containers map, and compare that to individual open/closes.
In the long run, i had more invisioned having a multi-tier server w/ a manager process that pre-forks multi-threaded worker daemons. Should a multi-threaded worker fall over, the manager catches the sigchld, and replaces them. This allows us to cap the threads per server to limit contention for the m_containers, while supporting further scaling on bigger iron. It also makes it very reliable. I've had very good experiences w/ similar designs for some of my clients.
I however am open to any new ideas...so pls don't hold your tongue!
I also would really like to see this project get some wind in its sails, and that means being rock solid, packaged, and doc'd. We've got a way to go, but i think this may have a purpose out there in the world!
So, please keep complaining, keep breaking my code, find my weaknesses, and challenge everything...you aren't going to hurt my feelings.
how long before we make a list announce for dbxml? That's what i'd like, is to see some of the sleepycat people getting on, and trying it out.
oh well, i'll shut up now...slight wind here today, but 2 small children and a wife kept me land bound!
I've remove all Java check_container references - now uses container_exists.
Check out the 100% JUnit tests passing!
Next up I'll write a few stress tests to expose all your memory leaks..
;>