From: Thompson, B. B. <BRY...@sa...> - 2006-04-17 13:39:40
|
All, I am looking over the WeakCache class and I am wondering if the following scenario is possible: The WeakCache has a large number of entries, greater than the size of the internal MRU cache when the transaction commits. This could happen if the application was maintaining its own hard references to those runtime objects, e.g., in some caches or temporary data structures. During the commit the CacheRecordManager will ask for an enumeration of the objects in the cache. The enumeration provided will visit the members of the internal MRU, not those in the weak value cache. Since the weak cache has more objects that can fit on the internal MRU cache, some dirty objects will not be reported to the cache record manager and the transaction will not commit all changes. I think that what we need to do is place the dirty objects onto a linked list whose elements are indexed by a hash table (much like the MRU entries, but without a reordering or eviction policy) and then scan just the list of dirty objects during the commit. Objects evicted from the MRU would be removed from the dirty list as they are laid down onto page images. (Per the semantics of the dirty list, an object whose state is current on the page image does not belong on the dirty list). -bryan |
From: Kevin D. <ke...@tr...> - 2006-04-17 14:42:39
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> <HTML><HEAD> <STYLE type=text/css> P, UL, OL, DL, DIR, MENU, PRE { margin: 0 auto;}</STYLE> <META content="MSHTML 6.00.2900.2802" name=GENERATOR></HEAD> <BODY leftMargin=1 topMargin=1 rightMargin=1><FONT face=Tahoma> <DIV><FONT face=Arial size=2>Bryan-</FONT></DIV> <DIV><FONT face=Arial size=2></FONT> </DIV> <DIV><FONT face=Arial size=2></FONT></DIV> <DIV><FONT face=Arial size=2>I don't think this is an issue. Under the current code, if an object evicts from the cache, and the object has been marked as updated, then it's content gets written to the page and marked dirty. In the case of the weak cache, it is fronted by an MRU cache, so the cache eviction technically happens when the object is evicted from the MRU (this technically results in an earlier page write than is actually needed, but there's no way to capture a weak reference eviction immediately before it occurs, so a compromise is made).</FONT></DIV> <DIV><FONT face=Arial size=2></FONT> </DIV> <DIV><FONT face=Arial size=2></FONT> </DIV> <DIV><FONT face=Arial size=2>- K</FONT></DIV> <DIV><FONT face=Arial size=2> </FONT> <TABLE> <TBODY> <TR> <TD width=1 bgColor=blue><FONT face=Arial size=2></FONT></TD> <TD><FONT face=Arial size=2><FONT color=blue>> <BR>All, <BR> <BR>I am looking over the WeakCache class and I am wondering if the following scenario is possible: The WeakCache has a large number of entries, greater than the size of the internal MRU cache when the transaction commits. This could happenif the application was maintaining its own hard references to those runtime objects, e.g., in some caches or temporary data structures. During the commit the CacheRecordManager will ask for an enumeration of the objects in the cache. The enumeration provided will visit the members of the internal MRU, not those in the weak value cache. Since the weak cache has more objects that can fit on the internal MRU cache, some dirty objects will not be reported to the cache record manager and the transaction will not commit all changes. <BR> <BR>I think that what we need to do is place the dirty objects onto a linked list whose elements are indexed by a hash table (much like the MRU entries, but without a reordering or eviction policy) and then scan just the list of dirty objects during the commit. Objects evicted from the MRU would be removed from the dirty list as they are laid down onto page images. (Per the semantics of the dirty list, an object whose state is current on the page image does not belong on the dirty list). <BR> <BR>-bryan <BR><<BR></FONT></FONT></TD></TR></TBODY></TABLE></DIV></FONT></BODY></HTML> |
From: Bryan T. <tho...@sp...> - 2006-04-22 10:32:09
|
All, CacheRecordManager inserts CacheEntry objects into the weak value cache. Those objects have attributes containing the recid, serializer, dirty flag and a hard reference to the object. The weak value cache will clear these entries as soon as they are no longer strongly reachable. Since the application does not hold onto the CacheEntry objects itself, but only the object reference, it seems that the CacheEntry entries in the weak value cache will be swept by the garbage collector and removed from the weak value cache shortly after they fall off of the internal hard reference MRU. That is, it does not look like the weak value cache is serving any purpose. Thoughts? -bryan -----Original Message----- From: jdb...@li... [mailto:jdb...@li...]On Behalf Of Kevin Day Sent: Monday, April 17, 2006 10:46 AM To: JDBM Developer listserv Subject: re: [Jdbm-developer] Soft/Weak cache question - it seems like we have a hole in the ca che commit behavior. Bryan- I don't think this is an issue. Under the current code, if an object evicts from the cache, and the object has been marked as updated, then it's content gets written to the page and marked dirty. In the case of the weak cache, it is fronted by an MRU cache, so the cache eviction technically happens when the object is evicted from the MRU (this technically results in an earlier page write than is actually needed, but there's no way to capture a weak reference eviction immediately before it occurs, so a compromise is made). - K > All, I am looking over the WeakCache class and I am wondering if the following scenario is possible: The WeakCache has a large number of entries, greater than the size of the internal MRU cache when the transaction commits. This could happenif the application was maintaining its own hard references to those runtime objects, e.g., in some caches or temporary data structures. During the commit the CacheRecordManager will ask for an enumeration of the objects in the cache. The enumeration provided will visit the members of the internal MRU, not those in the weak value cache. Since the weak cache has more objects that can fit on the internal MRU cache, some dirty objects will not be reported to the cache record manager and the transaction will not commit all changes. I think that what we need to do is place the dirty objects onto a linked list whose elements are indexed by a hash table (much like the MRU entries, but without a reordering or eviction policy) and then scan just the list of dirty objects during the commit. Objects evicted from the MRU would be removed from the dirty list as they are laid down onto page images. (Per the semantics of the dirty list, an object whose state is current on the page image does not belong on the dirty list). -bryan < ------------------------------------------------------- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642 _______________________________________________ Jdbm-developer mailing list Jdb...@li... https://lists.sourceforge.net/lists/listinfo/jdbm-developer |
From: Kevin D. <ke...@tr...> - 2006-04-23 23:14:20
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> <HTML><HEAD> <STYLE type=text/css> P, UL, OL, DL, DIR, MENU, PRE { margin: 0 auto;}</STYLE> <META content="MSHTML 6.00.2900.2802" name=GENERATOR></HEAD> <BODY leftMargin=1 topMargin=1 rightMargin=1><FONT face=Tahoma> <DIV><FONT face=Arial size=2>Bryan-</FONT></DIV> <DIV><FONT face=Arial size=2></FONT> </DIV> <DIV><FONT face=Arial size=2></FONT></DIV> <DIV><FONT face=Arial size=2>I've never understood why one would want a weakreferencecache... I always use softreference myself...</FONT></DIV> <DIV><FONT face=Arial size=2></FONT> </DIV> <DIV><FONT face=Arial size=2>- K</FONT></DIV> <DIV><FONT face=Arial size=2> </FONT> <TABLE> <TBODY> <TR> <TD width=1 bgColor=blue><FONT face=Arial size=2></FONT></TD> <TD><FONT face=Arial size=2><FONT color=red>> <BR>All, <BR> <BR>CacheRecordManager inserts CacheEntry objects into the weak value cache. Those objects have attributes <BR>containing the recid, serializer, dirty flag and a hard reference to the object. The weak value cache will clear <BR>these entries as soon as they are no longer strongly reachable. Since the application does not hold onto the <BR>CacheEntry objects itself, but only the object reference, it seems that the CacheEntry entries in the weak value <BR>cache will be swept by the garbage collector and removed from the weak value cache shortly after they fall off of <BR>the internal hard reference MRU. That is, it does not look like the weak value cache is serving any purpose. <BR> <BR>Thoughts? <BR> <BR>-bryan <BR>-----Original Message-----<BR>From: <A href="mailto:jdb...@li..."><FONT color=#0000ff>jdb...@li...</FONT></A> <A href="mailto:jdb...@li...]On"><FONT color=#0000ff>[mailto:jdb...@li...]On</FONT></A> Behalf Of Kevin Day<BR>Sent: Monday, April 17, 2006 10:46 AM<BR>To: JDBM Developer listserv<BR>Subject: re: [Jdbm-developer] Soft/Weak cache question - it seems like we have a hole in the ca che commit behavior.<BR><BR><BR>Bryan- <BR> <BR><BR>I don't think this is an issue. Under the current code, if an object evicts from the cache, and the object has been marked as updated, then it's content gets written to the page and marked dirty. In the case of the weak cache, it is fronted by an MRU cache, so the cache eviction technically happens when the object is evicted from the MRU (this technically results in an earlier page write than is actually needed, but there's no way to capture a weak reference eviction immediately before it occurs, so a compromise is made). <BR> <BR> <BR>- K <BR> <BR>><BR>All, <BR><BR>I am looking over the WeakCache class and I am wondering if the following scenario is possible: The WeakCache has a large number of entries, greater than the size of the internal MRU cache when the transaction commits. This could happenif the application was maintaining its own hard references to those runtime objects, e.g., in some caches or temporary data structures. During the commit the CacheRecordManager will ask for an enumeration of the objects in the cache. The enumeration provided will visit the members of the internal MRU, not those in the weak value cache. Since the weak cache has more objects that can fit on the internal MRU cache, some dirty objects will not be reported to the cache record manager and the transaction will not commit all changes. <BR><BR>I think that what we need to do is place the dirty objects onto a linked list whose elements are indexed by a hash table (much like the MRU entries, but without a reordering or eviction policy) and then scan just the list of dirty objects during the commit. Objects evicted from the MRU would be removed from the dirty list as they are laid down onto page images. (Per the semantics of the dirty list, an object whose state is current on the page image does not belong on the dirtylist). <BR><BR>-bryan <BR><<BR>------------------------------------------------------- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! <A href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642"><FONT color=#0000ff>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642</FONT></A> _______________________________________________ Jdbm-developer mailing list <A href="mailto:Jdb...@li..."><FONT color=#0000ff>Jdb...@li...</FONT></A> <A href="https://lists.sourceforge.net/lists/listinfo/jdbm-developer"><FONT color=#0000ff>https://lists.sourceforge.net/lists/listinfo/jdbm-developer</FONT></A> <<BR></FONT></FONT></TD></TR></TBODY></TABLE></DIV></FONT></BODY></HTML> |
From: Bryan T. <tho...@sp...> - 2006-04-24 11:11:49
|
Because it lets you adjust the degree of retention simply by changing the MRU size. But the point is the same either way, the weak/soft cache entries are inside of the CacheRecordManager CacheEntry objects. Since noone is holding onto the CacheEntry objects, the cache is not doing what we want. -bryan -----Original Message----- From: jdb...@li... [mailto:jdb...@li...]On Behalf Of Kevin Day Sent: Sunday, April 23, 2006 7:18 PM To: JDBM Developer listserv Subject: re[2]: [Jdbm-developer] Soft/Weak cache question - it seems like we have a hole in the cache commit behavior. Bryan- I've never understood why one would want a weakreferencecache... I always use softreference myself... - K > All, CacheRecordManager inserts CacheEntry objects into the weak value cache. Those objects have attributes containing the recid, serializer, dirty flag and a hard reference to the object. The weak value cache will clear these entries as soon as they are no longer strongly reachable. Since the application does not hold onto the CacheEntry objects itself, but only the object reference, it seems that the CacheEntry entries in the weak value cache will be swept by the garbage collector and removed from the weak value cache shortly after they fall off of the internal hard reference MRU. That is, it does not look like the weak value cache is serving any purpose. Thoughts? -bryan -----Original Message----- From: jdb...@li... [mailto:jdb...@li...]On Behalf Of Kevin Day Sent: Monday, April 17, 2006 10:46 AM To: JDBM Developer listserv Subject: re: [Jdbm-developer] Soft/Weak cache question - it seems like we have a hole in the ca che commit behavior. Bryan- I don't think this is an issue. Under the current code, if an object evicts from the cache, and the object has been marked as updated, then it's content gets written to the page and marked dirty. In the case of the weak cache, it is fronted by an MRU cache, so the cache eviction technically happens when the object is evicted from the MRU (this technically results in an earlier page write than is actually needed, but there's no way to capture a weak reference eviction immediately before it occurs, so a compromise is made). - K > All, I am looking over the WeakCache class and I am wondering if the following scenario is possible: The WeakCache has a large number of entries, greater than the size of the internal MRU cache when the transaction commits. This could happenif the application was maintaining its own hard references to those runtime objects, e.g., in some caches or temporary data structures. During the commit the CacheRecordManager will ask for an enumeration of the objects in the cache. The enumeration provided will visit the members of the internal MRU, not those in the weak value cache. Since the weak cache has more objects that can fit on the internal MRU cache, some dirty objects will not be reported to the cache record manager and the transaction will not commit all changes. I think that what we need to do is place the dirty objects onto a linked list whose elements are indexed by a hash table (much like the MRU entries, but without a reordering or eviction policy) and then scan just the list of dirty objects during the commit. Objects evicted from the MRU would be removed from the dirty list as they are laid down onto page images. (Per the semantics of the dirty list, an object whose state is current on the page image does not belong on the dirtylist). -bryan < ------------------------------------------------------- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642 _______________________________________________ Jdbm-developer mailing list Jdb...@li... https://lists.sourceforge.net/lists/listinfo/jdbm-developer < ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Jdbm-developer mailing list Jdb...@li... https://lists.sourceforge.net/lists/listinfo/jdbm-developer |
From: Kevin D. <ke...@tr...> - 2006-04-24 18:10:52
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> <HTML><HEAD> <STYLE type=text/css> P, UL, OL, DL, DIR, MENU, PRE { margin: 0 auto;}</STYLE> <META content="MSHTML 6.00.2900.2802" name=GENERATOR></HEAD> <BODY leftMargin=1 topMargin=1 rightMargin=1><FONT face=Tahoma> <DIV><FONT face=Arial size=2>Bryan-</FONT></DIV> <DIV><FONT face=Arial size=2></FONT> </DIV> <DIV><FONT face=Arial size=2></FONT></DIV> <DIV><FONT face=Arial size=2>hmmm - not sure I understand this one...</FONT></DIV> <DIV><FONT face=Arial size=2></FONT> </DIV> <DIV><FONT face=Arial size=2>If a weak reference is used, then the object is not garbage collected until all other, non-weak referenced objects have been collected. This means that the cache allows us to retrieve an instantiated form of a given object from the cache, even if all hard references to it have been severed.</FONT></DIV> <DIV><FONT face=Arial size=2></FONT> </DIV> <DIV><FONT face=Arial size=2>The MRU size drives when the changes to the object are actually serialized to bytes. The weak reference cache determines whether deserialization is necessary for a given fetch request.</FONT></DIV> <DIV><FONT face=Arial size=2></FONT> </DIV> <DIV><FONT face=Arial size=2>I believe that the comments in the weak reference cache implementation describe this in a lot of detail...</FONT></DIV> <DIV><FONT face=Arial size=2></FONT> </DIV> <DIV><FONT face=Arial size=2>- K</FONT></DIV> <DIV><FONT face=Arial size=2> </FONT> <TABLE> <TBODY> <TR> <TD width=1 bgColor=blue><FONT face=Arial size=2></FONT></TD> <TD><FONT face=Arial size=2><FONT color=blue>> Because it lets you adjust the degree of retention simply by changing the MRU size. But the point is the same <BR>either way, the weak/soft cache entries are inside of the CacheRecordManager CacheEntry objects. Since noone <BR>is holding onto the CacheEntry objects, the cache is not doing what we want. <BR>-bryan <BR>-----Original Message-----<BR>From: <A href="mailto:jdb...@li..."><FONT color=#0000ff>jdb...@li...</FONT></A> <A href="mailto:jdb...@li...]On"><FONT color=#0000ff>[mailto:jdb...@li...]On</FONT></A> Behalf Of Kevin Day<BR>Sent: Sunday, April 23, 2006 7:18 PM<BR>To: JDBM Developer listserv<BR>Subject: re[2]: [Jdbm-developer] Soft/Weak cache question - it seems like we have a hole in the cache commit behavior.<BR><BR><BR>Bryan- <BR> <BR><BR>I've never understood why one would want a weakreferencecache... I always use softreference myself... <BR> <BR>- K <BR> <BR>><BR>All, <BR><BR>CacheRecordManager inserts CacheEntry objects into the weak value cache. Those objects have attributes <BR>containing the recid, serializer, dirty flag and a hard reference to the object. The weak value cache will clear <BR>these entries as soon as they are no longer strongly reachable. Since the application does not hold onto the <BR>CacheEntry objects itself, but only the object reference, it seems that the CacheEntry entries in the weak value <BR>cache will be swept by the garbage collector and removed from the weak value cache shortly after they fall off of <BR>the internal hard reference MRU. That is, it does not look like the weak value cache is serving any purpose. <BR><BR>Thoughts? <BR><BR>-bryan <BR>-----Original Message-----<BR>From: <A href="mailto:jdb...@li..."><FONT color=#0000ff>jdb...@li...</FONT></A> <A href="mailto:jdb...@li...]On"><FONT color=#0000ff>[mailto:jdb...@li...]On</FONT></A> Behalf Of Kevin Day<BR>Sent: Monday, April 17, 2006 10:46 AM<BR>To: JDBM Developer listserv<BR>Subject: re: [Jdbm-developer] Soft/Weak cache question - it seems like we have a hole in the ca che commit behavior.<BR><BR><BR>Bryan- <BR><BR><BR>I don't think this is an issue. Under the current code, if an object evicts from the cache, and the object has been marked as updated, then it's content gets written to the page and marked dirty. In the case of the weak cache, it is fronted by an MRU cache, so the cache eviction technically happens when the object is evicted from the MRU (this technically results in an earlier page write than is actually needed, but there's no way to capture a weak reference eviction immediately before it occurs, so a compromise is made). <BR><BR><BR>- K <BR> <BR>><BR>All, <BR><BR>I am looking over the WeakCache class and I am wondering if the following scenario is possible: The WeakCache has a large number of entries, greater than the size of the internal MRU cache when the transaction commits. This could happenif the application was maintaining its own hard references to those runtime objects, e.g., in some caches or temporary data structures. During the commit the CacheRecordManager will ask for an enumeration of the objects in the cache. The enumeration provided will visit the members of the internal MRU, not those in the weak value cache. Since the weak cache has more objects that can fit on the internal MRU cache, some dirty objects will not be reported to the cache record manager and the transaction will not commit all changes. <BR><BR>I think that what we need to do is place the dirty objects onto a linked list whose elements are indexed by a hash table (much like the MRU entries, but without a reordering or eviction policy) and then scan just the list of dirty objects during the commit. Objects evicted from the MRU would be removed from the dirty list as they are laid down onto page images. (Per the semantics of the dirty list, an object whose state is current on the page image does not belong onthe dirtylist). <BR><BR>-bryan <BR><<BR>------------------------------------------------------- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! <A href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642"><FONT color=#0000ff>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642</FONT></A> _______________________________________________ Jdbm-developer mailing list <A href="mailto:Jdb...@li..."><FONT color=#0000ff>Jdb...@li...</FONT></A> <A href="https://lists.sourceforge.net/lists/listinfo/jdbm-developer"><FONT color=#0000ff>https://lists.sourceforge.net/lists/listinfo/jdbm-developer</FONT></A> <<BR>------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo <A href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642"><FONT color=#0000ff>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642</FONT></A> _______________________________________________ Jdbm-developer mailing list <A href="mailto:Jdb...@li..."><FONT color=#0000ff>Jdb...@li...</FONT></A> <A href="https://lists.sourceforge.net/lists/listinfo/jdbm-developer"><FONT color=#0000ff>https://lists.sourceforge.net/lists/listinfo/jdbm-developer</FONT></A> <<BR></FONT></FONT></TD></TR></TBODY></TABLE></DIV></FONT></BODY></HTML> |
From: Bryan T. <tho...@sp...> - 2006-04-24 18:48:59
|
Kevin, The objects that are actually inserted into the soft/weak cache are not the application objects directly, but CacheRecordManager.CacheEntry instances. Since the application is not holding references to those CacheEntry instances, they go out of scope as soon as the code exits a method on the CacheRecordManager class. This is counter to the intention for the soft/weak cache. What we need is to drive the dirty flag and serializer references into the SoftReference / WeakReference objects that are held as the cache values. Consider, String s = new String(); long recid = crm.insert(s); CRM does: entry = new CacheEntry( recid, s, serializer, isDirty ); cache.put( recid, entry ); Soft/WeakCache does: internal.put( recid, new Entry( recid, entry ) ); Where Entry extends Soft/WeakReference. So the WeakReference is to the CacheEntry, not to the String [s]. Holding onto a reference to [s] does not effect the retention of the soft/weak cache entry for [s]. The application would actualy have to hold onto the CacheEntry object, which it never sees. We could fix this be expanding the interface API for the CachePolicy to pass along the additional state {dirty flag, serializer} and driving that state into the Weak/SoftReference Entry class used by the Weak/SoftCache class. -bryan -----Original Message----- From: jdb...@li... [mailto:jdb...@li...]On Behalf Of Kevin Day Sent: Monday, April 24, 2006 2:15 PM To: JDBM Developer listserv Subject: re[4]: [Jdbm-developer] Soft/Weak cache question - it seems like we have a hole in the cache commit behavior. Bryan- hmmm - not sure I understand this one... If a weak reference is used, then the object is not garbage collected until all other, non-weak referenced objects have been collected. This means that the cache allows us to retrieve an instantiated form of a given object from the cache, even if all hard references to it have been severed. The MRU size drives when the changes to the object are actually serialized to bytes. The weak reference cache determines whether deserialization is necessary for a given fetch request. I believe that the comments in the weak reference cache implementation describe this in a lot of detail... - K > Because it lets you adjust the degree of retention simply by changing the MRU size. But the point is the same either way, the weak/soft cache entries are inside of the CacheRecordManager CacheEntry objects. Since noone is holding onto the CacheEntry objects, the cache is not doing what we want. -bryan -----Original Message----- From: jdb...@li... [mailto:jdb...@li...]On Behalf Of Kevin Day Sent: Sunday, April 23, 2006 7:18 PM To: JDBM Developer listserv Subject: re[2]: [Jdbm-developer] Soft/Weak cache question - it seems like we have a hole in the cache commit behavior. Bryan- I've never understood why one would want a weakreferencecache... I always use softreference myself... - K > All, CacheRecordManager inserts CacheEntry objects into the weak value cache. Those objects have attributes containing the recid, serializer, dirty flag and a hard reference to the object. The weak value cache will clear these entries as soon as they are no longer strongly reachable. Since the application does not hold onto the CacheEntry objects itself, but only the object reference, it seems that the CacheEntry entries in the weak value cache will be swept by the garbage collector and removed from the weak value cache shortly after they fall off of the internal hard reference MRU. That is, it does not look like the weak value cache is serving any purpose. Thoughts? -bryan -----Original Message----- From: jdb...@li... [mailto:jdb...@li...]On Behalf Of Kevin Day Sent: Monday, April 17, 2006 10:46 AM To: JDBM Developer listserv Subject: re: [Jdbm-developer] Soft/Weak cache question - it seems like we have a hole in the ca che commit behavior. Bryan- I don't think this is an issue. Under the current code, if an object evicts from the cache, and the object has been marked as updated, then it's content gets written to the page and marked dirty. In the case of the weak cache, it is fronted by an MRU cache, so the cache eviction technically happens when the object is evicted from the MRU (this technically results in an earlier page write than is actually needed, but there's no way to capture a weak reference eviction immediately before it occurs, so a compromise is made). - K > All, I am looking over the WeakCache class and I am wondering if the following scenario is possible: The WeakCache has a large number of entries, greater than the size of the internal MRU cache when the transaction commits. This could happenif the application was maintaining its own hard references to those runtime objects, e.g., in some caches or temporary data structures. During the commit the CacheRecordManager will ask for an enumeration of the objects in the cache. The enumeration provided will visit the members of the internal MRU, not those in the weak value cache. Since the weak cache has more objects that can fit on the internal MRU cache, some dirty objects will not be reported to the cache record manager and the transaction will not commit all changes. I think that what we need to do is place the dirty objects onto a linked list whose elements are indexed by a hash table (much like the MRU entries, but without a reordering or eviction policy) and then scan just the list of dirty objects during the commit. Objects evicted from the MRU would be removed from the dirty list as they are laid down onto page images. (Per the semantics of the dirty list, an object whose state is current on the page image does not belong onthe dirtylist). -bryan < ------------------------------------------------------- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642 _______________________________________________ Jdbm-developer mailing list Jdb...@li... https://lists.sourceforge.net/lists/listinfo/jdbm-developer < ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Jdbm-developer mailing list Jdb...@li... https://lists.sourceforge.net/lists/listinfo/jdbm-developer < ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Jdbm-developer mailing list Jdb...@li... https://lists.sourceforge.net/lists/listinfo/jdbm-developer |
From: Kevin D. <ke...@tr...> - 2006-04-24 19:47:23
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> <HTML><HEAD> <STYLE type=text/css> P, UL, OL, DL, DIR, MENU, PRE { margin: 0 auto;}</STYLE> <META content="MSHTML 6.00.2900.2802" name=GENERATOR></HEAD> <BODY leftMargin=1 topMargin=1 rightMargin=1><FONT face=Tahoma> <DIV><FONT face=Arial size=2>Bryan-</FONT></DIV> <DIV><FONT face=Arial size=2></FONT> </DIV> <DIV><FONT face=Arial size=2></FONT></DIV> <DIV><FONT face=Arial size=2>mmmm - this implementation is exactly per the jdk docs for the soft/weak reference classes...</FONT></DIV> <DIV><FONT face=Arial size=2></FONT> </DIV> <DIV><FONT face=Arial size=2>The implementation is solid as far as I can tell...</FONT></DIV> <DIV><FONT face=Arial size=2></FONT> </DIV> <DIV><FONT face=Arial size=2>Can you come up with a unit test that results in the failure mechanism that you are concerned with?</FONT></DIV> <DIV><FONT face=Arial size=2></FONT> </DIV> <DIV><FONT face=Arial size=2>- K</FONT></DIV> <DIV><FONT face=Arial size=2> </FONT> <TABLE> <TBODY> <TR> <TD width=1 bgColor=blue><FONT face=Arial size=2></FONT></TD> <TD><FONT face=Arial size=2><FONT color=green>> Kevin,<BR><BR>The objects that are actually inserted into the soft/weak cache are not the<BR>application objects directly, but CacheRecordManager.CacheEntry instances.<BR>Since the application is not holding references to those CacheEntry<BR>instances,<BR>they go out of scope as soon as the code exits a method on the<BR>CacheRecordManager<BR>class. This is counter to the intention for the soft/weak cache. What we<BR>need<BR>is to drive the dirty flag and serializer references into the SoftReference<BR>/ WeakReference objects that are held as the cache values.<BR><BR>Consider,<BR><BR>String s = new String();<BR>long recid = crm.insert(s);<BR><BR> CRM does: entry = new CacheEntry( recid, s, serializer, isDirty );<BR> cache.put( recid, entry );<BR><BR> Soft/WeakCache does: internal.put( recid, new Entry( recid, entry ) );<BR><BR>Where Entry extends Soft/WeakReference.<BR><BR>So the WeakReference is to the CacheEntry, not to the String [s]. Holding<BR>onto a reference to [s] does not effect the retention of the soft/weak cache<BR>entry for [s]. The application would actualy have to hold onto the<BR>CacheEntry<BR>object, which it never sees.<BR><BR>We could fix this be expanding the interface API for the CachePolicy to pass<BR>along the additional state {dirty flag, serializer} and driving that state<BR>into the Weak/SoftReference Entry class used by the Weak/SoftCache class.<BR><BR>-bryan<BR><BR>-----Original Message-----<BR>From: <A href="mailto:jdb...@li..."><FONT color=#0000ff>jdb...@li...</FONT></A><BR><A href="mailto:jdb...@li...]On"><FONT color=#0000ff>[mailto:jdb...@li...]On</FONT></A> Behalf Of Kevin Day<BR>Sent: Monday, April 24, 2006 2:15 PM<BR>To: JDBM Developer listserv<BR>Subject: re[4]: [Jdbm-developer] Soft/Weak cache question - it seems like we<BR>have a hole in the cache commit behavior.<BR><BR><BR>Bryan-<BR><BR>hmmm - not sure I understand this one...<BR><BR>If a weak reference is used, then the object is not garbage collected until<BR>all other, non-weak referenced objects have been collected. This means that<BR>the cache allows us to retrieve an instantiated form of a given object from<BR>the cache, even if all hard references to it have been severed.<BR><BR>The MRU size drives when the changes to the object are actually serialized<BR>to bytes. The weak reference cache determines whether deserialization is<BR>necessary for a given fetch request.<BR><BR>I believe that the comments in the weak reference cache implementation<BR>describe this in a lot of detail...<BR><BR>- K<BR> > Because it lets you adjust the degree of retention simply by changing<BR>the MRU size. But the point is the same<BR>either way, the weak/soft cache entries are inside of the<BR>CacheRecordManager CacheEntry objects. Since noone<BR>is holding onto the CacheEntry objects, the cache is not doing what we<BR>want.<BR>-bryan<BR>-----Original Message-----<BR>From: <A href="mailto:jdb...@li..."><FONT color=#0000ff>jdb...@li...</FONT></A><BR><A href="mailto:jdb...@li...]On"><FONT color=#0000ff>[mailto:jdb...@li...]On</FONT></A> Behalf Of Kevin Day<BR>Sent: Sunday, April 23, 2006 7:18 PM<BR>To: JDBM Developer listserv<BR>Subject: re[2]: [Jdbm-developer] Soft/Weak cache question - it seems like<BR>we have a hole in the cache commit behavior.<BR><BR><BR>Bryan-<BR><BR><BR>I've never understood why one would want a weakreferencecache... I always<BR>use softreference myself...<BR><BR>- K<BR><BR>><BR>All,<BR><BR>CacheRecordManager inserts CacheEntry objects into the weak value cache.<BR>Those objects have attributes<BR>containing the recid, serializer, dirty flag and a hard reference to the<BR>object. The weak value cache will clear<BR>these entries as soon as they are no longer strongly reachable. Since the<BR>application does not hold onto the<BR>CacheEntry objects itself, but only the object reference, it seems that the<BR>CacheEntry entries in the weak value<BR>cache will be swept by the garbage collector and removed from the weak<BR>value cache shortly after they fall off of<BR>the internal hard reference MRU. That is, it does not look like the weak<BR>value cache is serving any purpose.<BR><BR>Thoughts?<BR><BR>-bryan<BR>-----Original Message-----<BR>From: <A href="mailto:jdb...@li..."><FONT color=#0000ff>jdb...@li...</FONT></A><BR><A href="mailto:jdb...@li...]On"><FONT color=#0000ff>[mailto:jdb...@li...]On</FONT></A> Behalf Of Kevin Day<BR>Sent: Monday, April 17, 2006 10:46 AM<BR>To: JDBM Developer listserv<BR>Subject: re: [Jdbm-developer] Soft/Weak cache question - it seems like we<BR>have a hole in the ca che commit behavior.<BR><BR><BR>Bryan-<BR><BR><BR>I don't think this is an issue. Under the current code, if an object<BR>evicts from the cache, and the object has been marked as updated, then<BR>it's content gets written to the page and marked dirty. In the case of<BR>the weak cache, it is fronted by an MRU cache, so the cache eviction<BR>technically happens when the object is evicted from the MRU (this<BR>technically results in an earlier page write than is actually needed, but<BR>there's no way to capture a weak reference eviction immediately before it<BR>occurs, so a compromise is made).<BR><BR><BR>- K<BR><BR>><BR>All,<BR><BR>I am looking over the WeakCache class and I am wondering if the following<BR>scenario is possible: The WeakCache has a large number of entries, greater<BR>than the size of the internal MRU cache when the transaction commits.<BR>This could happenif the application was maintaining its own hard<BR>references to those runtime objects, e.g., in some caches or temporary<BR>data structures. During the commit the CacheRecordManager will ask for<BR>an enumeration of the objects in the cache. The enumeration provided will<BR>visit the members of the internal MRU, not those in the weak value cache.<BR>Since the weak cache has more objects that can fit on the internal MRU<BR>cache, some dirty objects will not be reported to the cache record manager<BR>and the transaction will not commit all changes.<BR><BR>I think that what we need to do is place the dirty objects onto a linked<BR>list whose elements are indexed by a hash table (much like the MRU<BR>entries, but without a reordering or eviction policy) and then scan just<BR>the list of dirty objects during the commit. Objects evicted from the MRU<BR>would be removed from the dirty list as they are laid down onto page<BR>images. (Per the semantics of the dirty list, an object whose state is<BR>current on the page image does not belong onthe dirtylist).<BR><BR>-bryan<BR><<BR>------------------------------------------------------- This SF.Net email<BR>is sponsored by xPML, a groundbreaking scripting language that extends<BR>applications into web and mobile media. Attend the live webcast and join<BR>the prime developer group breaking into this new coding territory!<BR><A href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642"><FONT color=#0000ff>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642</FONT></A><BR>_______________________________________________ Jdbm-developer mailing list<BR><A href="mailto:Jdb...@li..."><FONT color=#0000ff>Jdb...@li...</FONT></A><BR><A href="https://lists.sourceforge.net/lists/listinfo/jdbm-developer"><FONT color=#0000ff>https://lists.sourceforge.net/lists/listinfo/jdbm-developer</FONT></A> <<BR>------------------------------------------------------- Using Tomcat but<BR>need to do more? Need to support web services, security? Get stuff done<BR>quickly with pre-integrated technology to make your job easier Download IBM<BR>WebSphere Application Server v.1.0.1 based on Apache Geronimo<BR><A href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642"><FONT color=#0000ff>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642</FONT></A><BR>_______________________________________________ Jdbm-developer mailing list<BR><A href="mailto:Jdb...@li..."><FONT color=#0000ff>Jdb...@li...</FONT></A><BR><A href="https://lists.sourceforge.net/lists/listinfo/jdbm-developer"><FONT color=#0000ff>https://lists.sourceforge.net/lists/listinfo/jdbm-developer</FONT></A> <<BR><BR><BR>------------------------------------------------------- Using Tomcat but<BR>need to do more? Need to support web services, security? Get stuff done<BR>quickly with pre-integrated technology to make your job easier Download IBM<BR>WebSphere Application Server v.1.0.1 based on Apache Geronimo<BR><A href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642"><FONT color=#0000ff>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642</FONT></A><BR>_______________________________________________ Jdbm-developer mailing list<BR><A href="mailto:Jdb...@li..."><FONT color=#0000ff>Jdb...@li...</FONT></A><BR><A href="https://lists.sourceforge.net/lists/listinfo/jdbm-developer"><FONT color=#0000ff>https://lists.sourceforge.net/lists/listinfo/jdbm-developer</FONT></A><BR><BR><BR><<BR></FONT></FONT></TD></TR></TBODY></TABLE></DIV></FONT></BODY></HTML> |
From: Bryan T. <tho...@sp...> - 2006-04-24 19:55:17
|
Kevin, I think that the soft/weak cache classes operate correctly. However, I believe that the manner in which they are being used by the CacheRecordManager is not producing the desired result. In the example that I sent, if you hold a reference to [s] in your application, the cache entry for [s] will still be cleared from the soft/weak cache -- and those are the wrong semantics. -bryan -----Original Message----- From: jdb...@li... [mailto:jdb...@li...]On Behalf Of Kevin Day Sent: Monday, April 24, 2006 3:51 PM To: JDBM Developer listserv Subject: re[6]: [Jdbm-developer] Soft/Weak cache question - it seems like we have a hole in the cache commit behavior. Bryan- mmmm - this implementation is exactly per the jdk docs for the soft/weak reference classes... The implementation is solid as far as I can tell... Can you come up with a unit test that results in the failure mechanism that you are concerned with? - K > Kevin, The objects that are actually inserted into the soft/weak cache are not the application objects directly, but CacheRecordManager.CacheEntry instances. Since the application is not holding references to those CacheEntry instances, they go out of scope as soon as the code exits a method on the CacheRecordManager class. This is counter to the intention for the soft/weak cache. What we need is to drive the dirty flag and serializer references into the SoftReference / WeakReference objects that are held as the cache values. Consider, String s = new String(); long recid = crm.insert(s); CRM does: entry = new CacheEntry( recid, s, serializer, isDirty ); cache.put( recid, entry ); Soft/WeakCache does: internal.put( recid, new Entry( recid, entry ) ); Where Entry extends Soft/WeakReference. So the WeakReference is to the CacheEntry, not to the String [s]. Holding onto a reference to [s] does not effect the retention of the soft/weak cache entry for [s]. The application would actualy have to hold onto the CacheEntry object, which it never sees. We could fix this be expanding the interface API for the CachePolicy to pass along the additional state {dirty flag, serializer} and driving that state into the Weak/SoftReference Entry class used by the Weak/SoftCache class. -bryan -----Original Message----- From: jdb...@li... [mailto:jdb...@li...]On Behalf Of Kevin Day Sent: Monday, April 24, 2006 2:15 PM To: JDBM Developer listserv Subject: re[4]: [Jdbm-developer] Soft/Weak cache question - it seems like we have a hole in the cache commit behavior. Bryan- hmmm - not sure I understand this one... If a weak reference is used, then the object is not garbage collected until all other, non-weak referenced objects have been collected. This means that the cache allows us to retrieve an instantiated form of a given object from the cache, even if all hard references to it have been severed. The MRU size drives when the changes to the object are actually serialized to bytes. The weak reference cache determines whether deserialization is necessary for a given fetch request. I believe that the comments in the weak reference cache implementation describe this in a lot of detail... - K > Because it lets you adjust the degree of retention simply by changing the MRU size. But the point is the same either way, the weak/soft cache entries are inside of the CacheRecordManager CacheEntry objects. Since noone is holding onto the CacheEntry objects, the cache is not doing what we want. -bryan -----Original Message----- From: jdb...@li... [mailto:jdb...@li...]On Behalf Of Kevin Day Sent: Sunday, April 23, 2006 7:18 PM To: JDBM Developer listserv Subject: re[2]: [Jdbm-developer] Soft/Weak cache question - it seems like we have a hole in the cache commit behavior. Bryan- I've never understood why one would want a weakreferencecache... I always use softreference myself... - K > All, CacheRecordManager inserts CacheEntry objects into the weak value cache. Those objects have attributes containing the recid, serializer, dirty flag and a hard reference to the object. The weak value cache will clear these entries as soon as they are no longer strongly reachable. Since the application does not hold onto the CacheEntry objects itself, but only the object reference, it seems that the CacheEntry entries in the weak value cache will be swept by the garbage collector and removed from the weak value cache shortly after they fall off of the internal hard reference MRU. That is, it does not look like the weak value cache is serving any purpose. Thoughts? -bryan -----Original Message----- From: jdb...@li... [mailto:jdb...@li...]On Behalf Of Kevin Day Sent: Monday, April 17, 2006 10:46 AM To: JDBM Developer listserv Subject: re: [Jdbm-developer] Soft/Weak cache question - it seems like we have a hole in the ca che commit behavior. Bryan- I don't think this is an issue. Under the current code, if an object evicts from the cache, and the object has been marked as updated, then it's content gets written to the page and marked dirty. In the case of the weak cache, it is fronted by an MRU cache, so the cache eviction technically happens when the object is evicted from the MRU (this technically results in an earlier page write than is actually needed, but there's no way to capture a weak reference eviction immediately before it occurs, so a compromise is made). - K > All, I am looking over the WeakCache class and I am wondering if the following scenario is possible: The WeakCache has a large number of entries, greater than the size of the internal MRU cache when the transaction commits. This could happenif the application was maintaining its own hard references to those runtime objects, e.g., in some caches or temporary data structures. During the commit the CacheRecordManager will ask for an enumeration of the objects in the cache. The enumeration provided will visit the members of the internal MRU, not those in the weak value cache. Since the weak cache has more objects that can fit on the internal MRU cache, some dirty objects will not be reported to the cache record manager and the transaction will not commit all changes. I think that what we need to do is place the dirty objects onto a linked list whose elements are indexed by a hash table (much like the MRU entries, but without a reordering or eviction policy) and then scan just the list of dirty objects during the commit. Objects evicted from the MRU would be removed from the dirty list as they are laid down onto page images. (Per the semantics of the dirty list, an object whose state is current on the page image does not belong onthe dirtylist). -bryan < ------------------------------------------------------- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642 _______________________________________________ Jdbm-developer mailing list Jdb...@li... https://lists.sourceforge.net/lists/listinfo/jdbm-developer < ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Jdbm-developer mailing list Jdb...@li... https://lists.sourceforge.net/lists/listinfo/jdbm-developer < ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Jdbm-developer mailing list Jdb...@li... https://lists.sourceforge.net/lists/listinfo/jdbm-developer < ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Jdbm-developer mailing list Jdb...@li... https://lists.sourceforge.net/lists/listinfo/jdbm-developer |
From: Kevin D. <ke...@tr...> - 2006-04-24 20:13:22
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> <HTML><HEAD> <STYLE type=text/css> P, UL, OL, DL, DIR, MENU, PRE { margin: 0 auto;}</STYLE> <META content="MSHTML 6.00.2900.2802" name=GENERATOR></HEAD> <BODY leftMargin=1 topMargin=1 rightMargin=1><FONT face=Tahoma> <DIV><FONT face=Arial size=2>Bryan-</FONT></DIV> <DIV><FONT face=Arial size=2></FONT> </DIV> <DIV><FONT face=Arial size=2>If the application holds a reference to an object that is in turn pointed to by a weak reference, then the weak reference will not be purged from the cache.</FONT></DIV> <DIV><FONT face=Arial size=2></FONT> </DIV> <DIV><FONT face=Arial size=2></FONT></DIV> <DIV><FONT face=Arial size=2>Can you put together a stripped down unit test/example that demonstrates the problem behavior?</FONT></DIV> <DIV><FONT face=Arial size=2></FONT> </DIV> <DIV><FONT face=Arial size=2>- K</FONT></DIV> <DIV><FONT face=Arial size=2> </FONT> <TABLE> <TBODY> <TR> <TD width=1 bgColor=blue><FONT face=Arial size=2></FONT></TD> <TD><FONT face=Arial size=2><FONT color=red>> Kevin, <BR> <BR>I think that the soft/weak cache classes operate correctly. However, I believe that the manner in which they are being used by the CacheRecordManager is not producing the desired result. In the example that I sent, if you hold a reference to [s] in your application, the cache entry for [s] will still be cleared from the soft/weak cache -- and those are the wrong semantics. <BR> <BR>-bryan <BR>-----Original Message-----<BR>From: <A href="mailto:jdb...@li..."><FONT color=#0000ff>jdb...@li...</FONT></A> <A href="mailto:jdb...@li...]On"><FONT color=#0000ff>[mailto:jdb...@li...]On</FONT></A> Behalf Of Kevin Day<BR>Sent: Monday, April 24, 2006 3:51 PM<BR>To: JDBM Developer listserv<BR>Subject: re[6]: [Jdbm-developer] Soft/Weak cache question - it seems like we have a hole in the cache commit behavior.<BR><BR><BR>Bryan- <BR> <BR><BR>mmmm - this implementation is exactly per the jdk docs for the soft/weak reference classes... <BR> <BR>The implementation issolid as far as I can tell... <BR> <BR>Can you come up with a unit test that results in the failure mechanism that you are concerned with? <BR> <BR>- K <BR> <BR>>Kevin,<BR><BR>The objects that are actually inserted into the soft/weak cache are not the<BR>application objects directly, but CacheRecordManager.CacheEntry instances.<BR>Since the application is not holding references to those CacheEntry<BR>instances,<BR>they go out of scope as soon as the code exits a method on the<BR>CacheRecordManager<BR>class. This is counter to the intention for the soft/weak cache. What we<BR>need<BR>is to drive the dirty flag and serializer references into the SoftReference<BR>/ WeakReference objects that are held as the cache values.<BR><BR>Consider,<BR><BR>String s = new String();<BR>long recid = crm.insert(s);<BR><BR> CRM does: entry = new CacheEntry( recid, s, serializer, isDirty );<BR> cache.put( recid, entry );<BR><BR> Soft/WeakCache does: internal.put( recid, new Entry( recid, entry ) );<BR><BR>Where Entry extends Soft/WeakReference.<BR><BR>So the WeakReference is to the CacheEntry, not to the String [s]. Holding<BR>onto a reference to [s] does not effect the retention of the soft/weak cache<BR>entry for [s]. The application would actualy have to hold onto the<BR>CacheEntry<BR>object, which it never sees.<BR><BR>We could fix this be expanding the interface API for the CachePolicy to pass<BR>along the additional state {dirty flag, serializer} and driving that state<BR>into the Weak/SoftReference Entry class used by the Weak/SoftCache class.<BR><BR>-bryan<BR><BR>-----Original Message-----<BR>From: <A href="mailto:jdb...@li..."><FONT color=#0000ff>jdb...@li...</FONT></A><BR><A href="mailto:jdb...@li...]On"><FONT color=#0000ff>[mailto:jdb...@li...]On</FONT></A> Behalf Of Kevin Day<BR>Sent: Monday, April 24, 2006 2:15 PM<BR>To: JDBM Developer listserv<BR>Subject: re[4]: [Jdbm-developer] Soft/Weak cache question - it seems like we<BR>have a hole in the cache commit behavior.<BR><BR><BR>Bryan-<BR><BR>hmmm - not sure I understand this one...<BR><BR>If a weak reference is used, then the object is not garbage collected until<BR>all other, non-weak referenced objects have been collected. This means that<BR>the cache allows us to retrieve an instantiated form of a given object from<BR>the cache, even if all hard references to it have been severed.<BR><BR>The MRU size drives when the changes to the object are actually serialized<BR>to bytes. The weak reference cache determines whether deserialization is<BR>necessary for a given fetch request.<BR><BR>I believe that the comments in the weak reference cache implementation<BR>describe this in a lot of detail...<BR><BR>- K<BR>> Because it lets you adjust the degree of retention simply by changing<BR>the MRU size. But the point is the same<BR>either way, the weak/soft cache entries are inside of the<BR>CacheRecordManager CacheEntry objects. Since noone<BR>is holding onto the CacheEntry objects, the cache is not doing what we<BR>want.<BR>-bryan<BR>-----Original Message-----<BR>From: <A href="mailto:jdb...@li..."><FONT color=#0000ff>jdb...@li...</FONT></A><BR><A href="mailto:jdb...@li...]On"><FONT color=#0000ff>[mailto:jdb...@li...]On</FONT></A> Behalf Of Kevin Day<BR>Sent: Sunday, April 23, 2006 7:18 PM<BR>To: JDBM Developer listserv<BR>Subject: re[2]: [Jdbm-developer] Soft/Weak cache question - it seems like<BR>we have a hole in the cache commit behavior.<BR><BR><BR>Bryan-<BR><BR><BR>I've never understood why one would want a weakreferencecache... I always<BR>use softreference myself...<BR><BR>- K<BR><BR>><BR>All,<BR><BR>CacheRecordManager inserts CacheEntry objects into the weak value cache.<BR>Those objects have attributes<BR>containing the recid, serializer, dirty flag and a hard reference to the<BR>object. The weak value cache will clear<BR>these entries as soon as they are no longer strongly reachable. Since the<BR>application does not hold onto the<BR>CacheEntry objects itself, but only the object reference, it seems that the<BR>CacheEntry entries in the weak value<BR>cache will be swept by the garbage collector and removed from the weak<BR>value cache shortly after they fall off of<BR>the internal hard reference MRU. That is, it does not look like the weak<BR>value cache is serving any purpose.<BR><BR>Thoughts?<BR><BR>-bryan<BR>-----Original Message-----<BR>From: <A href="mailto:jdb...@li..."><FONT color=#0000ff>jdb...@li...</FONT></A><BR><A href="mailto:jdb...@li...]On"><FONT color=#0000ff>[mailto:jdb...@li...]On</FONT></A> Behalf Of Kevin Day<BR>Sent: Monday, April 17, 2006 10:46 AM<BR>To: JDBM Developer listserv<BR>Subject: re: [Jdbm-developer] Soft/Weak cache question - it seems like we<BR>have a hole in the ca che commit behavior.<BR><BR><BR>Bryan-<BR><BR><BR>I don't think this is an issue. Under the current code, if an object<BR>evicts from the cache, and the object has been marked as updated, then<BR>it's content gets written to the page and marked dirty. In the case of<BR>the weak cache, it is fronted by an MRU cache, so the cache eviction<BR>technically happens when the object is evicted from the MRU (this<BR>technically results in an earlier page write than is actually needed, but<BR>there's no way to capture a weak reference eviction immediately before it<BR>occurs, so a compromise is made).<BR><BR><BR>- K<BR><BR>><BR>All,<BR><BR>I am looking over the WeakCache class and I am wondering if the following<BR>scenario is possible: The WeakCache has a large number of entries, greater<BR>than the size of the internal MRU cache when the transaction commits.<BR>This could happenif the application was maintaining its own hard<BR>references to those runtime objects, e.g., in some caches or temporary<BR>data structures. During the commit the CacheRecordManager will ask for<BR>an enumeration of the objects in the cache. The enumeration provided will<BR>visit the members of the internal MRU, not those in the weak value cache.<BR>Since the weak cache has more objects that can fit on the internal MRU<BR>cache, some dirty objects will not be reported to the cache record manager<BR>and the transaction will not commit all changes.<BR><BR>I think that what we need to do is place the dirty objects onto a linked<BR>list whose elements are indexed by a hash table (much like the MRU<BR>entries, but without a reordering or eviction policy) and then scan just<BR>the list of dirty objects during the commit. Objects evicted from the MRU<BR>would be removed from the dirty list as they are laid down onto page<BR>images. (Per the semantics of the dirty list, an object whose state is<BR>current on the page image does not belong onthe dirtylist).<BR><BR>-bryan<BR><<BR>------------------------------------------------------- This SF.Net email<BR>is sponsored by xPML, a groundbreaking scripting language that extends<BR>applications into web and mobile media. Attend the live webcast and join<BR>the prime developer group breaking into this new coding territory!<BR><A href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642"><FONT color=#0000ff>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642</FONT></A><BR>_______________________________________________ Jdbm-developer mailing list<BR><A href="mailto:Jdb...@li..."><FONT color=#0000ff>Jdb...@li...</FONT></A><BR><A href="https://lists.sourceforge.net/lists/listinfo/jdbm-developer"><FONT color=#0000ff>https://lists.sourceforge.net/lists/listinfo/jdbm-developer</FONT></A> <<BR>------------------------------------------------------- Using Tomcat but<BR>need to do more? Need to support web services, security? Get stuff done<BR>quickly with pre-integrated technology to make your job easier Download IBM<BR>WebSphere Application Server v.1.0.1 based on Apache Geronimo<BR><A href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642"><FONT color=#0000ff>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642</FONT></A><BR>_______________________________________________ Jdbm-developer mailing list<BR><A href="mailto:Jdb...@li..."><FONT color=#0000ff>Jdb...@li...</FONT></A><BR><A href="https://lists.sourceforge.net/lists/listinfo/jdbm-developer"><FONT color=#0000ff>https://lists.sourceforge.net/lists/listinfo/jdbm-developer</FONT></A> <<BR><BR><BR>------------------------------------------------------- Using Tomcat but<BR>need to do more? Need to support web services, security? Get stuff done<BR>quickly with pre-integrated technology to make your job easier Download IBM<BR>WebSphere Application Server v.1.0.1 based on Apache Geronimo<BR><A href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642"><FONT color=#0000ff>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642</FONT></A><BR>_______________________________________________ Jdbm-developer mailing list<BR><A href="mailto:Jdb...@li..."><FONT color=#0000ff>Jdb...@li...</FONT></A><BR><A href="https://lists.sourceforge.net/lists/listinfo/jdbm-developer"><FONT color=#0000ff>https://lists.sourceforge.net/lists/listinfo/jdbm-developer</FONT></A><BR><BR><BR><<BR>------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo <A href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642"><FONT color=#0000ff>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642</FONT></A> _______________________________________________ Jdbm-developer mailing list <A href="mailto:Jdb...@li..."><FONT color=#0000ff>Jdb...@li...</FONT></A> <A href="https://lists.sourceforge.net/lists/listinfo/jdbm-developer"><FONT color=#0000ff>https://lists.sourceforge.net/lists/listinfo/jdbm-developer</FONT></A> <<BR></FONT></FONT></TD></TR></TBODY></TABLE></DIV></FONT></BODY></HTML> |
From: Bryan T. <tho...@sp...> - 2006-04-24 20:42:11
|
Since the only hard reference is from the CacheRecordManager.CacheEntry, it will not cause the application object to be retained. There is an extra indirection which is creating the problem. I'll look into creating a test case. -bryan -----Original Message----- From: jdb...@li... [mailto:jdb...@li...]On Behalf Of Kevin Day Sent: Monday, April 24, 2006 4:17 PM To: JDBM Developer listserv Subject: re[8]: [Jdbm-developer] Soft/Weak cache question - it seems like we have a hole in the cache commit behavior. Bryan- If the application holds a reference to an object that is in turn pointed to by a weak reference, then the weak reference will not be purged from the cache. Can you put together a stripped down unit test/example that demonstrates the problem behavior? - K > Kevin, I think that the soft/weak cache classes operate correctly. However, I believe that the manner in which they are being used by the CacheRecordManager is not producing the desired result. In the example that I sent, if you hold a reference to [s] in your application, the cache entry for [s] will still be cleared from the soft/weak cache -- and those are the wrong semantics. -bryan -----Original Message----- From: jdb...@li... [mailto:jdb...@li...]On Behalf Of Kevin Day Sent: Monday, April 24, 2006 3:51 PM To: JDBM Developer listserv Subject: re[6]: [Jdbm-developer] Soft/Weak cache question - it seems like we have a hole in the cache commit behavior. Bryan- mmmm - this implementation is exactly per the jdk docs for the soft/weak reference classes... The implementation issolid as far as I can tell... Can you come up with a unit test that results in the failure mechanism that you are concerned with? - K >Kevin, The objects that are actually inserted into the soft/weak cache are not the application objects directly, but CacheRecordManager.CacheEntry instances. Since the application is not holding references to those CacheEntry instances, they go out of scope as soon as the code exits a method on the CacheRecordManager class. This is counter to the intention for the soft/weak cache. What we need is to drive the dirty flag and serializer references into the SoftReference / WeakReference objects that are held as the cache values. Consider, String s = new String(); long recid = crm.insert(s); CRM does: entry = new CacheEntry( recid, s, serializer, isDirty ); cache.put( recid, entry ); Soft/WeakCache does: internal.put( recid, new Entry( recid, entry ) ); Where Entry extends Soft/WeakReference. So the WeakReference is to the CacheEntry, not to the String [s]. Holding onto a reference to [s] does not effect the retention of the soft/weak cache entry for [s]. The application would actualy have to hold onto the CacheEntry object, which it never sees. We could fix this be expanding the interface API for the CachePolicy to pass along the additional state {dirty flag, serializer} and driving that state into the Weak/SoftReference Entry class used by the Weak/SoftCache class. -bryan -----Original Message----- From: jdb...@li... [mailto:jdb...@li...]On Behalf Of Kevin Day Sent: Monday, April 24, 2006 2:15 PM To: JDBM Developer listserv Subject: re[4]: [Jdbm-developer] Soft/Weak cache question - it seems like we have a hole in the cache commit behavior. Bryan- hmmm - not sure I understand this one... If a weak reference is used, then the object is not garbage collected until all other, non-weak referenced objects have been collected. This means that the cache allows us to retrieve an instantiated form of a given object from the cache, even if all hard references to it have been severed. The MRU size drives when the changes to the object are actually serialized to bytes. The weak reference cache determines whether deserialization is necessary for a given fetch request. I believe that the comments in the weak reference cache implementation describe this in a lot of detail... - K > Because it lets you adjust the degree of retention simply by changing the MRU size. But the point is the same either way, the weak/soft cache entries are inside of the CacheRecordManager CacheEntry objects. Since noone is holding onto the CacheEntry objects, the cache is not doing what we want. -bryan -----Original Message----- From: jdb...@li... [mailto:jdb...@li...]On Behalf Of Kevin Day Sent: Sunday, April 23, 2006 7:18 PM To: JDBM Developer listserv Subject: re[2]: [Jdbm-developer] Soft/Weak cache question - it seems like we have a hole in the cache commit behavior. Bryan- I've never understood why one would want a weakreferencecache... I always use softreference myself... - K > All, CacheRecordManager inserts CacheEntry objects into the weak value cache. Those objects have attributes containing the recid, serializer, dirty flag and a hard reference to the object. The weak value cache will clear these entries as soon as they are no longer strongly reachable. Since the application does not hold onto the CacheEntry objects itself, but only the object reference, it seems that the CacheEntry entries in the weak value cache will be swept by the garbage collector and removed from the weak value cache shortly after they fall off of the internal hard reference MRU. That is, it does not look like the weak value cache is serving any purpose. Thoughts? -bryan -----Original Message----- From: jdb...@li... [mailto:jdb...@li...]On Behalf Of Kevin Day Sent: Monday, April 17, 2006 10:46 AM To: JDBM Developer listserv Subject: re: [Jdbm-developer] Soft/Weak cache question - it seems like we have a hole in the ca che commit behavior. Bryan- I don't think this is an issue. Under the current code, if an object evicts from the cache, and the object has been marked as updated, then it's content gets written to the page and marked dirty. In the case of the weak cache, it is fronted by an MRU cache, so the cache eviction technically happens when the object is evicted from the MRU (this technically results in an earlier page write than is actually needed, but there's no way to capture a weak reference eviction immediately before it occurs, so a compromise is made). - K > All, I am looking over the WeakCache class and I am wondering if the following scenario is possible: The WeakCache has a large number of entries, greater than the size of the internal MRU cache when the transaction commits. This could happenif the application was maintaining its own hard references to those runtime objects, e.g., in some caches or temporary data structures. During the commit the CacheRecordManager will ask for an enumeration of the objects in the cache. The enumeration provided will visit the members of the internal MRU, not those in the weak value cache. Since the weak cache has more objects that can fit on the internal MRU cache, some dirty objects will not be reported to the cache record manager and the transaction will not commit all changes. I think that what we need to do is place the dirty objects onto a linked list whose elements are indexed by a hash table (much like the MRU entries, but without a reordering or eviction policy) and then scan just the list of dirty objects during the commit. Objects evicted from the MRU would be removed from the dirty list as they are laid down onto page images. (Per the semantics of the dirty list, an object whose state is current on the page image does not belong onthe dirtylist). -bryan < ------------------------------------------------------- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642 _______________________________________________ Jdbm-developer mailing list Jdb...@li... https://lists.sourceforge.net/lists/listinfo/jdbm-developer < ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Jdbm-developer mailing list Jdb...@li... https://lists.sourceforge.net/lists/listinfo/jdbm-developer < ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Jdbm-developer mailing list Jdb...@li... https://lists.sourceforge.net/lists/listinfo/jdbm-developer < ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Jdbm-developer mailing list Jdb...@li... https://lists.sourceforge.net/lists/listinfo/jdbm-developer < ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Jdbm-developer mailing list Jdb...@li... https://lists.sourceforge.net/lists/listinfo/jdbm-developer |
From: Alex B. <boi...@in...> - 2006-04-24 16:56:55
|
The intent of the weak cache is application-supported caching. If your application already keeps (hard) references to specific objects, then JDBM will be able to provide those same objects back to the application instead of reloading them. alex Bryan Thompson wrote: > All, > > =20 > > CacheRecordManager inserts CacheEntry objects into the weak value > cache. Those objects have attributes > > containing the recid, serializer, dirty flag and a hard reference to > the object. The weak value cache will clear > > these entries as soon as they are no longer strongly reachable. Since > the application does not hold onto the > > CacheEntry objects itself, but only the object reference, it seems > that the CacheEntry entries in the weak value > > cache will be swept by the garbage collector and removed from the weak > value cache shortly after they fall off of > > the internal hard reference MRU. That is, it does not look like the > weak value cache is serving any purpose. > > =20 > > Thoughts? > > =20 > > -bryan > > -----Original Message----- > *From:* jdb...@li... > [mailto:jdb...@li...]*On Behalf Of > *Kevin Day > *Sent:* Monday, April 17, 2006 10:46 AM > *To:* JDBM Developer listserv > *Subject:* re: [Jdbm-developer] Soft/Weak cache question - it > seems like we have a hole in the ca che commit behavior. > > Bryan- > =20 > I don't think this is an issue. Under the current code, if an > object evicts from the cache, and the object has been marked as > updated, then it's content gets written to the page and marked > dirty. In the case of the weak cache, it is fronted by an MRU > cache, so the cache eviction technically happens when the object > is evicted from the MRU (this technically results in an earlier > page write than is actually needed, but there's no way to capture > a weak reference eviction immediately before it occurs, so a > compromise is made). > =20 > =20 > - K > =20 > > > > All, > =20 > I am looking over the WeakCache class and I am wondering if the > following scenario is possible: The WeakCache has a large number > of entries, greater than the size of the internal MRU cache when > the transaction commits. This could happenif the application was > maintaining its own hard references to those runtime objects, > e.g., in some caches or temporary data structures. During the > commit the CacheRecordManager will ask for an enumeration of the > objects in the cache. The enumeration provided will visit the > members of the internal MRU, not those in the weak value cache. > Since the weak cache has more objects that can fit on the > internal MRU cache, some dirty objects will not be reported to the > cache record manager and the transaction will not commit all = changes. > =20 > I think that what we need to do is place the dirty objects onto a > linked list whose elements are indexed by a hash table (much like > the MRU entries, but without a reordering or eviction policy) and > then scan just the list of dirty objects during the commit. > Objects evicted from the MRU would be removed from the dirty list > as they are laid down onto page images. (Per the semantics of the > dirty list, an object whose state is current on the page image > does not belong on the dirty list). > =20 > -bryan > < > > ------------------------------------------------------- This > SF.Net email is sponsored by xPML, a groundbreaking scripting > language that extends applications into web and mobile media. > Attend the live webcast and join the prime developer group > breaking into this new coding territory! > = http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D110944&bid=3D241720&dat=3D= 121642 > _______________________________________________ Jdbm-developer > mailing list Jdb...@li... > https://lists.sourceforge.net/lists/listinfo/jdbm-developer=20 > |