From: Xinwei X. <xi...@cs...> - 2009-04-06 01:54:37
|
Hi everyone, I have some problems with my instrument codes in baseline compiler. What I want to do is to insert some instrument codes into the update of pointers, such as putstatic, putfiled, etc. Actually I need to store into and remove some information from a data structure such as HashMap on the fly, as I need its information later. However, I observed that it would trigger GC when there isn't enough memory to allocate a new pair of <K,V>. It will rase some exception or even quit abnormally. It is very intuitive as I have to use "new" to new my <K,V> item in my instrument code, I can't make it uninterruptible. So rvm will allow GC while my code is being executed I check out the way GCspy uses in its server side. Using the C code, it allocates its primitive arrays to store space informations. However, if I want to use such data structure: HashMap<Integer, HashSet>, I've no idea whether I can just use malloc to create a new trunk to store "Object" object, as a lot of hashmap implementations are based on array. So how can i accomplish my idea while using HashMap without violating the uninterruptible rule? Even if I preallocate some trunk, it will run out eventually. At that time, the GC will be triggered anyway. Any suggestion is welcomed, thanks for your attention. Hopefully my question is clear enough. Regards, ----- Xinwei Xie School of Computer Science and Engineering University of New South Wales Sydney, Australia -- View this message in context: http://www.nabble.com/uninterruptible-implementation-while-using-new--tp22901101p22901101.html Sent from the jikesrvm-researchers mailing list archive at Nabble.com. |
From: Robin G. <rob...@an...> - 2009-04-06 03:59:21
|
When I do things like this, I do it in Java using uninterruptible code: - either allocate a fixed size data structure in raw memory (create a RawPageSpace for the purpose) or use static scalar data structures allocated at build time in the boot image) - Use magic types (Address, Offset etc) to manipulate the data structures It's definitely not as convenient as writing in plain Java, but (IMO) it's easier than writing in C. As a starting point, you might like to look at the class org.mmtk.utility.SimpleHashTable, which implements an extensible hash table structure in raw memory. org.mmtk.utility.sanity.SanityDataTable extends this as the basis of the sanity checker's data structures. You could either extend SimpleHashTable or just use it as an example. And BTW, it's not just allocation you need to worry about - if you're instrumenting write barriers, you must also avoid writing to pointer fields. Regards, Robin Xinwei Xie wrote: > Hi everyone, > > I have some problems with my instrument codes in baseline compiler. What I > want to do is to insert some instrument codes into the update of pointers, > such as putstatic, putfiled, etc. Actually I need to store into and remove > some information from a data structure such as HashMap on the fly, as I need > its information later. However, I observed that it would trigger GC when > there isn't enough memory to allocate a new pair of <K,V>. It will rase some > exception or even quit abnormally. It is very intuitive as I have to use > "new" to new my <K,V> item in my instrument code, I can't make it > uninterruptible. So rvm will allow GC while my code is being executed > > I check out the way GCspy uses in its server side. Using the C code, it > allocates its primitive arrays to store space informations. However, if I > want to use such data structure: HashMap<Integer, HashSet>, I've no idea > whether I can just use malloc to create a new trunk to store "Object" > object, as a lot of hashmap implementations are based on array. > > So how can i accomplish my idea while using HashMap without violating the > uninterruptible rule? Even if I preallocate some trunk, it will run out > eventually. At that time, the GC will be triggered anyway. > > Any suggestion is welcomed, thanks for your attention. Hopefully my question > is clear enough. > > Regards, > > ----- > Xinwei Xie > School of Computer Science and Engineering > University of New South Wales > Sydney, Australia > |
From: Xinwei X. <xi...@cs...> - 2009-04-08 00:19:56
|
Hi Robin, Thanks for your reply. IMO, in SimpleHashTable, it uses Address, Extent etc. to manipulate the raw memory straightforward. However, as you mentioned, it can only allocate the fixed size data structure. What if I want to allocate a data structure which is dynamiclly resized and insert it into the "container", such as map or set, whatever, is it still feasible? Nevertheless, maybe I don't have to have my code uninterruptible. If it is interruptible of my instrument code, can I just yield to some GC point explicitly when I want to new my dynamically resized data structure and resume when that is ready? Regards, Xinwei Robin Garner wrote: > > When I do things like this, I do it in Java using uninterruptible code: > - either allocate a fixed size data structure in raw memory (create a > RawPageSpace for the purpose) or use static scalar data structures > allocated at build time in the boot image) > - Use magic types (Address, Offset etc) to manipulate the data structures > > It's definitely not as convenient as writing in plain Java, but (IMO) > it's easier than writing in C. > > As a starting point, you might like to look at the class > org.mmtk.utility.SimpleHashTable, which implements an extensible hash > table structure in raw memory. org.mmtk.utility.sanity.SanityDataTable > extends this as the basis of the sanity checker's data structures. You > could either extend SimpleHashTable or just use it as an example. > > And BTW, it's not just allocation you need to worry about - if you're > instrumenting write barriers, you must also avoid writing to pointer > fields. > > Regards, > Robin > -- View this message in context: http://www.nabble.com/uninterruptible-implementation-while-using-new--tp22901101p22940884.html Sent from the jikesrvm-researchers mailing list archive at Nabble.com. |
From: Robin G. <rob...@an...> - 2009-04-08 04:08:26
|
Xinwei Xie wrote: > Hi Robin, > > Thanks for your reply. > > IMO, in SimpleHashTable, it uses Address, Extent etc. to manipulate the raw > memory straightforward. However, as you mentioned, it can only allocate the > fixed size data structure. What if I want to allocate a data structure which > is dynamiclly resized and insert it into the "container", such as map or > set, whatever, is it still feasible? > As long as you do your own allocation of objects and don't involve the GC, you can do whatever you like - the Deque structures that MMTk uses are a fairly simple example of dynamic data structures in raw memory. Of course if you're talking java.util.* classes - no, you won't be able to use them. > Nevertheless, maybe I don't have to have my code uninterruptible. If it is > interruptible of my instrument code, can I just yield to some GC point > explicitly when I want to new my dynamically resized data structure and > resume when that is ready? > It depends entirely on where you are inserting instrumentation. I'm assuming since you are running into this issue that you are inserting it in a write barrier or equivalent - if you can do your instrumentation in a way that doesn't require breaking out of uninterruptible code (eg bytecode rewriting), you can do whatever you like. On the other hand, once you are inside uninterruptible code, allowing GC to occur puts you in a world of pain. You can't just break out into standard java with @Interruptible - it's uninterruptible for a reason. In a write barrier, you need to ensure that you don't trigger any write barriers, severely curtailing what you can do. MMTk itself uses the 'asynchronous collection' mechanism to avoid performing a GC inside a write barrier. You *can* allow GCs during uninterruptible code (the ReferenceProcessor code does this), but it's difficult to get right and you need a pretty deep understanding of JikesRVM to do it successfully. If you really need to use dynamic data structures, one possible solution might be to use raw memory as a 'mailbox' to store data temporarily, and have a thread periodically empty out the mailbox and update your data structure. You could also potentially use one of the ExplicitFreeList spaces to allocate your dynamic structures, and don't let MMTk account for them. You can use MemoryManager.pickAllocator (or the classloader method it delegates to) to force your data structures into that space, and use @UninterruptibleNoWarn for the code that allocates. While this *may* work, I think it's probably a very difficult way to proceed, and you won't be able to use much library code. Another solution (and what I'd probably do) would be to write your data out to a file (or standard out/err), and do the analysis offline. cheers, Robin > Regards, > Xinwei > > > Robin Garner wrote: > >> When I do things like this, I do it in Java using uninterruptible code: >> - either allocate a fixed size data structure in raw memory (create a >> RawPageSpace for the purpose) or use static scalar data structures >> allocated at build time in the boot image) >> - Use magic types (Address, Offset etc) to manipulate the data structures >> >> It's definitely not as convenient as writing in plain Java, but (IMO) >> it's easier than writing in C. >> >> As a starting point, you might like to look at the class >> org.mmtk.utility.SimpleHashTable, which implements an extensible hash >> table structure in raw memory. org.mmtk.utility.sanity.SanityDataTable >> extends this as the basis of the sanity checker's data structures. You >> could either extend SimpleHashTable or just use it as an example. >> >> And BTW, it's not just allocation you need to worry about - if you're >> instrumenting write barriers, you must also avoid writing to pointer >> fields. >> >> Regards, >> Robin >> >> > > |