Menu
▾
▴
Re: [rvm-research] Separation of Spaces
|
From: Robin G. <rob...@gm...> - 2020-01-16 00:55:00
|
>> but this sounds like you're OK with having a corrupt heap
>
>The corrupt part should be totally separated if possible, so the rest
of the heap is not affected. I know it is not best practice, but if it
is achievable, it would be controlled corrupt :D
>
>> Perhaps you could elaborate a little more on your use case ?
>> When you say "special objects", in what way are they special ?
>
>A Java Agent creates objects for debugging purposes, these are GC root
referenced, but don't have any references to other objects. In this way
there should be no connection between the space they are allocated in
and the rest of >the heap.
OK, I'm getting a better idea of what you're trying to do. Not sure
it's a great idea, but what would I know :)
Maybe what you need to do is to exclude your roots from the root set ?
Might be harder than it sounds.
I'm thinking that implementing traceObject(obj) in your Space as simply
'return obj' should do it, but I assume that's what you're doing.
> Can you tell me what gray objects are?
This is a reference to the so-called tri-colour abstraction from
Dijkkstra, Lamport et al (1978). Gray objects are ones that you know to
be reachable, but that haven't been scanned: in MMTk, these are the
objects on the 'values' queue. 'processing gray objects' is when the GC
does its transitive closure over the heap. This message comes from the
method TraceLocal.completeTrace(). Your crash will be happening when
Jikes RVM attempts to scan an object in your space. You could try
adding some code to this method to report when it sees objects that are
in your space ?
Objects get transferred from the root set to the 'values' queue via the
traceObject method in a Space.
In ImmortalSpace, this method looks like
> @Override
> @Inline
> public ObjectReference traceObject(TransitiveClosure trace,
> ObjectReference object) {
> if (testAndMark(object, markState))
> trace.processNode(object);
> return object;
> }
but if you make yours something like
> @Override
> @Inline
> public ObjectReference traceObject(TransitiveClosure trace,
> ObjectReference object) {
> return object;
> }
then objects in your space won't be enqueued for scanning.
If this doesn't get you any closer, then send me a patch and I'll see if
I can see what's up.
HTH,
Robin
On 16/1/20 4:02 am, Johannes Sinsel wrote:
> > but this sounds like you're OK with having a corrupt heap
>
> The corrupt part should be totally separated if possible, so the rest
> of the heap is not affected. I know it is not best practice, but if it
> is achievable, it would be controlled corrupt :D
>
> > Perhaps you could elaborate a little more on your use case ?
> > When you say "special objects", in what way are they special ?
>
> A Java Agent creates objects for debugging purposes, these are GC root
> referenced, but don't have any references to other objects. In this
> way there should be no connection between the space they are allocated
> in and the rest of the heap.
>
> > If you run with -X:gc:verbose=3 (or above) you should at least be
> able to tell whether it's the GC or the mutator that's crashing.
> -X:gc:verbose=9 will help narrow it down to a particular GC phase.
>
> From these options I got the idea that the crash always happens at a
> full heap collection.
> On -X:gc:verbose=9, before the error message I added in the last mail,
> the terminal always writes something like this:
>
> Execute SimplePhase(closure) as Collector...
> Thread 10: Processing GC in parallel
> Thread 10: processing delayed root objects
> Thread 10: processing gray objects
>
> Can you tell me what gray objects are?
>
> > The Harness doesn't run real code: it runs a minimal language
> designed for exercising MMTk. There's nothing stopping you use it to
> write a test case that (for example) creates two objects, one in the
> 'main' space, pointing to a second object in your RBSpace, then run a
> GC and sanity-check the heap.
>
> Ok, I will check, if I can create something that fits my conditions.
>
> Thanks for your patience.
> Best,
> Johannes
>
> On 15.01.20 05:01, Robin Garner wrote:
>> > > And I assume you are interested in which objects are live, so
>> that you don't overwrite live objects ?
>> >
>> > In fact at this point I don't need this information, best case for
>> me would be to not determine their liveliness at all.
>>
>> OK, perhaps I'm missing something here, but this sounds like you're
>> OK with having a corrupt heap ... Jikes RVM will be less happy :)
>> Perhaps you could elaborate a little more on your use case ?
>>
>> Bottom line, if there is a reference from a live Java object to an
>> object in your space, you need to ensure it is alive, and that the
>> transitive closure of the objects it refers to are also kept alive.
>>
>> > > where does the VM crash ? Do you have a stack trace ?
>> >
>> > Unfortunately I can't get a stack trace, I think this is due to the
>> attempt to trace an object that was already overwritten. The terminal
>> output is something like this:
>>
>> If you run with -X:gc:verbose=3 (or above) you should at least be
>> able to tell whether it's the GC or the mutator that's crashing.
>> -X:gc:verbose=9 will help narrow it down to a particular GC phase.
>>
>> > > Can you reproduce this in the MMTk harness ?
>> >
>> > The "special objects" that are allocated in the RBSpace are created
>> by a Java Agent, so it is difficult for me to use the harness for
>> debugging.
>>
>> The Harness doesn't run real code: it runs a minimal language
>> designed for exercising MMTk. There's nothing stopping you use it to
>> write a test case that (for example) creates two objects, one in the
>> 'main' space, pointing to a second object in your RBSpace, then run a
>> GC and sanity-check the heap.
>>
>> When you say "special objects", in what way are they special ? From
>> the JVM's point of view, any object created via 'new' is pretty much
>> the same as any other.
>>
>>
>> cheers,
>> Robin
>>
>> On 15/1/20 2:25 am, Johannes Sinsel wrote:
>>> > This sounds like your problem. "Garbage Collection" isn't a very
>>> precise term when we get down to this level of detail :)
>>>
>>> Sorry for the vague term :D
>>>
>>> > And I assume you are interested in which objects are live, so
>>> that you don't overwrite live objects ?
>>>
>>> In fact at this point I don't need this information, best case for
>>> me would be to not determine their liveliness at all.
>>>
>>> > where does the VM crash ? Do you have a stack trace ?
>>>
>>> Unfortunately I can't get a stack trace, I think this is due to the
>>> attempt to trace an object that was already overwritten. The
>>> terminal output is something like this:
>>>
>>> JikesRVM: Failing instruction starting at 0 wasn't in RVM address space
>>> JikesRVM: unexpected hardware trap outside of RVM address space -
>>> (nil) 0xe00ffc37298
>>> fault address (nil)
>>> eip (nil)
>>> eax (T0) 0xe00fff5ce20
>>> ebx (ctrs) 0xe00fff5ce20
>>> ecx (S0) 0xc
>>> edx (T1) 0xe00ffc189e8
>>> esi (TR) 0xe00ffc37298
>>> edi (S1) 0xe00fff59578
>>> ebp 0x30
>>> esp (SP) 0xa00ffd30bb8
>>> r8 0xda7
>>> r9 0x1
>>> r10 (nil)
>>> r11 0x206
>>> r12 0xa00ffd31008
>>> r13 0x200029f2c38
>>> r14 0xe00ffc37298
>>> r15 0x20000080510
>>> cs, gs, fs 0x2b000000000033
>>> trapno 0x0000000e
>>> err 0x00000014
>>> eflags 0x00010207
>>> fpregs 0x7fd4580046c0
>>> attempting to dump proc map ...
>>>
>>> > Can you reproduce this in the MMTk harness ?
>>>
>>> The "special objects" that are allocated in the RBSpace are created
>>> by a Java Agent, so it is difficult for me to use the harness for
>>> debugging.
>>>
>>> > I'm guessing maybe you've missed an occurrence of immortalSpace or
>>> its per-thread components ?
>>> > Another debugging approach would be to add an unmodified version
>>> of ImmortalSpace to the plan in place of your modified space. Once
>>> you can verify that objects are being allocated and "collected" in
>>> the unmodified space, you'll know whether the integration of the
>>> space is correct, and can then turn to debugging your new space.
>>> > Also, while I'm aware you've modified the base plan classes,
>>> choice of collector for debugging is quite important. I would start
>>> debugging with MarkSweep, then move to SemiSpace, Immix and
>>> eventually to CopyMS and then the generational collectors.
>>>
>>> I'm quite sure the space is working in combination with all
>>> collectors, I verified this with a simple test class and
>>> VM.assertions.fail() conditions in the RBSpace.
>>>
>>> Thanks again!
>>> Best,
>>> Johannes
>>>
>>> On 14.01.20 00:53, Robin Garner wrote:
>>>> > I assumed that there is no garbage collection in an immortal
>>>> space in the first place and therefore also no tracing since
>>>> objects are always live.
>>>>
>>>> This sounds like your problem. "Garbage Collection" isn't a very
>>>> precise term when we get down to this level of detail :)
>>>>
>>>> It's true that objects in an immortal space are never reclaimed,
>>>> but they do participate in the transitive closure operation. And I
>>>> assume you are interested in which objects are live, so that you
>>>> don't overwrite live objects ?
>>>>
>>>>
>>>> > I added a modified version of an ImmortalSpace (called RBSpace,
>>>> short for RingBufferSpace) to the regular ones in Plan.
>>>>
>>>> I'm guessing maybe you've missed an occurrence of immortalSpace or
>>>> its per-thread components ?
>>>>
>>>> > > I assume you've done some of this, because by default the
>>>> traceObject method will crash the VM if it tries to visit an object
>>>> that hasn't got a special case.
>>>> >
>>>> > That's exactly where I have problems. A special case is added for
>>>> the RBSpace, but the VM crashes no matter what I return.
>>>>
>>>> The contract of traceObject is that it returns the address of the
>>>> traced object (the new address if the object was moved).
>>>>
>>>> where does the VM crash ? Do you have a stack trace ?
>>>>
>>>> Can you reproduce this in the MMTk harness ? It's a much more
>>>> friendly debugging environment.
>>>>
>>>>
>>>> Another debugging approach would be to add an unmodified version of
>>>> ImmortalSpace to the plan in place of your modified space. Once you
>>>> can verify that objects are being allocated and "collected" in the
>>>> unmodified space, you'll know whether the integration of the space
>>>> is correct, and can then turn to debugging your new space.
>>>>
>>>>
>>>> Also, while I'm aware you've modified the base plan classes, choice
>>>> of collector for debugging is quite important. I would start
>>>> debugging with MarkSweep, then move to SemiSpace, Immix and
>>>> eventually to CopyMS and then the generational collectors.
>>>>
>>>>
>>>> Lots to think about, I hope some of it is useful.
>>>> cheers
>>>>
>>>> On 14/1/20 1:11 am, Johannes Sinsel wrote:
>>>>> Hey Robin,
>>>>>
>>>>> thanks for your reply.
>>>>> In my current setup, I added a modified version of an
>>>>> ImmortalSpace (called RBSpace, short for RingBufferSpace) to the
>>>>> regular ones in Plan. Through MemoryManager.pickAllocator()
>>>>> specific objects get to be allocated in it. Problem number 2 you
>>>>> mentioned, does not arise since these objects won't point into the
>>>>> rest of the heap.
>>>>> As the RBSpace has a ring-buffer fashioned memory management,
>>>>> objects can be overwritten and therefore they are not live
>>>>> anymore. But by now the liveness information does not matter to me
>>>>> (Problem 1).
>>>>>
>>>>> > I assume you've done some of this, because by default the
>>>>> traceObject method will crash the VM if it tries to visit an
>>>>> object that hasn't got a special case.
>>>>>
>>>>> That's exactly where I have problems. A special case is added for
>>>>> the RBSpace, but the VM crashes no matter what I return.
>>>>>
>>>>> I assumed that there is no garbage collection in an immortal space
>>>>> in the first place and therefore also no tracing since objects are
>>>>> always live.
>>>>>
>>>>> Thanks again for your help.
>>>>> Kind regards,
>>>>> Johannes
>>>>>
>>>>> On 13.01.20 06:02, Robin Garner wrote:
>>>>>> Hi Johannes,
>>>>>>
>>>>>> I'm not quite clear on what you're trying to do.
>>>>>>
>>>>>> > I'm trying to separate a few objects in the heap, so the
>>>>>> garbage collection doesn't check them
>>>>>>
>>>>>> In a full heap collection, the garbage collector determines the
>>>>>> liveness of objects in the heap by doing a transitive closure
>>>>>> over the heap, starting with the roots (thread stacks and static
>>>>>> fields). If you somehow arrange for part of the heap to not be
>>>>>> checked during GC, then you have two problems:
>>>>>>
>>>>>> 1) Determining the liveness of the objects in your 'ignored'
>>>>>> space, and
>>>>>>
>>>>>> 2) Keeping objects in the rest of the heap that your 'ignored'
>>>>>> space objects point to alive.
>>>>>>
>>>>>> When adding a new space to a Plan, your problem is less one of
>>>>>> preventing the GC from visiting your objects, than getting them
>>>>>> to do it at all. There are several places in a plan where you
>>>>>> need to add calls to your new space. For example, if you look at
>>>>>> marksweep as one of the simpler examples, just search the package
>>>>>> org.mmtk.plan.marksweep for the constant MS.MARK_SWEEP, and for
>>>>>> the field MS.msSpace. In particular, the method
>>>>>> MSTraceLocal.traceObject and its superclass instances are how the
>>>>>> GC visits heap objects. I assume you've done some of this,
>>>>>> because by default the traceObject method will crash the VM if it
>>>>>> tries to visit an object that hasn't got a special case.
>>>>>>
>>>>>> So if you can tell us which plan you're basing your collector on,
>>>>>> and how you address the above 2 problems, then maybe I can be of
>>>>>> more specific help.
>>>>>>
>>>>>> cheers,
>>>>>>
>>>>>> Robin
>>>>>>
>>>>>> On 10/1/20 9:00 pm, Johannes Sinsel wrote:
>>>>>>> Hey there,
>>>>>>> in a current project of mine, I'm trying to separate a few
>>>>>>> objects in the heap, so the garbage collection doesn't check
>>>>>>> them. This is done by using the implementation of a
>>>>>>> ImmortalSpace and give it a ring buffer fashioned memory
>>>>>>> management. In this way objects are not necessarily live (after
>>>>>>> a new ring buffer cycle has started).
>>>>>>>
>>>>>>> Unfortunately in the "full heap" collection phase, it seems to
>>>>>>> me, that the GC checks for static references independent of the
>>>>>>> used heap space. This causes the VM to crash, since objects can
>>>>>>> be overwritten and an address of a previous object can't be
>>>>>>> resolved anymore.
>>>>>>>
>>>>>>> Is there a way to separate the space more, so the GC is not led
>>>>>>> into it? I couldn't find a solution yet.
>>>>>>>
>>>>>>> Thanks and kind regards,
>>>>>>> Johannes
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Jikesrvm-researchers mailing list
>>>>>>> Jik...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-researchers
|
