Re: [Kissme-general] Threads, I/O and Garbage Collection

[John L. <je...@pi...>]

On Thu, 2002-09-12 at 13:50, Stephen Crawley wrote:
> 
> > > That's the good news.  The bad news is that there are interactions
> > > between blocking I/O, Thread.interrupt() and Garbage Collection that
> > > are going to be hard to deal with.
> > > 
> > >   *  Calling Thread.interrupt() on a thread doing a blocking I/O
> > >      operation (e.g. read(), write(), socket accept()) is supposed
> > >      to unblock the I/O and (typically) cause the Thread to throw
> > >      InterruptedIOException.  Currently it doesn't.
> > 
> > Do we know if other VMs implement this properly?
> 
> I believe that Sun's recent JVMs for Solaris handle this correctly,
> but their JVMs for Linux and Windows don't.  There is an open bug
> in the Sun bug database related to this, and I've been running into
> problems in this area in my "day job".

I suspected it wasn't well implemented (although have never tried it
myself).

> > > The problem is that this behaviour all needs to be implemented
> > > in the JNI native code that does I/O.   That's OK, except that
> > > we currently use bog-standard GNU Classpath implementations of
> > > these methods; e.g. from libjavaio.so.  This seems to suggest
> > > that we need to go back to having our own implementations of
> > > at least some of these native methods.
> > > 
> > > Can anyone think of a better way around this?
> > 
> > I don't see a way around this. It might be possible to have two versions
> > in classpath, (two behaviours for the native code, and two behaviours
> > for the .java io code), although I'm not sure how feasible/manageable
> > that is.
> > 
> > Do the changes only affect the native code?
> 
> I believe that the changes can be kept to the native code.  I don't
> think it is solve the problem in pure Java.  Interactions with the
> GC inherently involve native stuff.
> 
> > I was thinking perhaps the calls could be asynchronous and the java
> > wrapper code might be able to handle early returns and partial reads etc
> 
> Making the calls asynchronous (non-blocking) would involve using I/O
> polling.  IMO, this approach is likely to be expensive and to give 
> non-responsive I/O.  The alternative of using select(2) would be complex.
> 
> My idea for Thread.interrupt() interactions with I/O is as follows:
> 
>   *  Before doing the I/O syscall, a thread sets its threadNode->status
>      to THREAD_STATE_IO.
> 
>   *  When another thread calling interrupt() and sees that the target
>      thread's status is THREAD_STATE_IO, it uses pthread_kill to 
>      send a UNIX signal to unblock the IO syscall.  This may cause
>      the syscall to EINTR or return with a short read.

Can this signal be sent reliably on all linuxes? Is it portable to
Solaris, Hurd?
 
>   *  After the syscall, the thread checks the return code for EINTR, 
>      and throws InterruptedIOException if this occurs.  A short
>      read just returns the number of bytes read.

Are there any race conditions where we might end up not signalling a
thread (and thus interrupting the syscall)?

> Garbage collection interaction with I/O can be handled similarly.
> 
>   *  The GC sends a UNIX signal to all threads blocked in I/O 
>      syscalls; i.e. those whose status is THREAD_STATE_IO
> 
>   *  The unblocked threads detect that they've been interrupted by 
>      the GC and enter a GC point.  
> 
>   *  The GC runs.
> 
>   *  When the GC is done, the threads exit the GC point and either 
>      restart the syscall or return a short read.  Before restarting 
>      their IO syscall, they would recalculate any buffer addresses 
>      in case the GC has moved the buffer object.
> 
> Obviously, the two bits of logic need to be integrated with each other,
> and with the existing GC and interrupt handling code. This approach
> avoids polling the read/write/etc call, or polling select.  So we don't
> get CPU burning or unresponsive I/O ... or lots of complexity to avoid
> these problems.

Agreed.

> Another alternative to GC I/O interactions would be to do I/O using 
> buffers reserved using sys_malloc.  However this entails double-copying
> data, and hence is inefficient.

Yes, your approach makes a lot of sense.

John