Re: [cothreads-discuss] raytracer crash on OS-X and mmap'ed files question

[<mik...@gm...>]

Thanks for answering

On 10/10/2007, zhe...@us...
<zhe...@us...> wrote:

> When I searched "Illegal instruction" through the OCaml mailing list achieve, I
> found that most of this kind of problems are related to OCaml runtime rather
> than user space libries. Especially, I found the bug report [1], which seems to
> be exactly related:
>
> [1] http://caml.inria.fr/mantis/view.php?id=4348&nbn=12

Yes, that could well be it. It was closed yesterday...

> Knowing quite little about the implementation of your application, I would
> predict two problems here:
>
>  - coThreads STM involves taking snapshot of transactional variables (OCaml
>    values). In your case, it seems that the real storage is in C side and the
>    OCaml-side values are just descriptor (possibly unchanged?). So I fail to
>    see what kind of benefit you can get here.

No as I said later on - I don't see I direct match. But for real world
usage STM will in general need to consider "out-of-band" storage -
providing locks is one way to do this, but that does not bring STM
benefits to the table.

>  - I have some experience with Bigarray which is based on mmap. It's unlikely,
...
>    update them independently, the efficiency is dramatically improved. The
>    problem is that, a Bigarray can hardly considered as a set of independently
>    array elements (On the other hand, you can do that, but the most functions
>    are no longer directly applied to Bigarray itself.

Yes - that is not exactly what I'm doing, but the concept is not that
different. You can define a set of atomic low-level operations on the
storage and coordinate these.

I guess coThreads actually could use a "rollback" trigger operation so
it doesn't just roll back a TVar but triggers external rollback in
that case. For example, you could register "appended x bytes to pos y"
in a TVar taking up limited of space and undo the operation without
storing a full log, or you could link it to an SQL server transaction.

>    You can actually read
>    Bigarray values into OCaml values and operate on OCaml values then write the
>    result back as Bigarray values, however I'm not sure if this is what you want)

Unless only small amounts are modified, that would load the ocaml heap
too much, but in some cases it might make sense - such as making
invariants over control elements guarding access to the rest of the
data. This is a kind of name based locking where name lookup is needed
so other threads can find a loaded copy. An STM hash table would be
one way to do this - might be convenient across processes compared to
alternatives like a hash table with a pool of recycled mutexes.

> > I'm fairly sure STM can't be used directly for my storage although
> > some principles are related, but it might solve my need for higher
> > level concurrency control including process coordination and provide
> > atomic operations on lower level structures.
> Considering your application, there are also traditional mutex and condition
> modules in the process engine, compatible with the standard ones for
> threads/vmthreads. On the other hand, you shouldn't have much difficult to
> implement your own control structure in term of STM.

Right - it's mostly about having a convenient abstraction to work with.

> I'm currently working on the following extensions for coThreads:
>
>   - special thread identifier: Cothread.anyone and Cothread.everyone
>   - raise exception in certain thread(s): raise_in
>   - Timer module: set timer, schedule and timeout
>   - Lock module: fine-grained reader/writer locks

Keep up the good work.

> > btw.: the web menu is broken on this page:
> > http://cothreads.sourceforge.net/example/index
> It's a problem of the sourceforge web hosting. Usually it works again if you
> reload the page.

Working again.


> I also CC the reply to coThreads mailing list, hope you don't mind.

OK

Mikkel