[Dri-devel] Re: PCI Bus and Mach64's DMA ring

[F. <j_r...@ya...>]

On 2002.06.14 18:20 Linus Torvalds wrote:
> 
> 
> On Fri, 14 Jun 2002, José Fonseca wrote:
> >
> > So to avoid being constantly checking for conclusion before asking to
> > process new entries we devised a different scheme:
> >
> >   - after adding new entries to the ring
> >
> >   - toggle the end flag of the previous last entry, so that the engine
> > will also process our just commited buffers
> 
> If this is in non-coherent memory (AGP), I hope you do an "sfence" in
> between those two stages?  You also need to make sure that the compiler
> hasn't re-ordered them (ie a compiler barrier() in between), regardless
> of memory ordering.

The actual code snippet is simply:

   /* append to ring here */
   ...
   mach64_flush_write_combine();
   ring[(tail - 2) & mask] &= cpu_to_le32( ~DMA_EOL );
   mach64_flush_write_combine();

where mach64_flush_write_combine() is defined as mb(), as all other 
drivers. So from what I've seen from snooping around in 
include/asm-i386/*.h this is a superset of barrier() and sfence (which 
AFAIK is an assembler instruction not found in all processors). I used the 
second barrier to have the write made ASAP.

> I also hope you do the toggle with a locked cycle so that you don't lose
> any information..

I'm not doing an atomic operation. Note that there isn't a two 
simultaneous writes, but one write and one read simultaneously at most. 
The DRM already has a locking mechanism which only allows one client.

I'm also doing the tests on a single AMD K6 processor machine with the RHL 
7.3 stock 2.4.18 linux kernel, but with "low latency scheduling" enabled. 
I'm not sure if problems should occurr because of this but I'll see if 
anything changes by using set_bit().

> 
> ...
> 
> >   - Has this problem ever appeared on the Linux kernel before, and how
> was
> > it solved?
> 
> There have been some similar issues on USB, although they weren't exactly
> the same.
> 
> On USB, as on the mach64, the controller DMA's the commands from memory,
> and you don't want to shut down the controller just to add or remove a
> command. So the USB drivers have some of the same issues.
> 
> However, the USB setup is different enough that they don't see your exact
> problem. The controller will walk a circular buffer of linked lists
> forever, so if you race with command insert, it will be seen the next
> time
> around the list instead. The remove case is the much more interesting one
> on USB (the USB controller might just be executing the thing that you're
> trying to remove), but you don't have that problem.

mmm.. perhaps their code to update the DMA'd memory can give some ideas..

> 
> The USB lists are also all in regular RAM (no AGP stuff that might turn
> off the normal cache protocol), so the regular wmb() macro works fine on
> it (although some architectures really want a separate "write barrier for
> DMA" and consider barriers for CPU vs IO to be very different, oh well).

Here you seem to imply that wmb() isn't enough for AGP... what should I 
use then?

> 
> You might ask the USB people if they have any ideas.
> 

Ok. I'll post this to linux-kernel once I've study it a little more.

> ..
> 				Linus

Thanks,

José Fonseca