[Dri-devel] RFC Shared AGP Memory

[Sottek, M. J <mat...@in...>]

I posted a RFC about a new type of "Shared" agp memory a while back
but didn't get any input. I thought I would try again since there has
been better communication as of late, and the idea has progressed
somewhat.

The problem:
The agpgart usage model is not well suited for UMA architectures because
each gart user is expected to allocate memory and only bind it into
the gart while it is active. Therefore on systems where all graphics
memory is obtained from the gart a huge amount of system memory is
wasted (16 Megs is probably a common amount of wasted memory on a
system running 2 X servers). To further the problem this memory is
non-swappable.
The issue has become exacerbated on embedded devices that must have
a framebuffer console for smooth graphics transitions during boot.
The framebuffer console must share a framebuffer among the virtual
terminals and eventually share resources with X.

Solution:
I have created a new type of agp memory in use in the i810's agpgart.
This type is "Shared" memory and works quite differently from regular
agpgart memory.
First, when memory of type shared is allocated the allocation will
basically always succeed, but does not allocate any pages. It only
allocates the agp memory structure. The pages are only allocated when
bound. There is a shared reference count table (8bit * Number of pages)
that allows the agpgart driver to determine if the pages actually need
to be allocated at bind time. If a page is being bound where a
shared page already exists the reference count is increased and the
page is reused. Pages are freed when the last reference is removed
(during unbind)

Second, in order to be compatible with existing memory types the
following rules apply:
 * Binding regular memory on top of shared memory does NOT result in
   an -EBUSY. The existing pages are left bound and the reference count
   is increased. The regular memory pages are wasted, and therefore
   should be avoided in the future.
 * Binding shared memory on top of regular memory is not allowed. This
   is primarily the case because allocate/free of regular memory is
   handled at the device independent layer. This also gives the first
   client the guarantee that the memory behavior will be as historically
   expected.
 * Binding of Local memory for i810 is carried out as normal. If local
   memory is being bound on top of shared memory the shared pages are
   kept instead. (I expect that local memory will be unbound when a
   client leaves since there is no duplication of memory)
 * Physical memory is currently still a problem. Since the client is
   given the physical address of the page at allocation time it can
   not be ignored when bound into a shared area. Currently I swap the
   physical page with the one in the shared area. This works fine but
   would fail if two clients bound a physical page at the same location.
   The physical type of memory has no actual need to be bound into the
   gart. It just makes it easy for XFree. So designs that are sharing
   memory with the frambuffer should just mmap the physical page via
   the framebuffer and not map it into the gart.


Using this scenario I have created a framebuffer that uses a different
Shared memory allocation for each virtual terminal. Each virtual
terminal allocates and binds the memory at startup and the agpgart
internally keep track of the reference counts. When a mode change
occurs, the virtual terminal allocates and binds a second memory
region (If the size changed), then unbinds and frees the first. Since
the memory for each vt in independently reference counted this system
guarantees that the minimum amount of memory needed to satisfy any
running virtual terminal is allocated into the gart.

This plan can be carried forward to accommodate all resource types.
DMA buffers, textures, video surfaces, back/depth buffers would
all be allocated dynamically only when needed. Thereby saving large
amount of memory during normal operation.

Comments welcome,
 -Matt