[Chromium-dev] Client-side display list management design

[Robert E. <pa...@tu...>]

Hello.  This is my first posting to this list, reflecting my intentions 
to implement my first contributions to this project.

In my former life, I was a display list expert at a major computing 
corporation; now, through the graces of my new employer (Tungsten 
Graphics) I am eagerly preparing to devote some of my efforts to 
bringing new functionality to Chromium.

This has been preliminarily reviewed by my co-workers; I would like to 
now offer it to a more global audience, for inspection and comments.

Thanks for your time and attention...

Bob Ellison
Tungsten Graphics

---------------------------------------------------------------

Problem Statement:

While using the "tilesort" SPU, all display lists are sent to all 
servers as they are created; in that way, they are treated like state, 
while display list references (i.e. glCallList & glCallLists) are 
treated like primitives.

Unfortunately, this distribution of display lists can saturate the 
network quickly; and in the general case, a particular display list may 
not be needed by a particular server (i.e. if the bounds of the rendered 
display list all fit in one tile being rendered by a single node, it is 
inefficient to send that display list to all nodes).

Proposed Solution:

In the client library (in a new SPU based on "tilesort"), manage display 
lists and content as they are created.  At reference time, determine 
which nodes need to know about the display list (by bounding box 
calculations), and, if that node hasn't seen the display list before, 
send the display list (and all sub-display lists, i.e. display lists 
referenced within the top-level display list itself).

Design: Client-managed Display Lists

Based on: Tile Sort SPU

Configuration:
     - CR_SHARED_DISPLAY_LISTS should be *off* for "normal" uses, or we
       run the risk of breaking a distributed application that relies on
       one node to create display lists and another node to use them.
       A warning will be issued at MakeCurrent if this is discovered.

New Configuration Variables:
     - none

New State, per client:
     - managedDisplayLists: a set (default empty) of struct {
         displayListIdentifier: the name of the associated DL
         acquaintedServers: a set of Chromium servers that know
             about this DL (i.e. the display list has already
             been created on each server in this list)
         retainedContent: a set of content buffers that
             can be sent to a server to create the display
             list there
       }


Override these functions:
     glNewList:
         should invoke a packer that saves all the DL content buffers
         without sending any to the Chromium server.
     glEndList:
         will add a new entry to the managedDisplayLists set, with:
             displayListIdentifier = identifier provided in glNewList
             acquaintedServers = empty set
             retainedContent = the list of content buffers

     /* helper function for glCallList; returns true if the
      * managedDisplayList record can be deleted, because all
      * servers have a copy
      */
     SendToServer(identifier, server) {
         get the managedDisplayLists record for the display list
             identified with "identifier"
         if the given server is not in the acquaintedServers list {
             send the retained content to the server, to create the
                 display list on that server
             add the server to the acquaintedServers list
             /* make sure all sub-display lists are also sent.  A
                mechanism to avoid an infinite loop if a recursive
                network is found will be implemented, but is not
                included in this design
              */
             for each display list referenced by this display list {
                 SendToServer(associated identifier, server)
             }
             if (all known servers now have a copy of this DL) {
                 return true
             }
          }
          return false
     }

     glCallList(identifier) {
         serverList = the list of servers to which this glCallList
           should be sent, based on bounding boxes and extents
         if (identifier is in the managedDisplayLists set {
             for each server in serverList {
                 knownEverywhere = SendToServer(identifier, server)
                 /* If the DL is known everywhere, the client no longer
                  * needs to manage it
                  */
                 if (knownEverywhere) {
                     delete the managedDisplayLists record
                     break
                 }
             }
          }
          for each server in serverList {
              send a glCallList reference to that server
          }
     }

     glCallLists: similar to above

     glDeleteLists:
         delete associated managedDisplayList records
         pass through the glDeleteLists call

                     /* Future: allow the use of the callback to
                      * define the DL, instead of relying on retained
                      * content
                      */

Possible future work:
     - Client-side management of display list proxies in addition to
       full display lists.  (To "proxy" a display list, the application
       identifies a callback function that will be used to create display
       lists on demand.  The application may then use glCallList et al.
       to invoke display lists without explicitly defining them first;
       the client-side display list manager will invoke the callback when
       needed.)

       The advantage to the application is that the client-side DL
       manager need not retain potentially large pools of DL content,
       which typically duplicates content storage in the application
       itself.  This could be useful to reduce resource contention at
       application nodes.

     - retention aging.  Every time a managed display list is referenced
       without having to recreate the DL on some server, its "age"
       increases.  (Every time it is sent to a new server, its age is
       reset to 0.)

       When it reaches a threshold, the client library can delete
       its retained content, in the assumption that the display list
       has been sent to every server that it will ever be sent to.
       (This should not be allowed unless a definition callback
       is defined, just in case the client library is wrong in its
       assumption.)

       This again can reduce memory usage and resource contention at
       the client nodes, by allowing the retained display list content
       to be freed in some cases that would otherwise have forced it to
       be retained indefinitely.

     - management of display lists created with glXUseXFont.

     - networked display list management; an application node may
       tell a server that it knows how to create a display list, without
       explicitly sending the content quite yet.  Later, if another node
       sends a reference to such a display list, the server can send a
       query back to the original node asking it to create the display
       list.

       This allows client-side display list management to be used even in
       an environment where one networked node creates shared display
       lists to be used by other nodes, but at the cost of requiring
       a server-client round-trip.