At 12:43 PM 8/1/2001 -0400, Clark C . Evans wrote:
Actually, your post here brings up the compromise that
I currently have in the API.  [...]

Your description reminds me of a MIME stream reader I designed.  Each MIME part could have its own virtual reader.  Different readers could do different things with the data.  One could deliver data only incrementally, one could persist the data in memory, one could persist it on disk, one could throw the data away, etc.  The goal was to support some of the crazy things people are doing with MIME for B2B.

When I said that the low-level API would be random-access, I meant that the API would provide methods for random access, not that it would always allow random access.  An application walking the data in increment-only fashion would use forward-traversing the random access method.  I just meant that the navigation methods would be the same.

The methods specific to incremental access would manage the width of the sliding window, deliver portions of the input stream as substreams instead of as objects, truncate or advance-load the stream, and manage streaming-related errors.  As you suggest, you can go even further if you want to control whether, how, and where the underlying data is persisted.

A YAML document does have a root-most node.  That node is the list of topmost maps.  I think the sliding window approach would work nicely.