From: Jody G. <jga...@re...> - 2004-02-06 21:23:30
|
Andrea Aime wrote: >we have to cache at least attributes in parse form, but without creating >the features, in a way that allows to throw away the attributes that are not >used... I visualize it as a matrix or dbms table, where the rows are features, and >columns are attribute types... the cache decides based on the last data usage >and the specific datastore capabilities wheter to purge whole rows (unused features) >or whole columns (unused attributes at the moment). > I had the same kind of visualization in mind, I was leaning toward the "row" based purge. But I can see now that column based load/purge may be more advantageous. > But I'm not really >sure that this would work, it just seems to represent well the idea of >clients asking for a subset of features (based on location) and subsets of attributes >(based on style needs, for example), and then asking for more attributes (maybe >attribute table visualization) and the datastore would only have to integrate the >current cache. In case of remote data sources (database on a different machine, >a WFS) the attributes that are to be removed from the in-memory cache could >just go to the disk (which makes me think to something like pickle). > > It may be nice walk the middle ground, store cache information based on featureId, and perform a lazy parse of attributes. >>There is another scary alternative - both the Databases and Shapefile >>can support random access. And can pull the kind of suggestions made >>above off - without breaking the existing API. >> >> >How do you support random access with a database with reasonable >performance? > > Keep the ResultSet abstraction and access it in a Random fashion. The current JDBC DataStore makes use of this (and has a complex QueryData object in order to keep several AttributeReaders all tracking the same ResultsSet). >>That is: >>- allow DataStores to use their own random support at the >>AttributeReader level >>- leave the higher level FeatureReader and FeatureSource APIs as is >> >>I will point out that in the PostgisDataStore case the Postgres JDBC >>Driver seems to just keep the ResultSet in memory, so we only would save >>on parsing, not memory. >> >> >What? Client side cursors? Oh my... in fact, I'd expected it to load features >on demand for forward only result sets... > > That is what I expected as well, it is not what QueryData was doing last time I checked. Jody |