[Geoserver-devel] Re: Pass through SQL and Nested XML source

[<ch...@op...>]

Ok, I've had a good review of the code, and am passing it onto the list
for others to check out.  Overall it looks like some great work,
definitely a nice solution to a very hard problem.  It makes use of
most all the right classes I believe, except for the fact that the work
is not in GeoTools, but in GeoServer.  But that is the recommended
strategy, as passing this stuff off to GeoTools will and should involve
some more minds looking it over.

It does imply some core changes in how a few things are done, and I
think this should be looked at in combination with the Expr work that
Jody and Andrea are undertaking.  I can get down and dirty with that
and look into how we should incorporate this.  In an ideal world we
make the DataStore access so good that a user would not have to
actually specify the bypassSQL at all, they would just provide a
mapping of the datastore values that they wanted, and some pre
processing would turn it into the appropriate sql.  But I'm not sure
that we have the resources to undertake that effort, as I think it will
involve coming up with a whole new xml syntax to specify such things
(though I may be wrong).  I think deegree has done some work in this
direction, specifying their table mappings in XML (though they just
limit it to single datastores, not cross datastores as we want to do). 
I asked them about it awhile ago, but never got a response.  If we
don't get there I think the bypass sql stuff is a very reasonable half
step to allow.

This does imply some changes to AttributeTypes and FeatureTypes, in
order to carry nested information.  Peter just did this in GeoServer
*TypeInfo classes, which is now becoming a popular way of working - I
remember Jody just stuffing all the extra stuff he wanted out of
AttributeTypes in there as well.  Peter's code provides a good jumping
off point, that we should take further to actually become a part of
GeoTools.

I think one area for improvement is the FeatureTransformer stuff.  The
approach taken is definitely the right one to get the work done without
messing with a bunch of geotools stuff, but I think both Jody and I
were hoping that it would be able to work more within the geotools
framework.  I definitely understand why you chose not to, since it is a
nasty beast, and it takes a good bit of work to get acquinted with how
it all works, let alone how to modify it.  Indeed I'm not yet sure that
it is possible to combine bypass sql with the geotools structure,
without coming up with the full syntax I mentioned of earlier.  I need
to dig into the code and think about it a lot more, but it is
definitely quite nice to have an example of how it needs to work.

Our thought was that a special ASFeatureTransformer and XMLElement
structure would not be needed - that that information could be
contained in the AttributeType and FeatureType, which would then be
appropriately transformed.  You could have nested AttributeTypes -
we've actually started work on this, but it's hidden away as static
classes in the DefaultAttributeType class.  For example there is a
DefaultAttributeType.Feature, that you can store a full nested feature
in.  I think perhaps it is time to move that hidden structure into its
own hiearachy, and to expand it to deal with the nested Attributes that
SCO wants to deal with.  Once the nested structure can be completely
represented in GeoTools then the FeatureTransformer should be able to
just take an Attribute and realize that it is a nested attribute and
pass each nested one to be expanded.  Hmmm....  Thinking on it though
this is definitely more work than I had imagined.  But I think it is
worthwhile to think through all this stuff long and hard and get it
into GeoTools for real.

So I'm going to spend the next couple of weeks doing that, and see what
I come up with.  As for strategies of what to do with this code, I
think the best thing would be to start a branch off of GeoServer 1.2. 
I would like to get it up and running on my machine as well, I think a
specialized cite/confBypass would be in order, perhaps even with a
database script to set things up, so that developers can all get a
similar set up.  Then we should attempt to refactor the work into
GeoTools, comparing against the working example on the branch.  I think
this will be a great way to keep us honest, to make sure the code comes
out the same.

But yeah, great job on getting this started, it's definitely a huge leap
forward in the right direction, on a problem we've been wanting to
solve for awhile.  I should be able to devote some decent resources to
it, and hopefully we can get GeoTools up to snuff.

best regards,

Chris



----- Forwarded message from Peter Barrs <pb...@so...>
-----
    Date: Fri, 17 Sep 2004 22:27:00 +1000
    From: Peter Barrs <pb...@so...>
Reply-To: pb...@so...
 Subject: Pass through SQL and Nested XML source
      To: Chris Holmes <ch...@op...>

Hi Chris,

Please finally find attached source tarballs of changed/new
geoserver/geootools source files

They are still quite rough - but there is enough working to get the
idea.

Also I've attached some brief outline notes to help.

If you want to have a chat about it that is fine - you'll just need to
email me in advance, I'm not using a permanently on connection for the
next few weeks, unless I go into town.

I know Jody would like this stuck on the list first up - but you might
want to look at it first and then decide. It hasn't been run through
jalopy or anuthing like that.

---

Some final notes: on nested XML element output


1) You must have a fid attribute for each recurrence group (see
schema.xml example)

2) You must sort the result set by these fids (the query in info.xml)

(There is a hook in the code for me to write some break checking code
that doesn't require fids, but I haven't done it yet and fids are the
most efficient way of doing break checking; the result set still has to
be sorted)

3)  To invoke the nested XML outputter, you have to set the outputFormat
attribute to GML2-AS in the WFS request:

<wfs:GetFeature service="WFS" version="1.0.0"
outputFormat="GML2-AS"
xmlns:wfs="http://www.opengis.net/wfs"
xmlns:ogc="http://www.opengis.net/ogc"
xmlns:gml="http://www.opengis.net/gml"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.opengis.net/wfs
                    http://schemas.opengis.net/wfs/1.0.0/WFS-basic.xsd">
<wfs:Query typeName="sco:geochem">
  <ogc:Filter>
       <ogc:PropertyIsEqualTo>
          <ogc:PropertyName>_arg_analyte</ogc:PropertyName>
          <ogc:Literal>Ag</ogc:Literal>
       </ogc:PropertyIsEqualTo>
   </ogc:Filter>      </wfs:Query>
</wfs:GetFeature>

(GML2-AS-ZIP should produce compressed output, although I have never
tested this)


Cheers
Peter Barrs


----- End forwarded message -----




----------------------------------------------------------
This mail sent through IMP: https://webmail.limegroup.com/