Re: [Xsltforms-support] [deegree-users] FW: deegree 3 postgis datastore config example
Brought to you by:
alain-couthures
Menu
▾
▴
Re: [Xsltforms-support] [deegree-users] FW: deegree 3 postgis datastore config example
|
From: Stephen C. <Ste...@ut...> - 2011-01-04 02:54:18
|
Hi Markus, Thanks for your detailed reply. A few quick thoughts: (1) Regarding Netkernel, I am very interested in the concepts behind it but have no first hand experience. At present I am coming to grips with the whole REST paradigm (read 'REST in Practice' (http://oreilly.com/catalog/9780596805838) over Christmas, which was very insightful). Perhaps a top level of XML caching (in CLOB) might be a useful optional feature to consider, falling through to lower levels based on specific request type/content. Netkernel might offer an easy way to try this out, but probably something for later. (2) Definitely ease of configuration is a major requirement to make deegree WFS software more widely appreciated, so simple configuration "by convention" at the simplest (beginner) level seems good. (3) "I think the way to go is a custom format for GML-version agnostic definition of feature types and relational mapping" -> Still not sure I agree about this, but I'll do some reading and make a separate list. (4) Perhaps auto-generation of a default WFS-T client web-interface based on server feature configuration (whatever it might be) would be a nice feature :) (5) I'll make what I have done with Xforms for WFS-T generation scripts available, no point in waiting. -----Original Message----- From: Markus Schneider [mailto:sch...@la...] Sent: Saturday, 1 January 2011 2:09 AM To: dee...@li... Subject: Re: [deegree-users] FW: deegree 3 postgis datastore config example Hi Stephen, I am sending this to the list, as it may be of interest to Just (and others) as well. Am 30.12.2010 10:57, schrieb Stephen Cameron: > I thought I'd put a few thoughts down in response to the previous comments that you sent to me (below). Understanding my thinking and what I have done to date might be helpful in regards to deegree 3 design (and I need to document my work anyway :) ). > > The first thing to say is that I think that the "hybrid" approach that you are taking makes a lot of sense. The caching of features in a XML format is highly desirable for performance reasons obviously [in regard to caching, if you haven't already take a look at the NetKernal product (http://www.1060research.com/netkernel/) that I came across recently, it has XML caching as a fundamental aspect of its design]. Exactly. One of the main benefits is reconstruction speed. If combined with a relational decomposition, the BLOB becomes a kind of cache. Thanks for the link. Do you think deegree 3 could still benefit from it? The actual serialization / deserialization is pretty much solved and we have to use our own feature model internally... > I guess the real issue is how this caching integrates with the filtered searching, in an XML database the raw XML is indexed so this is simple, but I imagine it is not so simple if you want to offer a hybrid XML& relational capability. [I don't see this subject as being in my field of expertise at all. but intuition makes me wonder at the possibility of using xlinks as a kind of reference within the server itself to integrate the two parts]. I considered using the XML datatypes (e.g. in PostGreSQL or Oracle). However, there are some downsides: - No evaluation of spatial predicates. - Evaluation of XPath-expressions that cross feature boundaries (from a feature to a referenced feature) is not possible. - Using a BLOB allows to use alternative encodings (e.g. Binary XML, GZipped XML, ...). PostGIS always uses CLOB for XML. - In some use-cases, relational decomposition is still what one wants. This is not handled by the XML datatype. All this is possible if one combines BLOB-mode with a traditional relational decomposition. The downside is that it's more complicated than just using the XML datatype and introduces redundancy. Note that relational mapping could also be partial, i.e. if you just require database filtering for specific properties of a feature type, it should be possible to just define tables/columns for these. > > To focus on my current interests, I see WFS-T from the perspective of a 'generic scientific data management tool' and want to leverage the deegree software infrastructure for this purpose. The primary considerations, in order of descending priority, in this respect are: > > 1. Server response is XML format, allowing complex data relationships to be modeled. > 2. Server response (XML) can be readily transformed for presentation purposes using XSLT. > 3. Create, Read, Update, Delete (CRUD) transaction capable with a well defined and understandable syntax. > 4. Filtering and searching capability, allowing you to deal effectively with large sets of data. > 5. Spatially aware as server supports GML and spatial operations. > > The first 4 capabilities above can be utilised in a browser based client, in particular via the use of a web standard designed for handling and manipulating XML, namely XForms. This is the task I set myself early this year, to demonstrate that WFS-T could be used with XForms in this way. [The 5th aspect is also handled by some web-apps (e.g. OpenLayers) but not in an nicely integrated way with 1-4 as yet]. > > It may be of interest to you to know that the genesis of this idea was my previous interest in XPath as a powerful and widely applicable declarative programming language, (which lead me to an interest in things like Apache Cocoon and JXPath) before starting my current job. > > In my current job I initially developed a web-client with a lot of custom Javascript code for doing filtered WFS-T GetFeature requests. After this was done I was asked to add the CRUD capability, on finding the XSLTForms project and with my prior XPath interests I decided to have a go at doing everything with XForms (which uses XPath) and hopefully to even replace my initial custom Javascript with an XForms based filtered search solution. > > The key advantage of XSLTForms is not so much that it can make use of XSLT support in the browser, but that it is a cross-browser, non-plugin solution. So, the user doesn't have to do anything special to make it work and you get a nice Ajax kind of web-app, due to the lack of page refreshes. However, XSLTForms is still very Javascript dependant, the XSLT transformation aspect in the name 'XSLTForms' is to translate the XForm XML files into Javascript and XHTML via an stylesheet instruction in the XForm XML. I have found this approach works fine in Firefox, Chrome and Safari. It is too slow in IE8, but it should be fine in IE9. > > Another motivating factor in this second version of my web-client was the idea that you can, using XML technologies, go a long way towards the grand goal of 'schema generated forms'. This is a simple concept that once you have designed a schema you can then use that as the model for generating data forms. I don't think there are too many web-application frameworks that don't allow you to do generate entity classes from a relational db schema, and web forms as well but usually very crude and no filtered search support. > > So to summarize my basic ideas and approach, (1) I wanted to make use of XForms for the client and to put any minimal 'business logic' required into the forms themselves, (2) Make use of the capabilities of the deegree WFS (items 1-5 above) on the server side and without any further custom server-side code. (3) Model the data in a schema document and generate the required XForms for CRUD, with filtered 'read' capability, from that schema. > > In regard to the last item, I have used a relational database schema (in XML format, for the sake of convenience more than anything else as my db design tool 'ERMapper' uses XML as its raw data file format) as the schema from which to generate the XForms and also to generate the deegree feature-configuration annotated XML Schema file. I had in mind from the start to swap this over to make use of the deegree feature-configuration file as the primary data model (XForms can use XML Schema files for data model validation by the way) but initially thought this was too complex to tackle, also I wanted to wait for deegree 3 release to see what this provided. > > The schema generated XForms aspect of this work has taken considerably more time than I anticipated but (thanks to some late hours and some understanding from my boss of the wider application of what I have been doing) it is now at a useful stage. I still have some work to do on the filtered search aspect but a working proof-of-concept is in place for that final part. All of this will be released under an GNU Library General Public License soon but my present thinking is that there is no value in doing this until WFS V2.0 support is available in deegree 3.0, with the view that this resolves a couple of issues that I have encountered in providing a complete set of CRUD functionality (mainly the code lists and update issues) > > So whilst in the short-term I am having to look to an alternative, more conventional, approach due to time constraints, I do think that what I have done to date with XForms and WFS-T has great potential. I'm not alone in this thinking, as its seems to me I am following a branch (or parallel) to the so-called XRX architecture. In my case XForms is common and REST and XQuery are replaced by WFS-T and OGC Filter Encoding Specification (it wouldn't surprise me to see REST and XQuery come into WFS spec though :) ). > > In conclusion then, my main interest is to see how the schema generated forms approach can work best with deegree 3. If the annotated schema is to be dropped in favour of FeatureStore configuration files, I wonder if this is going away from my philosophy of having a single point-of-truth schema (of one or other format) and generating (almost) everything else off that schema. Presently, I don't know how a FeatureStore configuration integrates with the DescribeFeatureType response, are these maintained independently? I think that what you write about annotated schemas being "bloated, complicated and offer many ways to express the same structure" is only true if you are trying to create them without the aid a schema editor. I feel that using a tool (built with XForms perhaps??) avoids these problems to a significant extent, after all XML was designed mainly as a machine-to-machine message format. In deegree 3, GML application schemas are still very important (this is the way the MemoryFeatureStore / PostGISFeatureStore BLOB mode) learns about the feature type hierarchy. And as GML application schemas are becoming more and more important, this representation will also be of primary interest in the future. However, for some use cases, I don't consider GML application schemas to be the best way to encode feature types. Problems: - A GML application schemas is always dependent on a GML-version. However, if one just wants to map a single PostGIS table, one usually doesn't care about that. It should be sufficient to specify the table to be able to query it via WFS in any GML version (2 / 3.0 / 3.1 / 3.2 / ...). This is already possible in deegree 3 and works well [1]. For DescribeFeatureType responses, the schema is generated on-the-fly (for the requested GML version). For GetFeature / GetGmlObject requests, the GML is exported for the requested version as well. - Writing a correct GML application schema is complex. I think it's much more approachable to think about feature types and properties instead of having to deal with GML core schemas, substitution groups, complex types, simple types and element declarations. One goal for the PostGISFeatureStore config is to make simple things as simple as possible, but support everything that's possible in GML schema. For schemas that use "custom property" declarations (which can virtually be anything that's possible in XML schema), the original schemas could be an option for augmenting the configuration (otherwise, one would end up redefining XML schema). - Of course it's not only about defining the feature types, but also about relational mapping. Besides being GML-version dependent and complicated, the annotation approach in deegree 2 has the additional downside that it cannot be validated. I think the way to go is a custom format for GML-version agnostic definition of feature types and relational mapping. If an application produces GML application schemas, there's always the possibility to derive the configuration automatically. > > If any aspect of what I have described above is of interest in relation to deegree 3 design or functionality I am keen to help further with this. In regard to XSLTForms in particular, if you have any specific interest in this, I know the developer Alain Couthures (in France) is very keen to find demo use-cases and collaborators. I also consider XSLTForms an interesting approach, as it has a very clean separation of client and server layers. However, at the moment, I don't have a use-case for it. If it pops up, I will let you guys know. > > As a final question, do you know of any other standard that provides the same kind of generic functionality as WFS? It seems to me that in providing a standard to deal with spatial data interoperability the OGC (and implementers such as yourselves) have addressed general problems and so produced software of much wider application. REST (as I'm currently reading) seems close but does not have the level of application to data as does WFS-T, any others? Hmm. Not really... Best regards and a happy new year, Markus [1] http://wiki.deegree.org/deegreeWiki/deegree3/WorkspaceConfiguration/FeatureStoreConfiguration#PostGISexamples > > > > > > > > > > -----Original Message----- > From: Markus Schneider [mailto:sch...@la...] > Sent: Thursday, 23 December 2010 8:27 PM > To: Stephen Cameron > Subject: Re: [deegree-users] FW: deegree 3 postgis datastore config example > > Hi Stephen, > > Am 23.12.2010 um 03:06 schrieb Stephen Cameron: > >> Hi Markus, >> >> On reflection I am now committed to a relational database solution, I need to finalize my project by the end of January, so very regretfully, in the short-term at least, I will have to look to replacing my WFS-T+XForms with another >> non-WFS based forms alternative. > > That's sad to hear. Staying with deegree 2 WFS is not an alternative for now? > >> >> This was the last thing I wish to do, as inevitably this means Java entity beans and an ORM mapping/persistence layer all of which is just not needed (IMHO) if you just want to move data between a database and XML (grrr!). >> >> After all the work I have put into this it's quite difficult to have to do switch course, but in the longer term perhaps for the best. I am very keen to contribute to the deegree 3.1 relational mapping work if I can, and getting rid of the current stress will be good. I'm sure my work with XForms will find a good use once WFS 2.0 is available. > > What is the special thing about WFS 2.0 that will change? I am currently not really aware of a change in the transaction behaviour between 1.1.0 and 2.0.0... > >> >> Just thinking the mapping last night I pictured a compilation process of an annotated schema (similar to compilation of JSPs) with some properties ('pseudo-XML-nodes') of the compiled product being (or wrapping) JDBC prepared statements. To make this work reliably I think there could be a validation method where the database schema is tested by the compiled XML schema in terms of column names, datatypes and other assumptions using the JDBC database metadata access. >> >> The need to generate an ORM mapping layer is only necessary for applying business logic, which for a pure data management system is not needed. In my view, thinking in the XML ways is much more productive, such as by attaching event listeners to parser events. >> >> No doubt you have some good ideas as to how you want to go forward with this as well. > > From my view (and for the use-cases we have in mind), the "hybrid" approach seems to be most desirable, i.e. combining BLOB mode with a (full or partial) object-relational decomposition. This would combine all benefits: > > - Fast reconstruction of features (from the BLOB), no need for subsequent SELECTs > - Evaluation of arbitrary filters on the database > - Possibility to access/alter the decomposed feature by third-party applications or plain SQL. If combined with triggers, an alteration of a property could also force a deletion of the BLOB, so the FeatureStore knows that it has to reconstruct the feature from the relational decomposition - afterwards it would recreate the BLOB. > > For the configuration of the relational mapping, I would rather see the use of the FeatureStore configuration files than annotated schemas, as schemas have two downsides: > > - They are bloated, complicated and offer many ways to express the same structure (e.g. choices vs. substitution groups) > - They are GML version dependent > > Have a good Christmas as well. > > Best regards, > Markus > >> >> Have a good Christmas. >> >> >> >> -----Original Message----- >> From: Markus Schneider [mailto:sch...@la...] >> Sent: Wednesday, 22 December 2010 3:39 AM >> To: dee...@li... >> Subject: Re: [deegree-users] FW: deegree 3 postgis datastore config example >> >> Hi Stephen, >> >> Am 21.12.2010 13:05, schrieb Stephen Cameron: >> >>> What is possible in BLOB mode as far as wfs:Update transactions go, can you update both simple and complex properties? As my web client is only interested in direct or XSLT transformed WFS GetFeature responses I might be able to go forward with that and maybe swap back to Relational later on. But it does seem to me like BLOB mode ( a.k.a. XML database :) ) is the future anyway. >> >> Actually, I am a bit unsure what works and what doesn't :-) We only tested with updates on primitive values / geometries so far. >> >> Would be very interesting to hear about your experiences so we can see what is still missing. >> >> A good starting point for setting up a complex application schema with deegree 3 would be deegree inspireNode [1]. It should be pretty easy to set up. The step-by-step guide (and a look at the configuration files) should be enough to give you an idea how it works. Afterwards, just adapt it to your schema and re-start with a clean PostGIS database. >> >> Best regards, >> Markus >> >> [1] http://wiki.deegree.org/deegreeWiki/InspireNode >> >>> >>> Thanks >>> >>> Steve Cameron >>> >>> -----Original Message----- >>> From: Markus Schneider [mailto:sch...@la...] >>> Sent: Tuesday, 21 December 2010 10:18 PM >>> To: dee...@li... >>> Subject: Re: [deegree-users] FW: deegree 3 postgis datastore config >>> example >>> >>> Hi Stephen, >>> >>> configuration examples for the PostGISFeatureStore can be found here [1]. >>> >>> >>> However, I am afraid that you have to hold your breath a little longer :-( In deegree 3.0, only BLOB mode and relational mapping for *simple feature types* is ready (GeometryProperty / SimpleProperty elements). >>> The config options you stumbled upon (CodeProperty, CustomMapping, ...) only work partially and were until now only used to test some ideas on complex relational mapping configuration. >>> >>> We hope to make progress on the complex relational mapping in January, so it should become available in 3.1 (February) [2]. >>> >>> If you're interested, I can sent the deegree 3 configuration for the Philosopher example to the list, once I find the time to start working on that again, so we can discuss some ideas. >>> >>> Best regards, >>> Markus >>> >>> [1] >>> http://wiki.deegree.org/deegreeWiki/deegree3/WorkspaceConfiguration/Fe >>> atureStoreConfiguration#PostGISfeaturestores >>> [2] http://wiki.deegree.org/deegreeWiki/deegree3/Roadmap >>> >>> Am 21.12.2010 11:59, schrieb Stephen Cameron: >>>> >>>> >>>> Hello, >>>> >>>> >>>> >>>> I just looked at the xml schema for the deegree 3 postgis datastore config. >>>> >>>> >>>> >>>> Is there possibly a simple 'demo' example of an actual config file >>>> (akin to the old philosophers example)? >>>> >>>> >>>> >>>> At first glance it looks very interesting, I am keen to try to use it! >>>> >>>> >>>> >>>> Particularly this bit looks like something I have been asking for J >>>> >>>> >>>> >>>> <!-- === Code property declaration / mapping === --> >>>> >>>> <element name="CodeProperty" >>>> substitutionGroup="postgisfs:AbstractProperty"> >>>> >>>> <complexType> >>>> >>>> <annotation> >>>> >>>> <documentation>Definition of a code-valued property of a >>>> feature type.</documentation> >>>> >>>> <appinfo> >>>> >>>> <jaxb:class name="CodePropertyDecl"/> >>>> >>>> </appinfo> >>>> >>>> </annotation> >>>> >>>> <complexContent> >>>> >>>> <extension base="postgisfs:AbstractPropertyType"> >>>> >>>> <attribute name="codeSpaceMapping" type="string" >>>> use="optional"> >>>> >>>> <annotation> >>>> >>>> <documentation>Mapping expression for the codeSpace >>>> attribute.</documentation> >>>> >>>> </annotation> >>>> >>>> </attribute> >>>> >>>> </extension> >>>> >>>> </complexContent> >>>> >>>> </complexType> >>>> >>>> </element> >>>> >>>> >>>> >>>> Also the mapping elements based on the following, how does this work?? >>>> >>>> >>>> >>>> <element name="AbstractCustomMapping" abstract="true" >>>> type="postgisfs:AbstractCustomMappingType"/> >>>> >>>> <complexType name="AbstractCustomMappingType"> >>>> >>>> <annotation> >>>> >>>> <documentation>Defines the mapping for (a part of) a custom >>>> property.</documentation> >>>> >>>> <appinfo> >>>> >>>> <jaxb:class name="CustomMapping"/> >>>> >>>> </appinfo> >>>> >>>> </annotation> >>>> >>>> <attribute name="path" type="string" use="required"> >>>> >>>> <annotation> >>>> >>>> <documentation>(Relative) XPath targeted by this >>>> mapping.</documentation> >>>> >>>> </annotation> >>>> >>>> </attribute> >>>> >>>> <attribute name="mapping" type="string"> >>>> >>>> <annotation> >>>> >>>> <documentation>Mapping expression.</documentation> >>>> >>>> </annotation> >>>> >>>> </attribute> >>>> >>>> </complexType> >>>> >>>> >>>> >>>> Regards >>>> >>>> >>>> >>>> Steve Cameron >>>> >>>> >>>> >>>> --------------------------------------------------------------------- >>>> - >>>> -------- >>>> Lotusphere 2011 >>>> Register now for Lotusphere 2011 and learn how to connect the dots, >>>> take your collaborative environment to the next level, and enter the >>>> era of Social Business. >>>> http://p.sf.net/sfu/lotusphere-d2d >>>> >>>> >>>> >>>> _______________________________________________ >>>> deegree-users mailing list >>>> dee...@li... >>>> https://lists.sourceforge.net/lists/listinfo/deegree-users >>> >>> >>> -- >>> l a t / l o n GmbH >>> Aennchenstrasse 19 53177 Bonn, Germany >>> phone ++49 +228 18496-0 fax ++49 +228 18496-29 >>> http://www.lat-lon.de http://www.deegree.org >>> Follow deegree on Twitter: http://twitter.com/deegree_org >>> >>> >>> ---------------------------------------------------------------------- >>> -------- >>> Lotusphere 2011 >>> Register now for Lotusphere 2011 and learn how to connect the dots, >>> take your collaborative environment to the next level, and enter the >>> era of Social Business. >>> http://p.sf.net/sfu/lotusphere-d2d >>> _______________________________________________ >>> deegree-users mailing list >>> dee...@li... >>> https://lists.sourceforge.net/lists/listinfo/deegree-users >> >> >> -- >> l a t / l o n GmbH >> Aennchenstrasse 19 53177 Bonn, Germany >> phone ++49 +228 18496-0 fax ++49 +228 18496-29 >> http://www.lat-lon.de http://www.deegree.org >> Follow deegree on Twitter: http://twitter.com/deegree_org >> > ------------------------------------------------------------------------------ Learn how Oracle Real Application Clusters (RAC) One Node allows customers to consolidate database storage, standardize their database environment, and, should the need arise, upgrade to a full multi-node Oracle RAC database without downtime or disruption http://p.sf.net/sfu/oracle-sfdevnl _______________________________________________ deegree-users mailing list dee...@li... https://lists.sourceforge.net/lists/listinfo/deegree-users |
