From: Andrea A. <and...@ge...> - 2011-11-27 10:28:29
|
Hi, the shapefile-ng work is nearing completion. I have all the tests passing now (at least, once the transaction stuff for content data store gets in) and the code seems to work, in test harness, for both indexed and non indexed shapefiles. In particular thanks to content data store this one should support sorting for good. General highlights: - the basic underpinnings (low level readers and writers) are the same as shapefile data store (did not get around to try a cleanup there, and bug reports are piling up, have stop doing R&D and fix at least some of them). - the top level is rewritten to use content data store - there is a single configurable store instead of two (indexed or not) - the non indexed version should be somewhat faster Issues I see in a migration: - not nearly as real world tested as the old code - the old code has a public interface large from here to the moon, with a ton of methods that should have never been made public (lots of methods that were not in the DataStore interface to grab readers and writers for example) - the old code had a billion constructors with all the possible options, the new one has _one_ constructor taking a URL, and then has getters/setters for all the various options (turn off indexing, timezone, charset) - the code is sitting, at least for the moment, in a org.geootools.data.shapefile.ng package Of course migration won't be instant for all that code that used directly the extended api the old store was providing. In any case, I was wondering how to handle the upgrade. Moving the classes in the same package might help the migration, but at the very least I would like to get rid of the "ng" since that's going to become the default thing (if we do another rewrite, are we going to mark it as "aab"? (above and beyond?)). Maybe be consistent with jdbc and move it to org.geotools.shapefile And then what... have the new code pick up the old params in case it's the only one in the classpath, otherwise redirect to the old store? We also need to test it out quite a bit in all the GeoTools code that depends on it (test, image mosaic), in GeoServer and uDig. Still quite a bit of work ahead. Cheers Andrea -- ------------------------------------------------------- Ing. Andrea Aime GeoSolutions S.A.S. Tech lead Via Poggio alle Viti 1187 55054 Massarosa (LU) Italy phone: +39 0584 962313 fax: +39 0584 962313 http://www.geo-solutions.it http://geo-solutions.blogspot.com/ http://www.youtube.com/user/GeoSolutionsIT http://www.linkedin.com/in/andreaaime http://twitter.com/geowolf ------------------------------------------------------- |
From: Jody G. <jod...@gm...> - 2011-11-27 12:35:10
|
I will update the ContentDataStore patch; it looks great and we have both had a kick at it. If I can ask you to review my update and then apply we can get this show on the road. > In any case, I was wondering how to handle the upgrade. > Moving the classes in the same package might help the migration, but at > the very least I would like to get rid of the "ng" since that's going to become > the default thing (if we do another rewrite, are we going to mark it as > "aab"? (above and beyond?)). > Maybe be consistent with jdbc and move it to org.geotools.shapefile > > It would be good to handle this as with the last "index" experiment - allow the factory to make the decision. Add it to the existing gt-shape module; use a connection parameter to allow people to try it out for a while. After it has seen a couple releases cut over to it as the usual implementation the factory produces; deprecate the AbstractDataStore implementation(s) etc.. Perhaps the connection parameter can be "sorting=true" with a default value of false. > And then what... have the new code pick up the old params in case it's the > only one in the classpath, otherwise redirect to the old store? > > it (test, image mosaic), in GeoServer and uDig. Still quite a bit of work ahead. > We also need to test it out quite a bit in all the GeoTools code that depends on > > Agreed. Cheers, Jody |
From: Justin D. <jde...@op...> - 2011-11-28 17:15:10
|
In terms of migration I think it would be ok to drop the data component of the package... however as you say what happens next time we want to rewrite... go back to data? What about keeping the same package and just telling users to try out the new one you can't have the old jar around? Too harsh? On Sun, Nov 27, 2011 at 12:34 PM, Jody Garnett <jod...@gm...>wrote: > I will update the ContentDataStore patch; it looks great and we have both > had a kick at it. If I can ask you to review my update and then apply we > can get this show on the road. > > In any case, I was wondering how to handle the upgrade. > Moving the classes in the same package might help the migration, but at > the very least I would like to get rid of the "ng" since that's going to > become > the default thing (if we do another rewrite, are we going to mark it as > "aab"? (above and beyond?)). > Maybe be consistent with jdbc and move it to org.geotools.shapefile > > It would be good to handle this as with the last "index" experiment - > allow the factory to make the decision. > > Add it to the existing gt-shape module; use a connection parameter to > allow people to try it out for a while. After it has seen a couple releases > cut over to it as the usual implementation the factory produces; deprecate > the AbstractDataStore implementation(s) etc.. > > Perhaps the connection parameter can be "sorting=true" with a default > value of false. > > And then what... have the new code pick up the old params in case it's the > only one in the classpath, otherwise redirect to the old store? > > it (test, image mosaic), in GeoServer and uDig. Still quite a bit of work > ahead. > We also need to test it out quite a bit in all the GeoTools code that > depends on > > Agreed. > > Cheers, > Jody > > > ------------------------------------------------------------------------------ > All the data continuously generated in your IT infrastructure > contains a definitive record of customers, application performance, > security threats, fraudulent activity, and more. Splunk takes this > data and makes sense of it. IT sense. And common sense. > http://p.sf.net/sfu/splunk-novd2d > _______________________________________________ > Geotools-devel mailing list > Geo...@li... > https://lists.sourceforge.net/lists/listinfo/geotools-devel > > -- Justin Deoliveira OpenGeo - http://opengeo.org Enterprise support for open source geospatial. |
From: Andrea A. <and...@ge...> - 2011-11-29 15:19:00
|
On Mon, Nov 28, 2011 at 6:14 PM, Justin Deoliveira <jde...@op...>wrote: > In terms of migration I think it would be ok to drop the data component of > the package... however as you say what happens next time we want to > rewrite... go back to data? What about keeping the same package and just > telling users to try out the new one you can't have the old jar around? Too > harsh? It would be a bit unprecedented and a bit annoying to handle. So far these upgrades have been handled in such a way that you can opt out by setting a switch (think jdbc upgrade, shapefile renderer phasing out) and I'd like to keep it that way, so that people are somewhat eager to try because the process is simple and safe. Jody's suggestion seems to be a good one, we could indeed have a flag in the factory that enables/disables the new store. This would allow us to test the thing a bit before the 8.0 release and if it works fine for most people we can just move the old code in unsupported for those that have troubles migrating to the new one Cheers Andrea -- ------------------------------------------------------- Ing. Andrea Aime GeoSolutions S.A.S. Tech lead Via Poggio alle Viti 1187 55054 Massarosa (LU) Italy phone: +39 0584 962313 fax: +39 0584 962313 http://www.geo-solutions.it http://geo-solutions.blogspot.com/ http://www.youtube.com/user/GeoSolutionsIT http://www.linkedin.com/in/andreaaime http://twitter.com/geowolf ------------------------------------------------------- |