[Geotools-devel] IRC Logs for Monday 17th, 2006

[Jody G. <jga...@re...>]

1: 2.2 Status
2: GridCoverage Branch Status
3: Build system improvements
4: WFS New Feature FID Generation
5: Index shapefile GPL clarification

rgould: Ok, first item: 2.2 status
rgould: I have completed all of the remaining deprecation items that 
Jody listed in his email last week.
simboss_: hi all
rgould: The mentioned email is here: 
http://www.mail-archive.com/geo...@li.../msg03374.html^ 

rgould: The only problems I encountered were that several sections 
(*.gml) were un-deprecated
rgould: and that removing some deprecated code from LiteRenderer breaks 
the build
rgould: so I left them
aaime: Richard, how did you distinguish stuff that has been deprecated 
long time
aaime: and the one that has been deprecated in 2.2?
rgould: Jody built the 2.0.x javadocs
rgould: and it listed all of the sections that were deprecated (see the 
attachment to his email)
aaime: Ah, I see. Good
acuster: the online docs have been tagged as "out of date" so 2.2 should 
not block on the docs
rgould: Then all of the tasks are done.
rgould: http://docs.codehaus.org/display/GEOTOOLS/Tasks+for+2.2.0+release^
aaime: Ready to release then?
acuster: where will the 2.2.0 javadocs live?
rgould: Looks like it. I will send out an email before I do so, to make 
sure there isn't anything missing.
acuster: on .fr?
rgould: I'm not sure. There should be a zip file distributed with the 
release that contains the javadocs
acuster: I need a stable link for the website
rgould: but they should be online somewhere
acuster: isn't that part of the release process?
rgould: I don't think there is anything on the release page about 
posting the javadocs online.
rgould: There needs to be though.
rgould: I'm looking at the navigation bar on codehaus, and under 
"Develop" there is a link to javadocs on udig.refractions.net
acuster: yeah, that's trunk
rgould: to the 2.3.x javadocs specifically
acuster: that's why we need one for 2.2 for the users
acuster: .fr gives us 2.2, 2.3 and something else
aaime: cd ..
acuster: http://javadoc.geotools.fr/^
aaime: (sorry, wrong window)
acuster: oh, and trunk (which is 2.3 for now)
rgould: I would be for putting the javadocs up on javadoc.geotools.fr
acuster: no.
acuster: they *are* up on javadoc.geotools.fr
aaime: That reminds me of a continuous build requirement: we would need 
a continous build for selected branches, and publishing the javadocs 
would be one of its duties
acuster: the question is if we rely on that off-site server
acuster: right now Martin has to upload new versions of the javadocs
acuster: if we don't have room on the confluence sites, then we should 
use .fr for now.
rgould: Hmm I don't know if we can easily put them up on codehaus
rgould: i'm not sure if they grant us a section to dump files or not
rgould: Need more information from Martin
acuster: okay, we'll use .fr until further notice. /me adds the link
rgould: I think it would be best to have them on codehaus
rgould: But if we have to force it through Confluence, that would be a 
real pain.
acuster: sorry, not confluence
acuster confuses codehaus and confluence
aaime: It would be nicer if the maven built site could be published 
(provided that it works...)
rgould: agreed
acuster: so when can 2.2.0 go out?
aaime: I'd say to generate aggregated javadocs and call it done
rgould: well, as long as there are no objections, as soon as I can make 
the release
rgould: yup
Jesse_Eichar77: objections
rgould: generating javadocs is part of the release process
Jesse_Eichar77:
aaime: Building the maven site and publishing is something we should aim 
at for the next release
rgould: What's up Jesse_Eichar77?
aaime: Otherwise we'll release 2.2 in September
rgould: Yeah, I won't worry about it
Jesse_Eichar77: Well we can release 2.2 but WFS has quite a few bugs 
that I'd like to fix before releasing.
Jesse_Eichar77: a final release at any rate.
Jesse_Eichar77: that's all.
acuster: how long will it take
acuster: ?
rgould: Do we want to give a cut-off time? End of the week?
acuster: do you know what the bugs are?
Jesse_Eichar77: Yes I do.
acuster: 2.2 is *way* overdue
Jesse_Eichar77: Well We can release and do a 2.2.1 very soon.
Jesse_Eichar77: its just that the bugs are pretty severe. At least with 
regard to editing.
acuster: how long will they take to fix?
acuster: does anyone have any time?
Jesse_Eichar77: I'm working on it full time right now.
Jesse_Eichar77: I'd say we can do the release friday and what ever bugs 
I still have outstanding can be done on 2.2.1
acuster: rgould, you're going to cut the release right?
rgould: correct
rgould: unless someone else wants to
rgould: but I am the most familiar with the process
acuster: so Jesse_Eichar77 needs to contact you when it's "good enough"
rgould: I'll probably do the release on Friday anyway
rgould: Ok. Let's move on
rgould: 2) Grid Coverage branch status
rgould: aaime, simboss_ ?
simboss_: I am here guys
simboss_: situation is as follows:
simboss_: Martin started with merging referencing
simboss_: and coverage
simboss_: he has done with referencing
simboss_: but he has given up pon coverage
simboss_: due to lack of time
simboss_: here is the back up plan
simboss_: I merged his work on the coverage branch last week
simboss_: in order to align the branch with trunk
simboss_: then I started to merge back to trunk
simboss_: on my machine
simboss_: up to now
simboss_: I am done with the following modules
simboss_: api
simboss_: referencing
simboss_: main
simboss_: coverage
simboss_: streaming_renderer
simboss_: shape_renderer
simboss_: and also
simboss_: all the others build and install fine
simboss_: At this point I would like to commit these changes to trunk
simboss_: in order to have people evaluate them
acuster: congratulations on all that work!
Jesse_Eichar77: Nice!
simboss_: at the same time
simboss_: I can start with the plugins
simboss_: so if people are happy with the first bunch of mdoules
simboss_: before the end of the week
simboss_: I can commit all the others
simboss_: that is
simboss_: all the plugins
simboss_: after that
simboss_: we can start
simboss_: looking for bugs
acuster: you could be totally merged by the end of the week?
rgould: hi jgarnett
simboss_: I am pretty positive I can
aaime: I'd say +1 to commit the intermediate work so
simboss_: once the main modules are done
jgarnett: sorry I am late
rgould: we're talking about gc branch
aaime: that I can eventually help him
simboss_: plguins work
simboss_: is about replacing the old ones
Jesse_Eichar77: I'm +1 to merge as well
simboss_: which means not much midn ork .-)
jgarnett: I am at about 60% of IP check, hopefully that is not causing 
problems for the merge?
simboss_: It should not
acuster: how big is PMC now?
jgarnett: it is at 7
acuster: 3 for, none against: rgould, jgarnett?
jgarnett: so need 5 to cause damage I think ...
jgarnett: I am waiting for someone to send me the logs I missed.
rgould: one sec
jgarnett: unless you want to recap.
rgould: i'll paste them for you
rgould: i'm +1
jgarnett: ah looking at history, dude you got the go ahead to merge, 
pleaes just commit things as you are done with them.
jgarnett: send email to the module maintainer(s) as they are impacted 
asking for a review.
acuster: Wohoo!
jgarnett: PMC is not needed
jgarnett: (am I reading that right? we did decided to do the GC merge, 
now that it is happening only email needs to go out ... right?)
CIA-3: 03jgarnett * r20575 10geotools/gt/ext/shape/ (67 files in 18 
dirs): IP review for indexed shapefile - oh why can't we end this cruel 
experiment
simboss_: summarizing
rgould: I think so jgarnett
simboss_: tomorrow
CIA-3: 03jgarnett * r20576 10geotools/gt/ext/shape/review.txt: 
review.txt actually added jesse please review
simboss_: I will do an update + a complete build
simboss_: and then if everything builds
simboss_: I will commit everything
simboss_: and i will start wi th plugins
simboss_: just an annotation
jgarnett: how do I talk to gbot - want to add "index shapefile GPL 
clairification"
jgarnett: thanxs.
rgould: ok.. let's move on
rgould: 3) Build system improvements
rgould: aaime?
aaime: Ok
aaime: I'm trying to get clover in
rgould: (sorry, be back in a sec)
aaime: Got again an open source license for it, just need
aaime: to add a workaround for an obscure clover bug
aaime: What scares me, is that main can be coverage tested, and code 
coverate is around 30%
aaime: Sob ;-(
acuster: what does that mean?
acuster: clover only sees 30%?
aaime: Only 30% of the code is used at least once in the unit tests...
acuster: ah
Jesse_Eichar77: that really isn't very good. We're supposed to have what 
60%?
Jesse_Eichar77: 80%?
aaime: Now, I remember old times when geotools could not be released 
under 60%
aaime: Jesse, exactly
aaime: 30% is really too low
rgould is back
Jesse_Eichar77: that is. Is there anyway to tell what is missed with clover?
aaime: Now, I understand the open source project nature, and asking 80% 
may be too much
acuster: can clover id all the "non covered" code?
aaime: but 60%, well, it's something worth
aaime: Sure
aaime: Line by line
Jesse_Eichar77: ok thats good at least.
aaime: I guess we can't ask 60% for 2.2.x
aaime: But for 2.3.x, hum, what do you think?
jgarnett: I imagine for main that the other testcases cover bits of it 
in passing?
aaime: Yes Jody, you are right
jgarnett: aaime I do not think we can set a minimum bar at this time ... 
we got bigger fish to fry.
jgarnett: (yum fish)
aaime: Modularized build does indeed hide actual code coverage
aaime: I'm trying to get an aggregated report, I'll let you know
Jesse_Eichar77: that would be interesting because the coverage was 
probably higher when referencing and render were part of main.
aaime: Yet, not having a way to know what's covered and what' not is bad
Jesse_Eichar77: (and coverage)
rgould: it would be nice if we could integrate it with cruise control 
and have up to date reports available
aaime: Catalog packages for example has 0% coverage
aaime: Yes, agreed, who is going to provide cc or continuum thought?
rgould: Well we have a cc instance running here
jgarnett: both refractions and topp have offered.
jgarnett: (but lack time)
aaime: Well, once the gc merge is done, I may be interested
aaime: but I need access to those machines
jgarnett: I would like to recommend that a couple people have access to 
any box, so we do not get another droid box sending us hatemail
aaime: Indeed
rgould: aaime, i'll see if it can be arranged
jgarnett: TOPP is well situtated, they offered me access to oracle for 
testing, perhaps refractions can place nice as well? Failing that the 
OSGEO people have some machines available for abuse via telascience.
rgould: Ok. Shall we move on?
rgould: 4) WFS New Feature FID Generation
Jesse_Eichar77: lets do that last.
rgould: ok
rgould: 5) Index shapefile GPL clarification
jgarnett: Okay jesse you are still on the spot
jgarnett: Jesse your indexed shapefile module has some GPL notes .. 
these notes give people alot of grief when they look at the geoTools 
code base.
jgarnett: I would like to see about changing the note to be (C) 
2002-2004 The Open Planning Project
jgarnett: aka the fact that TOPP made the same code available via GPL, 
has *no effect* on how it was donated to the GeoTools project ...
jgarnett: cholmes is that a fair assesment?
jgarnett: jesse this is your responsibility as module maintainer, so I 
am actually asking you
Jesse_Eichar77: Sure ask away but I'm afraid I don't know what can be 
done about this.
jgarnett: is cholmes alive?
Jesse_Eichar77: He was this morning but not during this meeting.
cholmes: yeah
jgarnett: well we can review email on the subject.
cholmes: sorry, reading and trying to figure out what was asked.
jgarnett: I am trying to clairfiy something that came up in the ip check 
of indexed shapefile.
cholmes: where is the copyright to us?
cholmes: In indexed shapefile?
jgarnett: it is in a couple places, indexed shapefile is the first place 
alphabetically
cholmes: I'm pretty sure any place it is is an accident.
cholmes: From a quick jalopy with the wrong jalopy file.
cholmes: So yes, change anything that's GPL to TOPP to LGPL to geotools.
cholmes: that was always the intent.
cholmes: Anything that geotools wants from geoServer is theirs to have.
aaime: Pretty bold statement Chris
jgarnett: okay so we are cool, (C) TOPP can stay to indicate prior 
ownership ... and then (C) PMC ...
jgarnett: jesse the file has been updated, the review.txt file will need 
to make a note of the resolution.
jgarnett: that is it for this one...
Jesse_Eichar77: ok
Jesse_Eichar77: Next item then?
jgarnett: (aside: existing text is fine, notes the resolution on the GPL 
thing)
jgarnett: yes, next agenda item jesse - floor is yours
Jesse_Eichar77: OK.
Jesse_Eichar77: THis is a technical Geotools/WFS issue
cholmes: (sorry was distracted), no go ahead and remove the copyright of 
TOPP.
Jesse_Eichar77: According to the Datastore API. A datastore has the 
right to decide what the FID of a new feature is.
Jesse_Eichar77: In the Case of a WFS how can this be done? Take the case 
where the WFS is geoserver and is backed on to a PostGIS.
jgarnett: (aside: (C) to TOPP is fine, it was the GPL note that hurt 
people when they did their review, GPL==Hands off for some)
jgarnett: Jesse, can I answer?
jgarnett: And the answer is not going to be pretty.
Jesse_Eichar77: sure
jgarnett: I need you to review the FeatureStore API in GeoAPI, and then 
we can have a breakout IRC on the topic.
jgarnett: In short the Transaction, added FIDS relationship is interesting.
jgarnett: And we did not get it correct for GeoTools, the revised 
workflow is captured in GeoAPI and is something we need to migrate to.
jgarnett: \In short Transaction.commit actually reports back the added FIDs.
jgarnett: not the add operation itself.
jgarnett: because that is the only time in the workflow, for WFS and for 
a normal database, where we will actually know what the heck is going on.
jgarnett: If needed I can take 1 min and find links and we can 
walkthrough the expected workflow now, the javadocs are complete and all 
in order.
Jesse_Eichar77: That doesn't really help for 2.2 though.
jgarnett: (and the change would not break much existing code as they do 
not expect a result from transaction commit.
jgarnett: depends if you want to fix the problem for 2.2.x?
Jesse_Eichar77: I do. We can't edit WFS without it.
jgarnett: aka I do not expect GeoAPI featureStore
cholmes: FID inserts actually get more interesting with WFS 1.1
jgarnett: I expect us to revise the DataStore api to fix this problem.
jgarnett: (GeoAPI FeatureStore is just where I have documented the fix)
Jesse_Eichar77: I see. SO for now a hack will be ok and I can do the 
correct thing later on.
cholmes: There's a default of 'Generate New' (what happens currently), 
but also 'UseExisting', and 'Replace Duplicate'
jgarnett: no we can do the right fix now
jgarnett: I just needed someone to ask for it.
Jesse_Eichar77: I'm not sure I believe you.
jgarnett: okay?
jgarnett: comes down to a LockRequest and a LockResponse object...
jgarnett: 
http://geoapi.sourceforge.net/2.0/javadoc/org/opengis/feature/LockRequest.html^ 

Jesse_Eichar77: thanks
jgarnett: 
http://geoapi.sourceforge.net/2.0/javadoc/org/opengis/feature/LockResponse.html^ 

jgarnett: and a walkthrough of how this works is here in the transaction 
javadoc... lets go through it.
jgarnett: 
http://geoapi.sourceforge.net/2.0/javadoc/org/opengis/feature/Transaction.html^ 

jgarnett: although ... perhaps the commit messge is more useful.
jgarnett: 
http://geoapi.sourceforge.net/2.0/javadoc/org/opengis/feature/Transaction.html#commit^ 
()
jgarnett: darn sorry this is the lock problem, which we also do not have 
licked ...
jgarnett: bleah. give me a second.
jgarnett: okay jesse I do not have this one documented, it was talked 
about in emai on the GeoAPI list.
jgarnett: do you need me to hunt down the informaiton?
jgarnett: The approach is similar, in that the FIDs need to be reported 
at the end of the transaction.
Jesse_Eichar77: Right. So how is the mapping between an added feature 
and a fid made?
Jesse_Eichar77: IE I add 10 features.
Jesse_Eichar77: commit.
Jesse_Eichar77: now I want to edit the 3rd feature added
jgarnett: it is made by the WFS, the new fids are returned as part of 
the transaction response.
jgarnett: Jesse?
jgarnett: (think he is chatting on the udig channel)
Jesse_Eichar77: I'm here
jgarnett: did my answer make sense, features do not get a FID until they 
are commited.
Jesse_Eichar77: so the only way is to add 1 feature. Commit.
Jesse_Eichar77: then add a feature.
Jesse_Eichar77: commit
jgarnett: that is the way it works right now with the reader ...
Jesse_Eichar77: add another...
Jesse_Eichar77: commit.
Jesse_Eichar77: now I have the fid.
Jesse_Eichar77: that I want.
jgarnett: or add a bunch and get back a list of fids...
Jesse_Eichar77: THat doesn't work because I want the fid of the 3rd 
feature added!
jgarnett: If I can guess, you would like to use FID to distingish 
between the features before they are commited...
cholmes: And the WFS doesn't return them in the proper order?
jgarnett: so I need to ask why should WFS return them in an order?
Jesse_Eichar77: Chris that's my question. Is there a way to map between 
added features and the fids assigned to them by the datastore.
jgarnett: is the third one magic? the WFS is organizing these things not 
you ...
Jesse_Eichar77: no magic number.
Jesse_Eichar77: its just and example.
jgarnett: okay ... so I am not seeing the problem.
Jesse_Eichar77: ok I'll start over again.
jgarnett: (jody is feeling thick)
cholmes: I think it probably returns them in the order intserted...
cholmes: Let me check the spec and see if it says anything specific 
about order.
Jesse_Eichar77: chris I get a collection back so I can't guarantee that. 
If I had a guarantee then that would be fine.
jgarnett: you get a collection back from what?
jgarnett: addFeatures?
jgarnett: So if order was significant we would be returning a list ...
Jesse_Eichar77: for new yes. I assume that's what I get back in the new 
model as well.
Jesse_Eichar77: OK my example.
cholmes: From the spec: 'One <InsertResult> element is reported per 
<Insert> element in the
cholmes: request. The insert results are reported in the order in which 
the <Insert> operations
cholmes: were contained in the <Transaction> element.'
cholmes: So they should be ordered in the response.
jgarnett: Jesse we could change the API to sorted set .. .where sort 
order is the order inserted.
cholmes: So I'd say make the WFS datastore response be a List, and then 
you can cast the collection to a list when you know you're working with WFS?
jgarnett: and that would not break any code.
Jesse_Eichar77: ok if the API agrees (jody) then I'm ok.
Jesse_Eichar77: yeah I'm totally ok with that.
jgarnett: sorted set woudl be more accurate and would not break code.
jgarnett: but jesse, I reallly would like to revisit the add/locking 
transaction interaction - I understand that 2.2.x is not the time.
cholmes: Indeed, our DataStore code is inspired by WFS, so probably 
should have been some sort of orderered response in the first place.
jgarnett: But sorting this out will be on my QA list before GeoTools 3.0
Jesse_Eichar77: sounds good. I'm working on WFS as you know and right 
now it is not a very happy beast when it comes to editing.
Jesse_Eichar77: for example.
Jesse_Eichar77: I add a feature.
Jesse_Eichar77: then update the feature using a FidFilter.
Jesse_Eichar77: I commit.
Jesse_Eichar77: I get an exception because the fid of the feature has 
been changed on commit!
Jesse_Eichar77: but it sounds like I have enough information that's to 
chris's talk about WFS transaction.
Jesse_Eichar77: and I'll see if I can't ensure that it will be forwards 
compatible.
jgarnett: trying to understand.
Jesse_Eichar77: I just find it hard to program against a id that might 
change sometime in the future when the user decides to commit the 
transaction.
jgarnett: so the problem is that you would like to use FidFilter to 
update contents...
jgarnett: right, that is why I like to provide a null ID when the 
feature does not really have one yet.
jgarnett: but that would just kill you
Jesse_Eichar77: Definately. How the heck can I edit a single feature 
when there is no way to identify it!
jgarnett: and you cannot just modify the selected feature...
jgarnett: thinking ...
jgarnett: the memory diff thing used to make temp1, temp2, temp3 for the 
FID number as a temporary hack.
jgarnett: I think for shapefile they replaced it to try and guess the 
max row number ... and left things open for threading issues.
jgarnett: do you have any information on this Jesse?
Jesse_Eichar77: I changed that so that it is indexed and there is a 
seperate fid index for shapefile that is marginally thread safe 
(wouldn't want to be my house on it).
jgarnett: well our template for this API is WFS, and they do not expect 
to hold information for editing before it has been commited.
jgarnett: (oh wait they bloody well do, you can add content and then 
update it...)
jgarnett: but it just so happens that the upddates will not be based on 
fid, because it is one big XML request)
Jesse_Eichar77: Its not useable though... At least as far as I can tell.
jgarnett: that is because our WFS implementation holds everything in memory
jgarnett: until you send the poor thing off in the commit message.
Jesse_Eichar77: I've been tempted change the add feature workflow to 
actually create a feature lock it then use that feature and fid for 
future updates.
Jesse_Eichar77: but that's a nightmare.
jgarnett: So a real answer here is that WFS needs you to do a 
transaction per commit, or you need to accept that the FID is not fixed 
until the transaction has been committed.
Jesse_Eichar77: That's no answer.
jgarnett: thinking about that ..
jgarnett: create, lock the resulting fid, hack away at it ... and then 
only release it to others when it has been set up the first time.
jgarnett: (nope I would not let that past validaiton)
Jesse_Eichar77: good point.
jgarnett: concerning the answer ...
Jesse_Eichar77: it was a terrible hack any ways.
jgarnett: so jesse can you loose all state on new features when commit 
occurs?
jgarnett: or do you want to take hacking new content out of the hands of 
the datastore api, and then send things out in batch...
jgarnett: aka if you lose the state, then you will be okay with using 
the "temp1" fid for editing ...
jgarnett: ah, hense your request for order.
Jesse_Eichar77: How could you enforce that? It is a guaranteed way to 
cause millions of bugs. Yes I can do that but I really do not like it 
cause it require a isDisposed or some such methods on feature.
jgarnett: they have selected fid #3, and after the commit goes out you 
want to change the selected fid from "temp3" to "!@#$#@$"
jgarnett: or whatever WFS returned.
rgould: Shall I post the logs or do you guys want this recorded?
jgarnett: heh, maybe end the meeting and we will keep bashing.
jgarnett: I think acuster wanted to chat.