Re: [PyWrapper-devel] [tdwg-tapir] RE: WG: tapir: capabilities

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

I agree that logging via GUID doesnt help in many cases where the =
provider wants to know what was searched for.

But searching on a portal-cache to find data from 20 different providers =
in 1 search and then sending of 20 log requests could also be annoying. =
Plus the burden of the portal of checking the registry if a providers =
really wants logging.

The most efficient is probably a portal specific logging as Donald =
suggests. But then providers would have a hard time agglomerating the =
logging data from several totaly different portals.

To get a comparable logging across different portals though it seems to =
me that Renatos suggestions are worth a try. It would definitely need =
guidlines for portal developers to know when to use a log request and =
how to use it. How to treat paging and map data are good examples where =
there is no obvious correct behaviour.

-- Markus
 =20

> -----Urspr=FCngliche Nachricht-----
> Von: tdw...@li...=20
> [mailto:tdw...@li...] Im Auftrag von=20
> John R. WIECZOREK
> Gesendet: Montag, 24. Juli 2006 18:28
> An: ro...@td...
> Cc: PyWrapper Developers mailing list; tdw...@li...
> Betreff: Re: [PyWrapper-devel] [tdwg-tapir] RE: WG: tapir:=20
> capabilities
>=20
> Logging that a GUID was used isn't sufficient; it doesn't=20
> tell how the data were used. What I'm after is to log the=20
> actual query that would have had to go to the provider to=20
> produce the results used. The example in your second example=20
> shouldn't happen, the query should specify a record limit per=20
> provider.
>=20
>=20
> On 7/24/06, Roger Hyam <ro...@td...> wrote:
>=20
>=20
> 	I thought this sounded like a good idea but since=20
> reading Renato's message I am now confused.
> =09
> 	If a user does a search on a portal and gets 100=20
> results and looks at the first 10 does the provider of the=20
> 11th record get notified? Their data has been used because it=20
> has been given in the count.  Another example would be if a=20
> portal gave a distribution map to 10km squares based in data=20
> from multiple providers. Each data point is made from several=20
> suppliers data and removal of any one supplier's data may not=20
> change the map. Do we notify them all?
> =09
> 	I could envisage a GUID based system just about. The=20
> call to the log function would basically say "Some one has=20
> accessed the data that I got from you that you tag with this=20
> GUID" but I can't see how this would work on a search based=20
> system. The log call would mean "Some one searched for=20
> something that made used of data I got from a search I did on=20
> you once".
> =09
> 	So really the only service we need is a GUID based one.=20
> Perhaps extending the LSID resolution spec would be more appropriate?
> =09
> 	Roger
> =09
> =09
> =09
> 	Renato De Giovanni wrote:=20
>=20
> 		Hi John,
> 	=09
> 		Implementing the log request was never a=20
> problem. We discussed about=20
> 		that again during the Madrid meeting, and only=20
> after that a feature=20
> 		freeze was suggested. It's true that PyWrapper=20
> is being adjusted now=20
> 	=09
> 		to conform to the new specs, and considering=20
> that DiGIR2 (or wasabi)=20
> 		postponed implementation of TAPIR, I suppose it=20
> should not be a big=20
> 		problem to make additional changes if necessary.
> 	=09
> 		The main problem I had with the log request was=20
> that it would=20
> 	=09
> 		probably not solve the issue behind it, which=20
> is to track usage by=20
> 		data aggregators. I still have the same=20
> feeling, and I can easily=20
> 		imagine situations when it would not be easy or=20
> even possible to=20
> 		translate searches on top of cached databases=20
> to TAPIR requests.
> 	=09
> 	=09
> 		But maybe I'm wrong, and if you all think it's=20
> a good feature then we=20
> 		can try to include it. However, I do think that=20
> providers should be=20
> 		able to advertise as the part of capabilities=20
> if they want to receive=20
> 	=09
> 		log requests or not.=20
> 	=09
> 		To me it also sounds like a new operation,=20
> especially if it's only=20
> 		related to search. It could make sense for=20
> view, inventory and=20
> 		metadata operations. Maybe capabilities too.=20
> But it doesn't make=20
> 	=09
> 		sense for ping. Well, maybe it could make sense=20
> for ping if the data=20
> 		aggregator monitors provider status and accepts=20
> similar requests on=20
> 		top of its results...
> 	=09
> 		So, yes, it could be a new attribute "logOnly"=20
> as part of the=20
> 	=09
> 		operationRequestGroup with an answer=20
> </received> (just after the=20
> 		response header). And we could add an attribute=20
> "acceptLogRequests"=20
> 		in the <operations> element in capabilities=20
> responses. The other=20
> 	=09
> 		option would be to include a new operation, but=20
> maybe it's better to=20
> 		just have it as an optional attribute for all=20
> operations.
> 	=09
> 		Best Regards,
> 		--
> 		Renato
> 	=09
> 		On 19 Jul 2006 at 10:32, John R. WIECZOREK wrote:
> 	=09
> 		 =20
>=20
> 			I appreciate that you will consider=20
> this request. I always thought
> 			it=20
> 			would be trivial to implement. Your=20
> simulation mode sounds very much
> 			like what I had in mind. I hadn't=20
> thought it necessary to get a=20
> 		=09
> 			response from a log request, but if=20
> there was a simple response, it
> 			could be used as a ping, or it could be=20
> used to retry logging until
> 			the provider did respond. So, something=20
> like <log request received>.
> 		=09
> 			I think the addition oflogOnly=20
> attribute is a good one, and could=20
> 			apply to every request type.=20
> 		=09
> 			Javi, I don't disagree that portals=20
> SHOULD log the data usage,=20
> 			especially to cover the situations=20
> where a provider doesn't respond.
> 		=09
> 			I also think that having the=20
> information logged at the provider is a
> 			responsible course of action, since=20
> they will have immediate access
> 			to the usage statistics that way. It=20
> will be much easier for a
> 			portal=20
> 		=09
> 			builder to send log requests than it=20
> will be to build the=20
> 			infrastructure and interfaces to logs,=20
> therefore it is more likely
> 			to=20
> 			actually get done.=20
> 		=09
> 		=09
> 			On 7/18/06, Javier de la Torre=20
> 			<ja...@mn...>=20
> <mailto:ja...@mn...>  wrote:=20
> 			    I am not sure about this,
> 			   =20
> 			    I still think that portals should=20
> be gathering this data and
> 			making
> 			    it available for data providers...=20
> 			   =20
> 			    But in any case if you like it then=20
> I agree with MArkus that the
> 		=09
> 			best
> 			    is to include another parameter in=20
> the operationRequestGroup.
> 			   =20
> 			    I havent checked but what happens=20
> if you do an extension there
> 			with
> 			    an attribute that is implementation=20
> specific? A qualified
> 		=09
> 			attribute.
> 			    Will this still validate against=20
> our schema? You were
> 			discussing
> 			    about qualification of attributes before no?
> 			   =20
> 			    Javi.
> 			   =20
> 			    On 18/07/2006, at 10:41, D=F6ring,=20
> Markus wrote:=20
> 		=09
> 			   =20
> 			    > John,
> 			    > all changes going on with TAPIR=20
> right now are really only
> 			changes
> 			    > in terminology or removing=20
> inconsistencies we did not detect
> 			before
> 			    > we started the documentation and=20
> final implementation.=20
> 		=09
> 			    >
> 			    >
> 			    > But nevertheless I would support=20
> your request. Especially from
> 			the
> 			    > implementation side of view this=20
> is a trivial change to the
> 			code.
> 			    > So why dont add it? Just some=20
> additional thoughts:=20
> 		=09
> 			    >
> 			    >
> 			    > - Ive added a "simulation" mode=20
> already to my code where no
> 			SQL
> 			    > gets executed but just logged. So=20
> you can test
> 			configurations
> 			    > without risking sending off=20
> killer statements. Thats similar
> 		=09
> 			to=20
> 			    > logOnly I guess, returning=20
> nothing but diagnostics. What would
> 			you
> 			    > suggest to be returned for a=20
> logOnly request? just the empty
> 			TAPIR
> 			    > envelope? Nothing? <OK>?
> 			    >
> 		=09
> 			    > - would this log-only request not=20
> be needed for all requests?
> 			at=20
> 			    > least for inventories? So it=20
> would be easiest to have a new
> 			logOnly
> 			    > parameter in the header or=20
> "request element" just after the
> 		=09
> 			header?
> 			    > something like <search logOnly=3D"true">
> 			    >
> 			    >
> 			    >
> 			    > -- Markus
> 			    >
> 			    >
> 			    >> -----Urspr=FCngliche Nachricht-----
> 			    >> Von:=20
> 			pyw...@li...
> 			=20
> 			    >>=20
> [mailto:pyw...@li...
> 			] Im
> 			    >> Auftrag von John R. WIECZOREK
> 			    >> Gesendet: Montag, 17. Juli 2006 23:30=20
> 			    >> An: Renato De Giovanni
> 			    >> Cc:=20
> 			pyw...@li...=20
> <mailto:pyw...@li...> ; tdwg-
> 			    ta...@li...
> 			=20
> 			    >> Betreff: Re: [PyWrapper-devel]=20
> [tdwg-tapir] RE: WG: tapir:
> 			    >> capabilities
> 			    >>
> 			    >> A little off topic, but it=20
> occurs to me that a great deal
> 			of
> 			    >> work is still ongoing with=20
> TAPIR, which suggests to me that
> 		=09
> 			    >> it may be warranted to re-state=20
> my request for a simple
> 			    >> message type - a log request.=20
> This request would be the
> 			same
> 			    >> as a search request, except that=20
> the caller doesn't need a
> 		=09
> 			    >> response. Providers would use=20
> this type of request to log=20
> 			    >> data usage if the data were=20
> retrieved from a cache
> 			elsewhere.
> 			    >> I remember talking about this in=20
> Berlin, at which time
> 		=09
> 			there
> 			    >> was supposed to be a feature=20
> freeze. Clearly we've gone
> 			    >> beyond that, so I'm requesting it again.=20
> 			    >>
> 			    >>
> 			    >> On 7/17/06, Renato De Giovanni=20
> 			<re...@cr...>=20
> <mailto:re...@cr...>  wrote:
> 			    >>
> 			    >>Hi,
> 			    >>
> 			    >>If I remember well, the "view"=20
> operation was re-included in=20
> 			    the=20
> 			    >>protocol just to handle query=20
> templates, especifically
> 		=09
> 			    >> for TapirLite
> 			    >>providers. So if someone wants to=20
> query a provider using
> 			some
> 			    >>external output model that should=20
> be dynamically=20
> 			    >> parsed, then the
> 			    >>"search" operation must be used=20
> instead (using either
> 		=09
> 			    >> XML or simple
> 			    >>GET request). View operations are=20
> really bound to query
> 			    >> templates,=20
> 			    >>and they are not allowed to=20
> specify "filter" or
> 			    >> "partial" parameters.
> 		=09
> 			    >>--
> 			    >>Renato
> 			    >>
> 			    >>On 17 Jul 2006 at 21:26, "D=F6ring,=20
> Markus" wrote:=20
> 			    >>
> 			    >>> I was just about to edit the=20
> schema and realizing
> 		=09
> 			    >> that output models
> 			    >>> are only specified for=20
> searches. but what about
> 			    >> views? they use
> 			    >>> query templates, yes. but only=20
> the ones listed in=20
> 			    >> capabilities? we
> 		=09
> 			    >>> should have dynamic ones here=20
> as well I think. And
> 			    >> they link back to
> 			    >>> static/dynamic models.
> 			    >>>
> 			    >>> So should models maybe become a=20
> seperate section not tight
> 		=09
> 			    to=20
> 			    >>> search/view operations? I am=20
> going to modify the
> 			    >> schema nevertheless
> 			    >>> already to accomodate the=20
> changes below - ignoring
> 			    >> views for now.
> 		=09
> 			    >>>
> 			    >>> Markus
> 			   =20
>=20
> 		_______________________________________________
> 		tdwg-tapir mailing list
> 	=09
> 		tdw...@li...=20
> <mailto:tdw...@li...>=20
> 		http://lists.tdwg.org/mailman/listinfo/tdwg-tapir
> 	=09
> 	=09
> 		 =20
>=20
>=20
>=20
> 	--=20
> =09
> 	-------------------------------------
> 	 Roger Hyam
> 	 Technical Architect
> 	 Taxonomic Databases Working Group
> 	-------------------------------------
> 	=20
> 	http://www.tdwg.org <http://www.tdwg.org>=20
> 	 ro...@td...
> 	 +44 1578 722782
> 	-------------------------------------
>=20
> 	_______________________________________________
> 	tdwg-tapir mailing list
> 	tdw...@li...
> 	http://lists.tdwg.org/mailman/listinfo/tdwg-tapir=20
> <http://lists.tdwg.org/mailman/listinfo/tdwg-tapir>=20
> =09
> =09
> =09
>=20
>=20
>=20