plastic-devs Mailing List for PLASTIC (Page 5)
Brought to you by:
johndavidtaylor,
thomasboch
You can subscribe to this list here.
| 2005 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(15) |
Nov
(11) |
Dec
(9) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2006 |
Jan
(1) |
Feb
(110) |
Mar
(36) |
Apr
(55) |
May
(42) |
Jun
|
Jul
(42) |
Aug
(14) |
Sep
(34) |
Oct
(7) |
Nov
(35) |
Dec
(5) |
| 2007 |
Jan
(3) |
Feb
(5) |
Mar
(18) |
Apr
(10) |
May
(10) |
Jun
(9) |
Jul
(27) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Mark T. <m.b...@br...> - 2006-09-21 17:54:55
|
On Thu, 14 Sep 2006, Noel Winstanley wrote: > I'd prefer 4) as > - I've just implemented it in 5 mins (a good sign). > - it requires no knowledge of SSAP specs encoded in the message- > sender. > - The sender can just pass-through whatever data was received from > the SSAP service response. > - requiring a client to understand a lot of SSAP, (and repeat for > SIAP, SLAP, VOEvent, etc, etc) is going to make everyone's lives > harder, and goes against 'do one thing well' > - it's more the message receiver's task to understand the > particular specs it supports. Noel, SPLAT can now send and receive this message - you can find a suitable version at http://andromeda.star.bris.ac.uk/starjavalib/splat.jnlp Seems to work OK with the new workbench. Have a play and if there are no problems I'll get Peter to update the 'official' jnlp version that's referenced off the plastic web page. I've also updated the message definition at http://eurovotech.org/twiki/bin/view/VOTech/PlasticMessagesProposal Mark -- Mark Taylor Astronomical Programmer Physics, Bristol University, UK m.b...@br... +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/ |
|
From: Mark T. <m.b...@br...> - 2006-09-21 17:28:10
|
On Sun, 17 Sep 2006, John Taylor wrote: > Hi Mark, > The solutions you propose here seem to be pretty straightforward to me. > I agree that of two apps running on the same machine under different > users does (at present) create a security hole. I think I've been > blinkered by the fact that most people will be running plastic on single > user machines, but nevertheless there's a significant proportion of > users that will want to run it on shared machines. [A propos - does > anyone know of any users that have actually done this....I know that the > Workbench scans for a free port for the xml-rpc server....but I'm not > certain that it does for the RMI server (clearly it should!)]. > > We _could_ make it clear that Plastic is not supposed to be a secure > protocol, and discourage app writers from plastic-enabling the > functionality you describe in GAIA without adding in their own > security. This seems a bit of a pity, but I'd be interested in knowing > what other people think so I'll bring it up in Moscow and see if any > potential plastic developers have a view. I'd like to get a feel for > how many developers we'd discourage with the extra complexity, versus > how many we'll attract by the extra security! The thing that bothers me I'm not expecting the additional burden to be high. Maybe passing an extra (fixed) cookie as an argument of all the method calls at worst. > is when people ask "why use Plastic, instead of messaging system X", our > usual response is: because it doesn't try to be an all-singing > all-dancing messaging protocol and as a result it's very simple to > use. Will people be wanting encryption next? And lo! SOAP was born. I certainly wasn't anticipating going as far as encryption for normal message contents. > A few comments on the ideas you put forward below: > 1) If we implement the "cookies" as part of the API, they won't need to > be sent as a digest since they won't be part of the regular argument > list, so won't be logged. On the other hand it wouldn't hurt. I wasn't intending sending cookies as digests - the solution I came up with for GAIA was a bit of a special case. > 2) In a way, the obfuscated xml-rpc URL already acts as a > cookie/password - is there anything similar we could do for RMI and let .. although it's down to the hub implementation whether/how it does this obfuscation. But I agree that it can in practice do the job. > the transport protocol take the strain? Perhaps when Noel has finished > reading "Java security for beginners" he can tell us. Mark -- Mark Taylor Astronomical Programmer Physics, Bristol University, UK m.b...@br... +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/ |
|
From: John T. <jon...@gm...> - 2006-09-17 17:16:22
|
Hi Mark, The solutions you propose here seem to be pretty straightforward to me. I agree that of two apps running on the same machine under different users does (at present) create a security hole. I think I've been blinkered by the fact that most people will be running plastic on single user machines, but nevertheless there's a significant proportion of users that will want to run it on shared machines. [A propos - does anyone know of any users that have actually done this....I know that the Workbench scans for a free port for the xml-rpc server....but I'm not certain that it does for the RMI server (clearly it should!)]. We _could_ make it clear that Plastic is not supposed to be a secure protocol, and discourage app writers from plastic-enabling the functionality you describe in GAIA without adding in their own security. This seems a bit of a pity, but I'd be interested in knowing what other people think so I'll bring it up in Moscow and see if any potential plastic developers have a view. I'd like to get a feel for how many developers we'd discourage with the extra complexity, versus how many we'll attract by the extra security! The thing that bothers me is when people ask "why use Plastic, instead of messaging system X", our usual response is: because it doesn't try to be an all-singing all-dancing messaging protocol and as a result it's very simple to use. Will people be wanting encryption next? And lo! SOAP was born. Another thing to note is that the Workbench allows far more dangerous access to a user's resources than any Plastic app currently does. That's not an excuse for us not to consider the issues, but clearly we need to work with Noel and whatever solutions he comes up with for the same issues. A few comments on the ideas you put forward below: 1) If we implement the "cookies" as part of the API, they won't need to be sent as a digest since they won't be part of the regular argument list, so won't be logged. On the other hand it wouldn't hurt. 2) In a way, the obfuscated xml-rpc URL already acts as a cookie/password - is there anything similar we could do for RMI and let the transport protocol take the strain? Perhaps when Noel has finished reading "Java security for beginners" he can tell us. 3) If we do bring in security, then we'll have to think about how to make it backwards compatible with what we have now. At some point in the future I see us bringing in a new API based on these PlasticRemould discussions, but keeping the old one around for as long as they're wanted. I guess the simplest thing to do is to warn the user that an app is trying to connect via the old API and ask them to OK it. Alternatively, we could wall off the "old" API apps so they can't send messages to the new ones (which sounds like a recipe for user confusion if ever I heard one). Apologies if this is garbled nonsense. It was my brother's wedding last night, and I'm sitting in Zurich airport with something of a headache. John Mark Taylor wrote: > On 14 Sep 2006, at 23:17, John Taylor wrote: > > >> My gut feeling is that I'd prefer to trade security for simplicity and >> assume that applications on a user's desktop are "friendly". That >> > > I think that's a fair assumption, where "on a user's desktop" > means "in a process owned by that user". > > >> said, we should make it clear to app writers what we are offering. Can >> anyone think of something really nasty that an app could do through >> Plastic that it couldn't do already? >> > > I've already come across a case of this. JAC wanted a GAIA-specific > PLASTIC message which allowed PLASTIC clients to perform remote > control. The way this would work is to send a snippet of Tcl > code to GAIA, for GAIA to execute in its own Tcl interpreter. > This could in principle include things like "exec rm -rf *". > So you only want clients whose intentions you trust to be > able to send requests to the PLASTIC hub. As it stands, anyone > who can make an XML-RPC or (more easily) RMI connection to the > hub can do this. The way I solved this was to have a GAIA-specific > 600 file in the user's home directory with a magic cookie in it, > which the client has to be able to read for GAIA to accept the > message. Sending the cookie on its own is not good enough, > since interloper applications might snoop on messages going through > the hub (using the as-yet unimplemented logging messages that > started this conversation off) and copy the cookie. So in > fact the message arguments are (1) the Tcl script snippet and > (2) an MD5 hash of the script snippet with the cookie. Since > third party applications (registered with the hub, but without > read access to the cookie file) never see the cookie, I think this > is reasonably secure. > > Admittedly, permitting this kind of remote control is an inherently > risky thing to do. > > >> enough to scan for it. So I think the best thing to do is >> a) make the .plastic file have 400 permissions >> > > .. and contain some kind of password which must be presented > to the hub in order to register > > >> b) disallow connections from remote machines _unless they have been >> explicitly enabled_ [there are some potential uses of this, as >> noted by Pierre]. >> > > I think that's about right, with (a) being the important one and > (b) being, well, might as well since it's fairly easy. I'd add one > thing, which is that each application should be given a secret key by the > hub at registration time, which it needs to present with each message > request. This would perform the same sort of function as the > existing client ID, but other apps would not know what it was. > This would prevent one application pretending to be another one > and, for instance, unregistering it. I'm not really motivated > here by serious security concerns (it would be more of an > annoyance than anything), but it seems untidy/wrong that there's > nothing in the specification to prevent this sort of thing > happening. > > Mark > > |
|
From: Noel W. <Noe...@ma...> - 2006-09-15 16:28:29
|
The AstroGrid Team is pleased to announce the 2006.3.rc3 release of Workbench and Astro Runtime. http://www.astrogrid.org/desktop Changes in this version include: New Features: o Astroscope, Myspace, Lookout now send Spectra Plastic Messages. o Within Query Builder, it is now possible to sort columns within metadata display. Clicking on column header sorts alphabetically, either ascending or descending. Issue: 1790. o Added community list combo box to login dialogue Issue: 1788. o Added votable manipulator (STIL) to AR interface Issue: 1585. o Reduced executable size, by stripping unused classes. o Added a generic disk-caching component. Use it to cache registry resource objects and documents for a day or so. o Implemented standard IVOA registry interface. Issue: 1413. Fixed bugs: o Added caching of application info to improve responsiveness of workflow editor and application launcher. Issue: 1814. o Astroscope now updates display when plastic apps register Issue: 1777. o Astroscope handles non-standard siap responses Issue: 1780. o Sorting of tables before metadata display. Correction of bug in metadata display preventing display when archive separately chosen. Correction of displaying of brackets in Query Builder Issue: 1768. o Fixed parameterized workflow dialogue, and got workflows running again. Issue: 1505. o Optimized registry interface - uses a streaming parser, improved data model. Marked old implementation as deprecated. Issue: 1574. o can now delete from list of workfows Issue: 1710. o Reorganised front UI tabs Corrected list of parameterised workflows Issue: 1760,1754. o Fixed helioscope to have a save button that was mising Changed 'To To Top' to 'Go To Top' and set shortcut key to 'G' for Astro and helioscope. Issue: 1719. o Fixed 'enter' or 'return' to work on Registry panel for workflow see bugzilla ticket 1366 Issue: 1366. o Rephrased pop-up text on plastic registration. Issue: 1734. o Removed "list" plastic applications button from all variants except plastic hub-only AR. Issue: 1757. o Assorted other fixes: hub now self registers properly and responds to messages such as getName. o XML-RPC return type conversion would fail in certain situations. XML-RPC plastic clients couldn't use getRegisteredIds for example. Issue: 1762. o Improved concurrency behaviour of Plastic hub, added more tests. Made hub more robust to duff clients. Hub now only unregisters xmlrpc clients when there's an IOException, not if the client faults. Changes: o Improved formatting of registry entries o Reworked registry google / application launcher / astroscope components to use new registry interface. Deleted applications and services aren't displayed anymore. Removed features: o Reverted PW-launcher to load list of workflows from wiki page. Enjoy! -The AstroGrid Team --- http://wiki.astrogrid.org/bin/view/Main/NoelWinstanley Senior Java Developer, AstroGrid Project Jodrell Bank Observatory, University of Manchester |
|
From: Mark T. <m.b...@br...> - 2006-09-15 09:01:15
|
Hi Isa, On Fri, 15 Sep 2006, Isa Barbarisi wrote: > yes you are right, having the column name could be a mess at the time > you have to parse the keys. > I was just thinking that maybe having as much information as possible > could be useful in the future for other Spectra Viewers, at least the > fields of the row coming from SSAP servers. I can see your point, but I think you need to strike a compromise to avoid namespace clashes if everything's going in one map. > Are you going to Moscow? I'm afraid not. Mark -- Mark Taylor Astronomical Programmer Physics, Bristol University, UK m.b...@br... +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/ |
|
From: Mark T. <m.b...@br...> - 2006-09-15 08:57:18
|
On 14 Sep 2006, at 23:17, John Taylor wrote:
> My gut feeling is that I'd prefer to trade security for simplicity and
> assume that applications on a user's desktop are "friendly". That
I think that's a fair assumption, where "on a user's desktop"
means "in a process owned by that user".
> said, we should make it clear to app writers what we are offering. Can
> anyone think of something really nasty that an app could do through
> Plastic that it couldn't do already?
I've already come across a case of this. JAC wanted a GAIA-specific
PLASTIC message which allowed PLASTIC clients to perform remote
control. The way this would work is to send a snippet of Tcl
code to GAIA, for GAIA to execute in its own Tcl interpreter.
This could in principle include things like "exec rm -rf *".
So you only want clients whose intentions you trust to be
able to send requests to the PLASTIC hub. As it stands, anyone
who can make an XML-RPC or (more easily) RMI connection to the
hub can do this. The way I solved this was to have a GAIA-specific
600 file in the user's home directory with a magic cookie in it,
which the client has to be able to read for GAIA to accept the
message. Sending the cookie on its own is not good enough,
since interloper applications might snoop on messages going through
the hub (using the as-yet unimplemented logging messages that
started this conversation off) and copy the cookie. So in
fact the message arguments are (1) the Tcl script snippet and
(2) an MD5 hash of the script snippet with the cookie. Since
third party applications (registered with the hub, but without
read access to the cookie file) never see the cookie, I think this
is reasonably secure.
Admittedly, permitting this kind of remote control is an inherently
risky thing to do.
> enough to scan for it. So I think the best thing to do is
> a) make the .plastic file have 400 permissions
.. and contain some kind of password which must be presented
to the hub in order to register
> b) disallow connections from remote machines _unless they have been
> explicitly enabled_ [there are some potential uses of this, as
> noted by Pierre].
I think that's about right, with (a) being the important one and
(b) being, well, might as well since it's fairly easy. I'd add one
thing, which is that each application should be given a secret key by the
hub at registration time, which it needs to present with each message
request. This would perform the same sort of function as the
existing client ID, but other apps would not know what it was.
This would prevent one application pretending to be another one
and, for instance, unregistering it. I'm not really motivated
here by serious security concerns (it would be more of an
annoyance than anything), but it seems untidy/wrong that there's
nothing in the specification to prevent this sort of thing
happening.
Mark
--
Mark Taylor Astronomical Programmer Physics, Bristol University, UK
m.b...@br... +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/
|
|
From: Isa B. <Isa...@sc...> - 2006-09-15 08:40:03
|
Mark Taylor wrote: >On Thu, 14 Sep 2006, Isa Barbarisi wrote: > > > >>Hi, >>I also agree with point 4, it seems to be the most flexible solution, >>with just a couple of remarks. >>It should be clear in the standard that all the information from a SSAP >>Servers has to be sent and no information should be lost, in order to >>not lose important metadata. >>The other point is that we would prefer to have the keys of the map as a >>combination of UCD and utype (both of them) so that if one of them is >>not found (maybe because the SSAP server doesn't have it) we can make >>the search on the other one. Eventually all the fields of the row (like >>ucd, utype, name...) could be sent. >> >> > >If they're all going in the same map I don't think that column name >should be included as well. It's probably reasonably safe to say >that there won't be (confusing) namespace clashes between UCD and >utype, but column names could be anything (and are unlikely to be >machine-intelligible), so it seems like asking for trouble to use >them as keys in the same map. > >Mark > > > Hi Mark, yes you are right, having the column name could be a mess at the time you have to parse the keys. I was just thinking that maybe having as much information as possible could be useful in the future for other Spectra Viewers, at least the fields of the row coming from SSAP servers. Are you going to Moscow? Cheers, Isa |
|
From: Noel W. <Noe...@ma...> - 2006-09-14 22:33:45
|
On 14 Sep 2006, at 23:17, John Taylor wrote: >> <snip> > Certainly, Plastic is pretty insecure at the moment. I think we > probably need to think carefully before trying to make it completely > secure....its selling point is that it's simple, and we don't want to > complicate it with extra baggage unless we have to. If we want an all > purpose messaging system then perhaps we should pick one off the > shelf. > > As noted on > > http://wiki.eurovotech.org/twiki/bin/view/VOTech/PlasticRemould > > there is a danger that some script kiddie could take control of > your hub > remotely. Even if we did randomise the RMI port, it would be easy > enough to scan for it. So I think the best thing to do is > a) make the .plastic file have 400 permissions > b) disallow connections from remote machines _unless they have been > explicitly enabled_ [there are some potential uses of this, as > noted by > Pierre]. in fact, it's possible to do this already using the java security policy stuff. One of the built-in permissions lets the user control the behaviour of sockets - in particular, from which ip addresses a server socket is allowed to accept a connection from. I should think this cleanly controls the XML--RPC, RMI and any other network interfaces into the hub. So, by providing the user with a suitable security policy file, and showing them how to execute java with the security manager turned on, it's possible to have a secure, configurable system with no additional code. (as you might have gathered, I'm halfway through reading 'My First Java Security For Dummies For Idiots'.) 'course, actually running java with the security manager enabled is probably beyond the ken of most users - hence me looking at installers for the workbench / hub - once users are running a shell script / shortcut / whatever, it becomes easier to pass in the security policy on the commandline. Programattically controlling the policy used is much harder, unsurprisingly. If anyone's got some experience of this, I'd be interested. cheers noel. |
|
From: Alasdair A. <aa...@as...> - 2006-09-14 22:29:59
|
John Taylor wrote: > My gut feeling is that I'd prefer to trade security for simplicity and > assume that applications on a user's desktop are "friendly". Agreed! > That said, we should make it clear to app writers what we are > offering. Can > anyone think of something really nasty that an app could do through > Plastic that it couldn't do already? Erm, can't think of anything off the top of my head. Al. |
|
From: John T. <jon...@gm...> - 2006-09-14 22:16:57
|
Mark Taylor wrote: > On Thu, 14 Sep 2006, Alasdair Allan wrote: > > >> Mark Taylor wrote: >> >>> Longer term, I'd like to see hub implementations supporting a >>> message or messages which is/are sent (to anyone who's asked) >>> logging all traffic. This would be able to provide more >>> information, including who messages have been sent to and what the >>> responses were. >>> >> Of course at this point we wander into the wonderful world of >> authentication and security. If I direct a message to a specific >> application via the hub, rather than a broadcast message to all >> applications supporting that message, should the hub be permitted to >> pass the existence of that message on to a 3rd party logging >> application? >> > > That is a good point. Maybe the best thing to do there is to arrange > that by default the hub will not publicise potentially private message > content, but that it can be configured to do so (for debugging > purposes etc). If you're being paranoid you shouldn't send > sensitive content in the first place, since there's nothing in > PLASTIC which says that the hub is not allowed to do this, > and precedent in V0.4 for it doing so. There are various > other security issues, some of which I can think of fixes for > (e.g. see sec. 6 of > > http://wiki.eurovotech.org/twiki/bin/view/VOTech/PlasticRemould > > ), but I doubt we're ever going to be in a position to introduce an > ivo://votech.org/pay(String creditCardNum, int pin, float amount) > message. > Certainly, Plastic is pretty insecure at the moment. I think we probably need to think carefully before trying to make it completely secure....its selling point is that it's simple, and we don't want to complicate it with extra baggage unless we have to. If we want an all purpose messaging system then perhaps we should pick one off the shelf. As noted on http://wiki.eurovotech.org/twiki/bin/view/VOTech/PlasticRemould there is a danger that some script kiddie could take control of your hub remotely. Even if we did randomise the RMI port, it would be easy enough to scan for it. So I think the best thing to do is a) make the .plastic file have 400 permissions b) disallow connections from remote machines _unless they have been explicitly enabled_ [there are some potential uses of this, as noted by Pierre]. Point a) should give the user some limited protection against other users on the same machine. I don't have a problem with needing to explicitly enable the logging feature - it's probably only us dev types that are going to want it anyway, and it doesn't complicate the spec. I do wonder whether we need to worry ourselves too much though....is it really likely that GAIA will snoop on messages to Aladin, and rebroadcast them with corrupted arguments to try to steal market share ;-) ! My gut feeling is that I'd prefer to trade security for simplicity and assume that applications on a user's desktop are "friendly". That said, we should make it clear to app writers what we are offering. Can anyone think of something really nasty that an app could do through Plastic that it couldn't do already? John |
|
From: Mark T. <m.b...@br...> - 2006-09-14 18:28:28
|
On Thu, 14 Sep 2006, Alasdair Allan wrote: > > Mark Taylor wrote: > > Longer term, I'd like to see hub implementations supporting a > > message or messages which is/are sent (to anyone who's asked) > > logging all traffic. This would be able to provide more > > information, including who messages have been sent to and what the > > responses were. > > Of course at this point we wander into the wonderful world of > authentication and security. If I direct a message to a specific > application via the hub, rather than a broadcast message to all > applications supporting that message, should the hub be permitted to > pass the existence of that message on to a 3rd party logging > application? That is a good point. Maybe the best thing to do there is to arrange that by default the hub will not publicise potentially private message content, but that it can be configured to do so (for debugging purposes etc). If you're being paranoid you shouldn't send sensitive content in the first place, since there's nothing in PLASTIC which says that the hub is not allowed to do this, and precedent in V0.4 for it doing so. There are various other security issues, some of which I can think of fixes for (e.g. see sec. 6 of http://wiki.eurovotech.org/twiki/bin/view/VOTech/PlasticRemould ), but I doubt we're ever going to be in a position to introduce an ivo://votech.org/pay(String creditCardNum, int pin, float amount) message. Mark -- Mark Taylor Astronomical Programmer Physics, Bristol University, UK m.b...@br... +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/ |
|
From: Alasdair A. <aa...@as...> - 2006-09-14 18:12:05
|
Mark Taylor wrote: > Longer term, I'd like to see hub implementations supporting a > message or messages which is/are sent (to anyone who's asked) > logging all traffic. This would be able to provide more > information, including who messages have been sent to and what the > responses were. Of course at this point we wander into the wonderful world of authentication and security. If I direct a message to a specific application via the hub, rather than a broadcast message to all applications supporting that message, should the hub be permitted to pass the existence of that message on to a 3rd party logging application? Al. |
|
From: Mark T. <m.b...@br...> - 2006-09-14 17:04:49
|
On Thu, 14 Sep 2006, John Taylor wrote: > Hi All, > I've put a small application that you can use to log plastic message > traffic on: > > http://plastic.sourceforge.net/tupperware/PlasticLog/ > > It attaches to a hub like any other application, and will "mimic" any > other attached applications (in that it will claim to support the same > messages). You can also specify extra messages for it to support on > the commandline. good thinking, I hadn't thought of snooping on other registered applications to find out likely message names. I may pinch that. Longer term, I'd like to see hub implementations supporting a message or messages which is/are sent (to anyone who's asked) logging all traffic. This would be able to provide more information, including who messages have been sent to and what the responses were. Mark -- Mark Taylor Astronomical Programmer Physics, Bristol University, UK m.b...@br... +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/ |
|
From: John T. <jd...@ro...> - 2006-09-14 16:40:11
|
Hi All, I've put a small application that you can use to log plastic message traffic on: http://plastic.sourceforge.net/tupperware/PlasticLog/ It attaches to a hub like any other application, and will "mimic" any other attached applications (in that it will claim to support the same messages). You can also specify extra messages for it to support on the commandline. Mark - I know there's a class in plaskit that does similar...but I needed something for Noel that would receive as yet unspecified messages, and since we changed the spec to not support "promiscuous" mode, the plaskit logger would need recompiling to take the new messages. -- ------------------------------------------------------------------------ AstroGrid/VOTech & Institute for Astronomy, Edinburgh Skype:johndavidtaylor <skype:johndavidtaylor?chat> ------------------------------------------------------------------------ |
|
From: Mark T. <m.b...@br...> - 2006-09-14 14:02:13
|
On Thu, 14 Sep 2006, Isa Barbarisi wrote: > Hi, > I also agree with point 4, it seems to be the most flexible solution, > with just a couple of remarks. > It should be clear in the standard that all the information from a SSAP > Servers has to be sent and no information should be lost, in order to > not lose important metadata. > The other point is that we would prefer to have the keys of the map as a > combination of UCD and utype (both of them) so that if one of them is > not found (maybe because the SSAP server doesn't have it) we can make > the search on the other one. Eventually all the fields of the row (like > ucd, utype, name...) could be sent. If they're all going in the same map I don't think that column name should be included as well. It's probably reasonably safe to say that there won't be (confusing) namespace clashes between UCD and utype, but column names could be anything (and are unlikely to be machine-intelligible), so it seems like asking for trouble to use them as keys in the same map. Mark -- Mark Taylor Astronomical Programmer Physics, Bristol University, UK m.b...@br... +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/ |
|
From: Isa B. <Isa...@sc...> - 2006-09-14 13:56:39
|
Hi, I also agree with point 4, it seems to be the most flexible solution, with just a couple of remarks. It should be clear in the standard that all the information from a SSAP Servers has to be sent and no information should be lost, in order to not lose important metadata. The other point is that we would prefer to have the keys of the map as a combination of UCD and utype (both of them) so that if one of them is not found (maybe because the SSAP server doesn't have it) we can make the search on the other one. Eventually all the fields of the row (like ucd, utype, name...) could be sent. In this way there won't be any problem for Spectra Viewers to reach the information needed. Cheers, Isa The current Thomas Boch wrote: >Hi Mark, > > > >>Four possible solutions: >> >> 1. Have an additional parameter which names the identity of the >> standard which defines what the keys in the map are >> (e.g. "SSAP1.0" (which means they'll be utypes as per the >> SSAP v1 document) or "SSAP-pre1.0" (which means they'll be >> UCDs as per the existing practice) or something). >> Probably these IDs ought to be URIs really. >> >> 2. Have one map for UCDs and another one for Utypes. >> >> 3. Prefix UCD keys with "ucd:" and utype keys with "utype:", >> and put them all in the same map. >> >> 4. Just stick all the UCDs and all the utypes, or both, or neither, >> in the same metadata map, and figure that software can probably >> guess which is which when it sees them. >> >>(1) is the most rigorous, and tackles the case of utypes whose semantics >>change between different versions of the standard. (2-4) ignore that >>concern. Am kind of tempted to plump for (4) following the "bit scrappy >>but it'll probably do the trick" PLASTIC philosophy. Anyone else? >> >> > >I am in favour of solution (4). In a perfect world, I'd say (1) is the >best, but I'am afraid very few spectra message senders will do this >effort. >Of course, (4) is not perfect, but in many cases, it will be enough. > >Thomas > >------------------------------------------------------------------------- >Using Tomcat but need to do more? Need to support web services, security? >Get stuff done quickly with pre-integrated technology to make your job easier >Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo >http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 >_______________________________________________ >Plastic-devs mailing list >Pla...@li... >https://lists.sourceforge.net/lists/listinfo/plastic-devs > > |
|
From: Thomas B. <bo...@ne...> - 2006-09-14 13:06:38
|
Hi Mark, > > Four possible solutions: > > 1. Have an additional parameter which names the identity of the > standard which defines what the keys in the map are > (e.g. "SSAP1.0" (which means they'll be utypes as per the > SSAP v1 document) or "SSAP-pre1.0" (which means they'll be > UCDs as per the existing practice) or something). > Probably these IDs ought to be URIs really. > > 2. Have one map for UCDs and another one for Utypes. > > 3. Prefix UCD keys with "ucd:" and utype keys with "utype:", > and put them all in the same map. > > 4. Just stick all the UCDs and all the utypes, or both, or neither, > in the same metadata map, and figure that software can probably > guess which is which when it sees them. > > (1) is the most rigorous, and tackles the case of utypes whose semantics > change between different versions of the standard. (2-4) ignore that > concern. Am kind of tempted to plump for (4) following the "bit scrappy > but it'll probably do the trick" PLASTIC philosophy. Anyone else? I am in favour of solution (4). In a perfect world, I'd say (1) is the best, but I'am afraid very few spectra message senders will do this effort. Of course, (4) is not perfect, but in many cases, it will be enough. Thomas |
|
From: Noel W. <Noe...@ma...> - 2006-09-14 12:12:34
|
On 14 Sep 2006, at 12:52, Mark Taylor wrote: > On Thu, 14 Sep 2006, Noel Winstanley wrote: > >> Just trying to implement this now. >> >> The SSAP responses I see coming back from current services seem to do >> a good job of annotating their columns with UCDs. However, utypes are >> either incomplete or utterly missing. >> >>> the spectrum. The entries of this map take the form of >>> utype->value pairs, where the keys (utypes) are as defined >>> in the SSAP specification, e.g. "Access.Format" for MIME >>> type. As many or few of these entries as are known may be >>> filled in. >> >> would UCD->Value pairs be acceptable to everyone? > > Probably time for me to bring up a complication I left out of my > original proposal (on the grounds that I'd see how it was going down > before I got to the nitty gritty). > > As you say, SSAP as currently practiced seems to be mostly UCD-based. > I'm not sure to what extent this was ever standardised. > The real-soon-now SSAP V1.0 standard, following the real-soon-now > Spectral Data Model V1.0 standard, talks in terms of Utypes > (which is probably the Right Thing to do). No doubt our grandchildren > will invent SSAP v1.1+ not mine. I'm going to warn them off, and suggest they do something less daft instead. like alligator wrestling, or something. > to be based on VoTypes or something. > Whether that's true or not, different versions of the standard will > define more or less grossly different usages of actual utypes/UCDs. > So exactly what version of SSAP we're using determines what's the > right key-value pairs to pass in the metadata map, and what these > mean. > > Four possible solutions: > ... > > (1) is the most rigorous, and tackles the case of utypes whose > semantics > change between different versions of the standard. (2-4) ignore that > concern. Am kind of tempted to plump for (4) following the "bit > scrappy > but it'll probably do the trick" PLASTIC philosophy. Anyone else? > I'd prefer 4) as - I've just implemented it in 5 mins (a good sign). - it requires no knowledge of SSAP specs encoded in the message- sender. - The sender can just pass-through whatever data was received from the SSAP service response. - requiring a client to understand a lot of SSAP, (and repeat for SIAP, SLAP, VOEvent, etc, etc) is going to make everyone's lives harder, and goes against 'do one thing well' - it's more the message receiver's task to understand the particular specs it supports. A further thought.. If a specta tool prefers a _lot_ of metadata, I'd suggest it should also declare that it accepts the ivo://votech.org/ votable/loadFromURL message. Then a client can pass it the queryURL to get the SSAP service response itself, and then maybe follow up with some ivo://votech.org/votable/showObjects messages to indicate which items the user is interested in. I beleive that Aladin does something similar when passed a VOTABLE that is a SIAP service response. cheers noel > Mark > > -- > Mark Taylor Astronomical Programmer Physics, Bristol > University, UK > m.b...@br... +44-117-928-8776 http://www.star.bris.ac.uk/ > ~mbt/ > > > ---------------------------------------------------------------------- > --- > Using Tomcat but need to do more? Need to support web services, > security? > Get stuff done quickly with pre-integrated technology to make your > job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache > Geronimo > http://sel.as-us.falkag.net/sel? > cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Plastic-devs mailing list > Pla...@li... > https://lists.sourceforge.net/lists/listinfo/plastic-devs > |
|
From: Mark T. <m.b...@br...> - 2006-09-14 11:52:10
|
On Thu, 14 Sep 2006, Noel Winstanley wrote:
> Just trying to implement this now.
>
> The SSAP responses I see coming back from current services seem to do
> a good job of annotating their columns with UCDs. However, utypes are
> either incomplete or utterly missing.
>
> > the spectrum. The entries of this map take the form of
> > utype->value pairs, where the keys (utypes) are as defined
> > in the SSAP specification, e.g. "Access.Format" for MIME
> > type. As many or few of these entries as are known may be
> > filled in.
>
> would UCD->Value pairs be acceptable to everyone?
Probably time for me to bring up a complication I left out of my
original proposal (on the grounds that I'd see how it was going down
before I got to the nitty gritty).
As you say, SSAP as currently practiced seems to be mostly UCD-based.
I'm not sure to what extent this was ever standardised.
The real-soon-now SSAP V1.0 standard, following the real-soon-now
Spectral Data Model V1.0 standard, talks in terms of Utypes
(which is probably the Right Thing to do). No doubt our grandchildren
will invent SSAP v1.1+ to be based on VoTypes or something.
Whether that's true or not, different versions of the standard will
define more or less grossly different usages of actual utypes/UCDs.
So exactly what version of SSAP we're using determines what's the
right key-value pairs to pass in the metadata map, and what these mean.
Four possible solutions:
1. Have an additional parameter which names the identity of the
standard which defines what the keys in the map are
(e.g. "SSAP1.0" (which means they'll be utypes as per the
SSAP v1 document) or "SSAP-pre1.0" (which means they'll be
UCDs as per the existing practice) or something).
Probably these IDs ought to be URIs really.
2. Have one map for UCDs and another one for Utypes.
3. Prefix UCD keys with "ucd:" and utype keys with "utype:",
and put them all in the same map.
4. Just stick all the UCDs and all the utypes, or both, or neither,
in the same metadata map, and figure that software can probably
guess which is which when it sees them.
(1) is the most rigorous, and tackles the case of utypes whose semantics
change between different versions of the standard. (2-4) ignore that
concern. Am kind of tempted to plump for (4) following the "bit scrappy
but it'll probably do the trick" PLASTIC philosophy. Anyone else?
Mark
--
Mark Taylor Astronomical Programmer Physics, Bristol University, UK
m.b...@br... +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/
|
|
From: Noel W. <Noe...@ma...> - 2006-09-14 11:31:16
|
Just trying to implement this now. The SSAP responses I see coming back from current services seem to do a good job of annotating their columns with UCDs. However, utypes are either incomplete or utterly missing. > the spectrum. The entries of this map take the form of > utype->value pairs, where the keys (utypes) are as defined > in the SSAP specification, e.g. "Access.Format" for MIME > type. As many or few of these entries as are known may be > filled in. would UCD->Value pairs be acceptable to everyone? cheers noel. On 13 Sep 2006, at 12:23, Mark Taylor wrote: > On Tue, 12 Sep 2006, Isa Barbarisi wrote: > >> Dear Mark, >> >> I agree with the approach about defining additional arguments in the >> message related more to the type of object (spectrum) than to the >> format of this object (votable, fits..). In fact, this is what we >> tried >> to do when we defined the first SSAP definition. >> During the process of implementation of VOSpec, we found that only >> a few >> parameters are needed to manipulate most of the spectra formats. Of >> course, not all the possible spectra serializations can be used in >> VOSpec, but that collection of parameters allows our application to >> display thousand of spectra in different formats from ~15 different >> projects. >> >> List of parameters: C: compulsory - O: Optional >> --------------------------------------------------------------------- >> --------------------------------------------------------------------- >> - >> >> Title - <title> - C - e.g. "ISO SWS01 Spectrum Target: M31_BULGE" >> URL - <url> - C - e.g. >> "http://pma.i....../jsp/product.jsp?obsno=40001501" >> ServerTitle - <server_title> - O - e.g. "ISO Data Archive" >> WaveLengthColumnName - <wave_column_name> - C - e.g. SWAAWAVE >> FluxColumnName - <flux_column_name> - C - e.g. SWAAFLUX >> WaveUnitsScaleq - <wave_units_scaleq> - C - e.g. 1.E-6 >> WaveUnitsDimeq - <wave_units_dimeq> - C - e.g. L >> FluxUnitsScaleq - <flux_units_scaleq> - C - e.g. 1.E-23 >> FluxUnitsDimeq - <flux_units_dimeq> - C - e.g. MT-2 >> Ra - <ra> - O - e.g 10.691809995 >> Dec - <dec> - O - e.g 41.27003 >> Format - <format> - C - e.g spectrum/fits >> --------------------------------------------------------------------- >> --------------------------------------------------------------------- >> --- >> >> >> Our proposal is to have (at least) all the compulsory fields in the >> message, so VOSpec (as message consumer) and e.g. Aladin and >> Astroscope >> (as message senders) could be immediately updated to load spectra >> using it. >> Note that all the information requested is already present in the >> existing SSAP services, and it is going to be included (perhaps with >> some minimal syntax changes) in the next SSAP specification. > > Isa, > > thanks for your message. I hadn't given much thought to these > additional > metadata items. I suppose in most cases they can be obtained from > the spectrum itself after retrieval, but in some cases the information > is available before retrieval (and useful) because the relevant fields > are filled in for each row of the SSAP response table. > > So I've revised my proposal, as follows: > > boolean ivo://votech.org/spectrum/loadFromURL( String url, > String id, > Map meta ) > > Loads a spectrum or SED. id is an identifier as for the VOTable > load message - use the URL if there's no reason to do otherwise. > The meta argument is a map/struct containing additional > information about the type of data found at url which encodes > the spectrum. The entries of this map take the form of > utype->value pairs, where the keys (utypes) are as defined > in the SSAP specification, e.g. "Access.Format" for MIME > type. As many or few of these entries as are known may be > filled in. > > Discussions on the SSAP standard including the most recent > draft can be found at http://www.ivoa.net/twiki/bin/view/IVOA/ > SsaInterface > (see sec 7.5 of the SSA query interface V0.95 doc). > > Sending this message should be easy to implement by an SSAP client > (e.g. AstroScope) which wants to pass the spectrum selection on to > viewers, since it can just scoop up the fields from the relevant > row of the SSAP response VOTable and pass them on in a Map without > having to know or care what they mean. > > VO-friendly spectrum viewers such as VOSpec should be well-equipped > to receive and act on such messages, since in order to understand > SSAP queries which they already do, they must know how to make > sense of fields keyed by utypes in this way. The various optional > and compulsory parameters that you describe above can be included, > as well as any others that may be available. > > And an added bonus is that decisions about what metadata items > are/can be/should be passed by the message is taken care of by the > DAL/DM groups whose job it is to worry about such things, rather > than us plastickers who (well, speaking for myself) may not > understand spectra so well. > > This message can of course be used to pass spectra around which > do not result from actual SSAP queries, since you can add metadata > map entries by hand as if they'd come from SSAP (or just leave the > metadata map empty). > > Opinions? > > Mark > > -- > Mark Taylor Astronomical Programmer Physics, Bristol > University, UK > m.b...@br... +44-117-928-8776 http://www.star.bris.ac.uk/ > ~mbt/ > > > ---------------------------------------------------------------------- > --- > Using Tomcat but need to do more? Need to support web services, > security? > Get stuff done quickly with pre-integrated technology to make your > job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache > Geronimo > http://sel.as-us.falkag.net/sel? > cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Plastic-devs mailing list > Pla...@li... > https://lists.sourceforge.net/lists/listinfo/plastic-devs > |
|
From: John T. <jd...@ro...> - 2006-09-14 08:35:53
|
Interesting - we seem to have made the top 15: http://freeembroider.info/plastic/19089.html Bizarre. John -- ------------------------------------------------------------------------ AstroGrid/VOTech & Institute for Astronomy, Edinburgh Skype:johndavidtaylor <skype:johndavidtaylor?chat> ------------------------------------------------------------------------ |
|
From: John T. <jd...@ro...> - 2006-09-14 08:28:06
|
Thinking about it further, if this is true for VOSpec and all the rest, then passing the serialisation format as an arg rather than as part of the message may well be more user-friendly. If an app doesn't understand the format it can throw a fault saying precisely what the problem is, rather than the user wondering why AstroScope won't allow him to send the file to his favourite viewer. I think once this spectrum message is sorted we should continue our discussion about exception handling that we started at the Strasbourg meeting. John Mark Taylor wrote: > On Wed, 13 Sep 2006, John Taylor wrote: > > >> Could someone with knowledge of spectra viewers confirm that "most >> commonly used spectra viewers will support all commonly used formats"? >> > > that's more or less true of SPLAT (well, "most" rather than "all" > commonly used formats). > > -- ------------------------------------------------------------------------ AstroGrid/VOTech & Institute for Astronomy, Edinburgh Skype:johndavidtaylor <skype:johndavidtaylor?chat> ------------------------------------------------------------------------ |
|
From: Mark T. <m.b...@br...> - 2006-09-14 08:20:24
|
On Wed, 13 Sep 2006, John Taylor wrote: > Could someone with knowledge of spectra viewers confirm that "most > commonly used spectra viewers will support all commonly used formats"? that's more or less true of SPLAT (well, "most" rather than "all" commonly used formats). -- Mark Taylor Astronomical Programmer Physics, Bristol University, UK m.b...@br... +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/ |
|
From: John T. <jd...@ro...> - 2006-09-13 16:07:44
|
Personally I'm not too worried about the "purity" of PLASTIC and the fact that we're using messages to indicate file format for tables, but to indicate content for spectra....if people outside the current clique start defining their own messages (which I would see as a good thing) we'll end up with a a mixed approach anyway. I'd like to make a few comments on the "user experience". We made the "load table" message explicitly specify VOTable in the message string as it seems reasonable to mandate this as the standard table interchange format. That way, each tool need only support a single format in addition to what it supports natively. I have found this to be a little restrictive when you have FITS files in MySpace that you know to be tables but cannot send directly to Topcat. However, when the new VOSpaces are ready with their ability to export data in different formats this won't be an issue. The "load table" message doesn't specify the semantic content since tables are so general. So far, I haven't encountered any usability issues here...though you can imagine a tool that requires RA&DEC failing if it's supplied with any-old-VOTable. For spectra, having the message string reflect the fact that a spectrum is being sent seems to be the right approach since spectra are more specialised animals. However, if we are going to pass in the data format as an argument we should check that most spectrum consumers will support most spectrum formats...otherwise the user is going to be frequently frustrated by "format not understood" exceptions. Are we confident that this is the case? If this is not the case, then we could still consider making the data format part of the message. The difference for the user would simply be that he won't be allowed to send the spectrum to an app that won't understand it. I'm not too worried about the proliferation of messages - provided we used a sensible naming strategy such as ivo://votech.org/[data format]/spectrum/loadFromURL then it wouldn't be too difficult to manage (and strings are really cheap). It could get a little unwieldy if a message included two or more spectrum arguments, but as our manifesto ways "do what works 90% of the time". (or if it doesn't say that it should do). I'm not necessarily advocating this approach - it means a little more programming work than that proposed by Mark - but I just want to say that we shouldn't rule it out because we're scared of inventing messages, and that picking one approach this time doesn't commit us to do it next time. Could someone with knowledge of spectra viewers confirm that "most commonly used spectra viewers will support all commonly used formats"? John Mark Taylor wrote: > Plastic Pals, > > I have documented a new proposed message for loading spectra at > > http://eurovotech.org/twiki/bin/view/VOTech/PlasticMessagesProposal > > I had discussions with various people at Strasbourg about this; > it is unlike the votable/loadFromURL and fits/image/loadFromURL > messages in that it relates to the semantic content (it's a spectrum) > rather than the data format (it's a FITS file) of the resource > at the URL. The reason for this is that there is a very wide range > of possible serializations of a spectrum/SED (FITS 1-d array, FITS > binary or ascii table, VOTable, FITS or VOTable serialisations of > DM spectral data model as described at > http://www.ivoa.net/internal/IVOA/IvoaDataModel/spec98c.pdf, ...), > trying to cope with all possibilities could lead to an explosion > in the number of messages. > > There are arguments for and against doing it this way, but on pragmatic > grounds I think this is the most workable solution. If you disagree, > feel free to continue the debate here. > > Mark > > -- ------------------------------------------------------------------------ AstroGrid/VOTech & Institute for Astronomy, Edinburgh Skype:johndavidtaylor <skype:johndavidtaylor?chat> ------------------------------------------------------------------------ |
|
From: Noel W. <Noe...@ma...> - 2006-09-13 14:45:48
|
On 13 Sep 2006, at 12:42, Thomas Boch wrote: > Hi Mark, > > >> So I've revised my proposal, as follows: >> >> boolean ivo://votech.org/spectrum/loadFromURL( String url, >> String id, >> Map meta ) >> >> Loads a spectrum or SED. id is an identifier as for the VOTable >> load message - use the URL if there's no reason to do >> otherwise. >> The meta argument is a map/struct containing additional >> information about the type of data found at url which encodes >> the spectrum. The entries of this map take the form of >> utype->value pairs, where the keys (utypes) are as defined >> in the SSAP specification, e.g. "Access.Format" for MIME >> type. As many or few of these entries as are known may be >> filled in. >> >> (...) >> >> Sending this message should be easy to implement by an SSAP client >> (e.g. AstroScope) which wants to pass the spectrum selection on to >> viewers, since it can just scoop up the fields from the relevant >> row of the SSAP response VOTable and pass them on in a Map without >> having to know or care what they mean. > > I agree, this sounds sensible and pragmatic to me. agreed. > >> This message can of course be used to pass spectra around which >> do not result from actual SSAP queries, since you can add metadata >> map entries by hand as if they'd come from SSAP (or just leave the >> metadata map empty). > > Indeed, it is important to leave the door open for spectra *not* > coming > from an SSAP service (eg. local files, MySpace account). For those > spectra, we certainly won't have all the metadata Isa has > described, and > it will be up to the spectrum tool to try to obtain needed metadata > from > the spectrum itself. > Thus, I think we should precise that tools sending a > ivo://votech.org/spectrum/loadFromURL message should give as many > metadata information as they can, but that no parameter is really > 'compulsory'. makes perfect sense to me. If there's no objections, I'll make a start on implementing this in astroscope - hope to have it done for Moscow. > > Cheers, > > Thomas > > ---------------------------------------------------------------------- > --- > Using Tomcat but need to do more? Need to support web services, > security? > Get stuff done quickly with pre-integrated technology to make your > job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache > Geronimo > http://sel.as-us.falkag.net/sel? > cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Plastic-devs mailing list > Pla...@li... > https://lists.sourceforge.net/lists/listinfo/plastic-devs > |