plastic-devs — Plastic Developers

Re: [Plastic-devs] Select or show colums message

[Mark T. <m.b...@br...>]

On Thu, 20 Jul 2006, John Taylor wrote:

> Some things I haven't worked out yet:
> * if the fragment is going to appear in a UI, then it needs to be 
> human-friendly - so we should probably use the usual URI encodings for 
> whitespace etc http://www.w3.org/Addressing/URL/uri-spec.html ??
> * if the fragment is going to appear in a UI, then any 
> internationalisation needs to be done by the recipient changing the 
> fragments it offers according to locale

If we're processing URIs then yes we have to process %-escaped 
sequences as defined by the relevant standards.  However, especially
if we're thinking about internationalisation, I think I'd rather
see fragment IDs which are minimally human-readable 
("select_columns" etc) with some (optionally supported) messages
which translate these for presentation to users, e.g.

   String ivo://votech.org/info/message/getName( String msgIdIncludingFrag )
   String ivo://votech.org/info/message/getDescription( String msgIdIncludingFrag, String locale )
   String ivo://votech.org/info/message/writeMeAnEssayInLatinOnThisTopicPlease( String msgIdIncludingFrag );

etc.  In the case that these messages are not supported, the sending
application will just have to present the user with "select_cols".


> * if you've registered as understanding certain fragments, should the 
> hub pass you messages URIs that don't contain any fragment, or contain a 
> fragment you don't understand?  I'd say yes for the former and not sure 
> for the latter.  It might be safest just to say yes in both cases and 
> leave it up to individual apps to decide what to do.

think I agree, yes & yes.

-- 
Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
m.b...@br... +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/

Re: [Plastic-devs] Select or show colums message

[Alasdair A. <aa...@as...>]

Mark Taylor wrote:
> John Taylor wrote:
>> Comments?
>
> I am in full agreement with your proposal.

Ditto. Good thinking...

Al.

Re: [Plastic-devs] Select or show colums message

[R H. <ric...@co...>]

I like it too ...

On Thu, 20 Jul 2006, Mark Taylor wrote:

> On Thu, 20 Jul 2006, John Taylor wrote:
>
>> Thinking again about this, I have an idea that might allow us to combine
>> the increased "pluggability" of having a generic message with the
>> control implied by a more specific one and thus keep both you and
>> Richard happy.
>>

... but this would be too much :) , since we still have the problem of 
sending classifications. I propose an extra:

//pass classes all at once
selectCols( id, { cols[], class_labels[] } )

or:

  //pass classes one at time
selectCols#new_subset( id, { cols[], class_label } )

so each observation could have a classification to be represented, for 
example, by different groups/colours in topcat. The alternative (from my 
point of view) is for Eirik to write another table with an extra column 
and pass that. This has several ramifications, not least being the 
generation of mutiple temporary files and duplication of effort (Eirik 
can't write tables at present, and I'm not really sure it should).

Without such a well-defined class label, it's going to be difficult to 
represent multiple clusters and subsets successfully, something Eirik 
should (coming soon ;) be doing a lot of.

Richard

Re: [Plastic-devs] Select or show colums message

[Mark T. <m.b...@br...>]

On Thu, 20 Jul 2006, John Taylor wrote:

> Thinking again about this, I have an idea that might allow us to combine 
> the increased "pluggability" of having a generic message with the 
> control implied by a more specific one and thus keep both you and 
> Richard happy.
> 
> If we think of a generic selectCols(id, cols[]) message as just implying 
> "here are a bunch of columns, do with them what you will", then we need 
> a way of the recipient stating what it is able to do with the message, 
> and the sender choosing amongst the options.   We could do this by 
> decorating the message with the extra information.   URI fragments are a 
> natural way to do this, since (IIRC) the fragment is intended to be 
> interpreted by the client (the recipient of the message in this case).

  ...

> Comments?

John, unless there's some subtle gotcha that hasn't occurred to me yet, 
I call that brilliant thinking.  It's the message receiver, not the
sender which is able to give specifics on what the effect of a 
generic message will be, and trying to tackle it the other way
round by defining a load of disjoint messages which cover all the
things that a client might want to do with data is doomed to 
not working very well.  Top marks for spotting this and working out
what to do about it.  I am in full agreement with your proposal.

Mark

-- 
Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
m.b...@br... +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/

Re: [Plastic-devs] Select or show colums message

[John T. <jd...@ro...>]

Thinking again about this, I have an idea that might allow us to combine 
the increased "pluggability" of having a generic message with the 
control implied by a more specific one and thus keep both you and 
Richard happy.

If we think of a generic selectCols(id, cols[]) message as just implying 
"here are a bunch of columns, do with them what you will", then we need 
a way of the recipient stating what it is able to do with the message, 
and the sender choosing amongst the options.   We could do this by 
decorating the message with the extra information.   URI fragments are a 
natural way to do this, since (IIRC) the fragment is intended to be 
interpreted by the client (the recipient of the message in this case).

To take a specific example.  Topcat could do one of two things with a 
bunch of columns: create a new subset based on them, or create a plot 
(2D or 3D depending on the number of columns).
It would register as understanding
ivo://votech.org/votable/selectCols#plot
ivo://votech.org/votable/selectCols#new_subset

The fragment string is completely defined by Topcat - no other agreement 
is needed with any other application, and no other application needs to 
understand what it means.  Instead, the sending application (e.g. Weka) 
strips off the fragments and uses them to populate its UI.  Thus it 
might offer the user the following options for the selected columns:
send cols -> topcat -> plot
                               -> new_subset
               -> tabview-> highlight
               -> Anomaly Detector

[assuming tabview has registered with selectCols#highlight, and Anomaly 
Detector has registered just with selectCols]
The appropriate fragment is then tacked on to any sent messages.

If Topcat should receive a message without a fragment then it could a) 
ignore it (not so good for backwards compatibility) b) do some default 
behaviour c) prompt the user

The nice thing about this is that behaviour is extendible without 
further agreement amongst application vendors.  Should astronomers ask 
Mark to offer the ability to differentiate between overwriting an 
existing plot, or creating a new one, it's easy to do: simply register with
ivo://votech.org/votable/selectCols#create_plot
ivo://votech.org/votable/selectCols#overwrite_plot
ivo://votech.org/votable/selectCols#new_subset

and all compliant apps pick up the new option automatically.

This is _almost_ implementable without breaking any existing apps.  
Since there's no selectCols message yet, there's no reason why we can't 
define
selectCols#plot and selectCols#new_subset as distinct messages under the 
current version of Plastic.  Changing the hub to route messages while 
ignoring the fragment is, of course, a change to the spec, but I don't 
think it breaks anything since no currently defined messages use the "#" 
symbol.

Comments?

John








B Walshe wrote:
> Hi,
>
> Bob, John and myself were discussing the showCols message this morning 
> and we decided that it would make sense to have a general message 
> message for selecting columns as well as more specific messages such as 
> for plotting graphs based on those columns.
>
> Personally I see the general purpose message as being passive; i.e. it 
> can be used to tell other applications which columns the user is 
> currently interested in, but it doesn't say what to do with them. This 
> can be useful when dealing with tables with large numbers of columns 
> where there is insufficient space on the screen to display their names 
> properly and/or they are in a different order.
>
> Plot messages, on the other hand, actually tell the application to do 
> something with the data. While it would be easy enough to send the 
> showCols message and then have the user manually click on a "visualise" 
> button in the other application, what happens if the user does not want 
> to change which columns they had selected just to do a quick plot? On 
> the flip side: for applications that take a lot of time and resources 
> to generate a graph it would make sense that they only do so when they 
> receive messages that specifically ask for a plot.
>
>
> Sound,
> Brian
>
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys -- and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Plastic-devs mailing list
> Pla...@li...
> https://lists.sourceforge.net/lists/listinfo/plastic-devs
>
>   

-- 
------------------------------------------------------------------------
AstroGrid/VOTech
&
Institute for Astronomy, Edinburgh
Skype:johndavidtaylor <skype:johndavidtaylor?chat>

------------------------------------------------------------------------

Re: [Plastic-devs] Select or show colums message

[B W. <s05...@sm...>]

Hi,

Bob, John and myself were discussing the showCols message this morning 
and we decided that it would make sense to have a general message 
message for selecting columns as well as more specific messages such as 
for plotting graphs based on those columns.

Personally I see the general purpose message as being passive; i.e. it 
can be used to tell other applications which columns the user is 
currently interested in, but it doesn't say what to do with them. This 
can be useful when dealing with tables with large numbers of columns 
where there is insufficient space on the screen to display their names 
properly and/or they are in a different order.

Plot messages, on the other hand, actually tell the application to do 
something with the data. While it would be easy enough to send the 
showCols message and then have the user manually click on a "visualise" 
button in the other application, what happens if the user does not want 
to change which columns they had selected just to do a quick plot? On 
the flip side: for applications that take a lot of time and resources 
to generate a graph it would make sense that they only do so when they 
receive messages that specifically ask for a plot.


Sound,
Brian

Re: [Plastic-devs] Future API changes

[Mark T. <m.b...@br...>]

On Wed, 19 Jul 2006, John Taylor wrote:

> > Thinking about it, with this in place you can achieve anything using
> > the asynchronous hub request methods that you could before using 
> > the synchronous ones (though with more client-side work).  
> > The possibility therefore arises of ditching the synchronous 
> > hub request methods.  Is that a bit too radical?
> >   
> We'd then lose the ability to get data back to non-callable environments 
> such as command-line python.

yeah you're right.  Bad idea then.

-- 
Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
m.b...@br... +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/

Re: [Plastic-devs] Future API changes

[John T. <jon...@gm...>]

>> Mark Taylor wrote:
>>     
>>> One possibility is to replace hub methods getName(), getRegisteredIds(),
>>> getUnderstoodMessages() and getMessageRegisteredHubIdsPleaseIfYouDontMind()
>>> with a single getRegisteredApplications() which returns an 
>>> <array>-of-<struct>s/List-of-Maps giving all the information that
>>> the hub knows from registration time about every registered application
>>> (name and supported message IDs).  I'm not saying it's a good idea mind.
>>>
>>>       
>> Actually I quite like this.  Instead of registering with your name, you 
>> register with a struct containing your name, ivorn, iconurl, description 
>> + whatever else we decide to put in there in the future.  We'll define a 
>> few well-known keys to get started, but people can expand the struct as 
>> they see fit.  And you have a single method to return the lot, as you 
>> just described. 
>>     
>
> I hadn't thought of doing it for registration as well, but that probably
> makes sense if we're going to do that.  Downsides:
>
>   1.  Different from current practice (but looks like we may be heading
>       for a backwardly-incompatible change to the protocol sometime
>       soon so may as well make the most of it).
>   
I don't know about soon, but it would certainly be good to think of as 
many such changes as we can and get them done in one go.
>   2.  One call fits all may mean you're transmitting more data than you
>       need to - however we're probably not talking about very large
>       amounts so this is probably not important.
>
>   3.  Means you can neither register not obtain any application metadata
>       without the ability to process structs.  This is probably going to 
>       scupper any attempt to use HTTP-GET for PLASTIC access.  There
>       may be other platforms too that don't lend themselves to an
>       XML-RPC implementation which can decode structs (i.e. don't
>       have langauge features resembling associative arrays/hashes) - 
>       can anyone think of examples of this which might be important (IDL?)
>   
This could be an issue, though even with our current spec, http-get is 
very limited and is pretty much limited to "load-this-file" type 
messages so perhaps it's not much of a loss.
We should check up on the progress (if any) of the IDL xml-rpc library.  
The other environments I'd like to have plastic-access from are "R" and 
web pages.  I think that having a very small applet to do the comms is 
probably the way to go for web access.  As for R, I currently have an 
http-get solution, but it's not particularly good.

[]
>
>   
>>> Now (v0.5) that you can't register to receive all messages by sending 
>>> an empty supportedMessages array at registration time, we're
>>> missing a way for monitor-type applications to see everything that's
>>> going on in the hub (good for debugging).  This could be addressed
>>> by a new message
>>>
>>>    id:
>>>        ivo://votech.org/hub/event/requestSent
>>>    args:
>>>        String senderId
>>>        String messageId
>>>        List recipientIds
>>>        String requestId
>>>    return type:
>>>        void
>>>
>>> which the hub sends to interested parties whenenever it makes a 
>>> request to listeners (er, with the possible exception of requestSent or 
>>> responseRecieved requests...).  The requestId argument is just a 
>>> string unique to this request.
>>>
>>> We also introduce a second message
>>>
>>>    id:
>>>        ivo://votech.org/hub/event/responseReceived
>>>    args:
>>>        String requestId
>>>        String recipientId
>>>        Object response
>>>    return type:
>>>        void
>>>
>>> Here the requestId is used to match the response up with the
>>> earlier-notified request, for those that care.
>>>   
>>>       
>> I like it.  Let's put the senderId in there as well, just so that the 
>> client doesn't _have_ to match up the two messages to find out who the 
>> response is going to.
>>
>>     
>>> Monitor-type applications can now see all the requests that go out from
>>> and all the responses that come back to the hub, which is good.
>>>
>>> A bonus is that this pair of messages can also be used by applications
>>> which want to use callbacks to get responses from the hub; either
>>> because they're not multithreaded or because response times are
>>> potentially slow.  Instead of calling request() synchronously,
>>> they call requestAsynch() and make sure they're registered to 
>>> receive requestReceived messages (ignoring ones they didn't send
>>> in the first place if necessary).  None of this requires any additional
>>> hub methods.
>>>   
>>>       
>> This is promising, but I have a couple of comments:
>> 1) We could reduce the burden on the client of sifting through all the 
>> responses by having a similar
>>
>> ivo://votech.org/hub/event/myResponseReceived
>>
>> message that only gets sent to the sender of the original message.
>>     
>
> OK.
>
>   
>> 2) It could be a little tricky matching up responses to messages.  If 
>> you're monitoring the requestReceived message you can tie them together 
>> using your proposed requestId, but it's not obvious to me how you tie 
>> together (with certainty) your original call the the hub with the 
>> response message.  Maybe it doesn't matter.  
>>     
>
> Hadn't thought of that.  However, I can think of a good solution:
> get the hub requestAsynch() methods to return the responseId for the
> request rather than void.
>   
Yes, that sounds workable.
> Thinking about it, with this in place you can achieve anything using
> the asynchronous hub request methods that you could before using 
> the synchronous ones (though with more client-side work).  
> The possibility therefore arises of ditching the synchronous 
> hub request methods.  Is that a bit too radical?
>   
We'd then lose the ability to get data back to non-callable environments 
such as command-line python.

John

Re: [Plastic-devs] Select or show colums message [was: Re: introducing Brian Walshe

[R H. <ric...@co...>]

For my two pence, I'd say the less specific the better, since there are=20
many ways to present 2D/3D/more plots and no doubt more will be added to=20
the various tools in time. In any case, doesn't sending a showCols messag=
e=20
with two columns give sufficient a hint, if Topcat/whoever wants to take=20
it?

Similarly, showObjects is really one message being interpreted in slightl=
y=20
different ways: it creates some kind of logical division which could aris=
e=20
from any number of manual tweaking methods or computational algorithms (e=
g=20
clustering)  and be exhibited in many different forms.

On the other hand, having some way to keep classifications (as, say,=20
clusters) could be important. In R-speak, we would talk about 'factors':=20
each observation has a value in the factor column which gives the class,=20
so for 10 objects with 3 clusters we might have the subsidiary array

{0,0,0,1,1,2,2,2,2,2}

[Age-group and sex are classic factors of course.]

2p spent I reckon :)

Richard



On Tue, 18 Jul 2006, John Taylor wrote:

> (Copied to the list...hope no-one in the original discussion minds.)=A0=
 What follows is a discussion
> about using plastic messages to select or show columns in a table.
>=20
>=20
> Mark Taylor wrote:
>
>  On Tue, 20 Jun 2006, Bob Mann wrote:
>
>=20
>
>  1. Selecting columns (rather than rows).
>=20
>
>  So should the message contain some hint about what to do
> with the named columns?  Should there be one message
>
>   showCols( int[] colIds );
>=20
> or several
>
>   showCols( colId );
>   showCols2( colId1, colId2 );
>   showCols3( colId1, colId2, colId3 );
>=20
> ?  etc.  Open questions.
>=20
>
>  Good question. I can see two different use cases here, which might
> call for different messages. In the first Weka/Eirik act as an initial=20
> filter of a dataset before one starts exploring it with, say, TOPCAT. I=
n
> that case, one might select, say, 10 columns out of a 50-column table a=
nd
> then, as you said, the appropriate response from TOPCAT might be to hid=
e
> the remaining 40 columns, on the assumption that the user is only going
> to work with the selected 10. In the second use case, TOPCAT is just be=
ing
> used as a 3D display tool, and Eirik might fire a series of messages=20
> saying "make a 3D scatter plot of these three columns" and want TOPCAT
> to do just that. Maybe one would want the second use case to be=20
> implemented with a plotCols message, rather than the more general showC=
ols
> one which would work for the first one...but I don't know how that woul=
d
> square with the current philosophy over PLASTIC messaging...
>=20
>
>  that sounds about right, and probably more sensible than my initial=20
> thought above.
>=20
>=20
> I think this could spark an interesting debate about how generic plasti=
c messages should be.=A0 To take
> this example, the two extreme positions are to have a completely generi=
c
> votech.org/votable/showCols(id, cols[]) message leaving it completely u=
p to the recipient to decide
> what its being asked to do, and to have an application-specific message=
 such as
> starlink/topcat/create2DPlot(id, col1, col2).
> We get "better" interop (in the sense of more apps can play together) w=
ith general messages at the cost
> of clarity for the end-user and (maybe) reliability.=A0 From the user's=
 point of view, the difference
> would be that with general messages they might have to configure the re=
ceiving application to tell it
> what to do (create a new subset, open a new plot, overwrite the existin=
g plot...) whereas with a
> specific message they'd make the choice in the sending application (rig=
ht clicking might show two
> options: create new subset/open new plot).
> A middle ground that might be worth exploring is having a generic messa=
ge, but with an optional hint
> argument to the recipient.
>=20
> My gut feeling is that for the user, making a plot of the data in some =
columns is a different concept
> to creating a new subset and so they should probably be different messa=
ges even though they share the
> same argument list.=A0 Here's how Bob's suggestion (slightly modified) =
could work in practice:
>=20
> Recipient
> plotCols(id, cols[])
> selectCols(id, cols[])
> Topcat
> Opens a new 2D or 3D cartesian plot depending on the number of args.=A0=
 Cols>3 are ignored.
> Creates a new subset consisting only of those cols.=A0 Returns a new id=
?
> VisIVO
> Opens a new 3D plot, using the cols>3 as indices into the colourmap, gl=
yph sizes etc....
> ?
> TabView
> Highlights the columns
> Creates a new subset consisting only of those cols.
> xmdv-lite
> On the parallel axes plot, highlights the chosen columns.=A0 On the sca=
tter plots, could highlight or
> show only the relevant scatter plots
> New plots only of selected cols
> Anomaly Detector
>=20
> Runs the EM-algorithm only on the selected columns
> Aladin
> Maybe opens VOPlot?
> create a new pane with the reduced columns?
>=20
> This separation of "showing" and "selecting a subset" makes me think th=
at we've confounded the two with
> our showObjects message.=A0 I wonder if we ought to split it so that we=
 can distinguish between creating
> a new row subset (a la Topcat) and simply highlighting the selected row=
s (a la Aladin).=A0 Indeed,
> perhaps the two "create subset" messages could be merged into one.
>=20
>
>  2. Identifying several sets of rows.
>=20
>
>  Actually you can sort of do this in TOPCAT as it stands.  Each receive=
d
> ivo://votech.org/votable/showObjects message results in the creation
> of a new row subset (old ones are not discarded (even if you want them
> to be, which admittedly can be an annoyance)).  Within TOPCAT you
> can then select which subsets are plotted etc.
>=20
>
>  OK - that is very useful. Do you give the row subset a name in
> the message in which you define it?
>=20
>
>  No, TOPCAT assigns names automatically - currently e.g. weka-1, weka-2=
,
> weka-3, ....  May or may not be ideal for your purposes.
>=20
> Mark
>
>=20
>=20
>=20
>=20
> --
>=20
> _______________________________________________________________________=
_______________________________________
>                                                AstroGrid/VOTech
>                                                       &
>                                       Institute for Astronomy, Edinburg=
h
>                                             Skype:johndavidtaylor
>=20
>=20
> _______________________________________________________________________=
_______________________________________
>=20
>

Re: [Plastic-devs] Select or show colums message [was: Re: introducing Brian Walshe

[Marco C. <mc...@oa...>]

John Taylor wrote:
> (Copied to the list...hope no-one in the original discussion minds.)  
> What follows is a discussion about using plastic messages to select or 
> show columns in a table.
> 
> 
> Mark Taylor wrote:
> 

[...]

> VisIVO
> 	Opens a new 3D plot, using the cols>3 as indices into the colourmap, 
> glyph sizes etc....
> 	?

Dear all,

just to let you now that we changed the way we deal with data and their
visualization...The next release will not create a 3d object just after
loading the dataset, it will create a "storage object" from which the
user can create many visualization of the data. With the current
implementation of PLASTIC, a loadFromURL message will generate a
"storage", a showObject message will highlight the interested rows in
all the currently open views and the user can then create new storages
from the selection.

Cheers,
Marco

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

[Plastic-devs] Select or show colums message [was: Re: introducing Brian Walshe

[John T. <jd...@ro...>]

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <DEFANGED_meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
(Copied to the list...hope no-one in the original discussion minds.)&nbsp;
What follows is a discussion
about using plastic messages to select or show columns in a table.<br>
<br>
<br>
Mark Taylor wrote:
<blockquote
 cite="mid...@an..."
 type="cite">
  <pre wrap="">On Tue, 20 Jun 2006, Bob Mann wrote:

  </pre>
  <blockquote type="cite">
    <blockquote type="cite">
      <blockquote type="cite">
        <pre wrap=""> 	1. Selecting columns (rather than rows).
        </pre>
      </blockquote>
      <pre wrap="">So should the message contain some hint about what to do
with the named columns?  Should there be one message

  showCols( int[] colIds );

or several

  showCols( colId );
  showCols2( colId1, colId2 );
  showCols3( colId1, colId2, colId3 );

?  etc.  Open questions.
      </pre>
    </blockquote>
    <pre wrap=""> 	Good question. I can see two different use cases here, which might
call for different messages. In the first Weka/Eirik act as an initial 
filter of a dataset before one starts exploring it with, say, TOPCAT. In
that case, one might select, say, 10 columns out of a 50-column table and
then, as you said, the appropriate response from TOPCAT might be to hide
the remaining 40 columns, on the assumption that the user is only going
to work with the selected 10. In the second use case, TOPCAT is just being
used as a 3D display tool, and Eirik might fire a series of messages 
saying "make a 3D scatter plot of these three columns" and want TOPCAT
to do just that. Maybe one would want the second use case to be 
implemented with a plotCols message, rather than the more general showCols
one which would work for the first one...but I don't know how that would
square with the current philosophy over PLASTIC messaging...
    </pre>
  </blockquote>
  <pre wrap=""><!---->
that sounds about right, and probably more sensible than my initial 
thought above.
  </pre>
</blockquote>
I think this could spark an interesting debate about how generic
plastic messages should be.&nbsp; To take this example, the two extreme
positions are to have a completely generic
votech.org/votable/showCols(id, cols[]) message leaving it completely
up to the recipient to decide what its being asked to do, and to have
an application-specific message such as<br>
starlink/topcat/create2DPlot(id, col1, col2).<br>
We get "better" interop (in the sense of more apps can play together)
with general messages at the cost of clarity for the end-user and
(maybe) reliability.&nbsp; From the user's point of view, the difference
would be that with general messages they might have to configure the
receiving application to tell it what to do (create a new subset, open
a new plot, overwrite the existing plot...) whereas with a specific
message they'd make the choice in the sending application (right
clicking might show two options: create new subset/open new plot).<br>
A middle ground that might be worth exploring is having a generic
message, but with an optional hint argument to the recipient.<br>
<br>
My gut feeling is that for the user, making a plot of the data in some
columns is a different concept to creating a new subset and so they
should probably be different messages even though they share the same
argument list.&nbsp; Here's how Bob's suggestion (slightly modified) could
work in practice:<br>
<br>
<table border="1" cellpadding="2" cellspacing="2" width="100%">
  <tbody>
    <tr>
      <td valign="top"><b>Recipient</b><br>
      </td>
      <td valign="top"><b>plotCols(id, cols[])</b><br>
      </td>
      <td valign="top"><b>selectCols(id, cols[])</b><br>
      </td>
    </tr>
    <tr>
      <td valign="top">Topcat<br>
      </td>
      <td valign="top">Opens a new 2D or 3D cartesian plot depending on
the number of args.&nbsp; Cols&gt;3 are ignored.<br>
      </td>
      <td valign="top">Creates a new subset consisting only of those
cols.&nbsp; Returns a new id?<br>
      </td>
    </tr>
    <tr>
      <td valign="top">VisIVO<br>
      </td>
      <td valign="top">Opens a new 3D plot, using the cols&gt;3 as
indices into the colourmap, glyph sizes etc....<br>
      </td>
      <td valign="top">?<br>
      </td>
    </tr>
    <tr>
      <td valign="top">TabView<br>
      </td>
      <td valign="top">Highlights the columns<br>
      </td>
      <td valign="top">Creates a new subset consisting only of those
cols.<br>
      </td>
    </tr>
    <tr>
      <td valign="top">xmdv-lite<br>
      </td>
      <td valign="top">On the parallel axes plot, highlights the chosen
columns.&nbsp; On the scatter plots, could highlight or show only the
relevant scatter plots<br>
      </td>
      <td valign="top">New plots only of selected cols<br>
      </td>
    </tr>
    <tr>
      <td valign="top">Anomaly Detector<br>
      </td>
      <td valign="top"><br>
      </td>
      <td valign="top">Runs the EM-algorithm only on the selected
columns<br>
      </td>
    </tr>
    <tr>
      <td valign="top">Aladin<br>
      </td>
      <td valign="top">Maybe opens VOPlot?<br>
      </td>
      <td valign="top">create a new pane with the reduced columns?<br>
      </td>
    </tr>
  </tbody>
</table>
<br>
This separation of "showing" and "selecting a subset" makes me think
that we've confounded the two with our showObjects message.&nbsp; I wonder
if we ought to split it so that we can distinguish between creating a
new row subset (a la Topcat) and simply highlighting the selected rows
(a la Aladin).&nbsp; Indeed, perhaps the two "create subset" messages could
be merged into one.<br>
<br>
<br>
<blockquote
 cite="mid...@an..."
 type="cite">
  <pre wrap="">  </pre>
  <blockquote type="cite">
    <blockquote type="cite">
      <blockquote type="cite">
        <pre wrap=""> 	2. Identifying several sets of rows.
        </pre>
      </blockquote>
    </blockquote>
    <blockquote type="cite">
      <pre wrap="">Actually you can sort of do this in TOPCAT as it stands.  Each received
ivo://votech.org/votable/showObjects message results in the creation
of a new row subset (old ones are not discarded (even if you want them
to be, which admittedly can be an annoyance)).  Within TOPCAT you
can then select which subsets are plotted etc.
      </pre>
    </blockquote>
    <pre wrap=""> 	OK - that is very useful. Do you give the row subset a name in
the message in which you define it?
    </pre>
  </blockquote>
  <pre wrap=""><!---->
No, TOPCAT assigns names automatically - currently e.g. weka-1, weka-2,
weka-3, ....  May or may not be ideal for your purposes.

Mark

  </pre>
</blockquote>
<br>
<br>
<div class="moz-signature">-- <br>
<hr>
<center>AstroGrid/VOTech<br>
&amp;<br>
Institute for Astronomy, Edinburgh<br>
<a href="skype:johndavidtaylor?chat">Skype:johndavidtaylor</a><br>
</center>
<hr>
</div>
</body>
</html>

Re: [Plastic-devs] Future API changes

[John T. <jon...@gm...>]

Just to clarify for everyone on the list - this thread started with Mark 
and me* chatting about our gripes with the current Plastic spec.   As 
Mark says, we should be discussing this on the list.  For people 
implementing plastic right now....don't panic...we're not about to 
suddenly change things in a way that breaks anything (though we may 
introduce functionality if it doesn't break things).   In my opinion the 
spec is adequate for what we want to do at the moment.  Thinking to the 
future though,  there will inevitably be changes when/if we propose our 
work as a formal standard so we're thinking about things that we'd like 
to change at the same time.  Before this happens I hope we can do a 
thorough review of existing messaging systems (cf Doug's email of a 
while back) and see what we can learn.

Feel free to add your complaints or wishlist to:

http://eurovotech.org/twiki/bin/view/VOTech/PlasticRemould

John

*excuse the poor grammar

Mark Taylor wrote:
> John, I'm exploding this to the mailing list, where I probably should
> have started off the conversation, hope you don't object.  
> I think there's enough context here for other readers to pick up 
> what we're talking about.
>
> On Tue, 18 Jul 2006, John Taylor wrote:
>
>   
>> Mark Taylor wrote:
>>     
>>> One possibility is to replace hub methods getName(), getRegisteredIds(),
>>> getUnderstoodMessages() and getMessageRegisteredHubIdsPleaseIfYouDontMind()
>>> with a single getRegisteredApplications() which returns an 
>>> <array>-of-<struct>s/List-of-Maps giving all the information that
>>> the hub knows from registration time about every registered application
>>> (name and supported message IDs).  I'm not saying it's a good idea mind.
>>>
>>>       
>> Actually I quite like this.  Instead of registering with your name, you 
>> register with a struct containing your name, ivorn, iconurl, description 
>> + whatever else we decide to put in there in the future.  We'll define a 
>> few well-known keys to get started, but people can expand the struct as 
>> they see fit.  And you have a single method to return the lot, as you 
>> just described. 
>>     
>
> I hadn't thought of doing it for registration as well, but that probably
> makes sense if we're going to do that.  Downsides:
>
>   1.  Different from current practice (but looks like we may be heading
>       for a backwardly-incompatible change to the protocol sometime
>       soon so may as well make the most of it).
>
>   2.  One call fits all may mean you're transmitting more data than you
>       need to - however we're probably not talking about very large
>       amounts so this is probably not important.
>
>   3.  Means you can neither register not obtain any application metadata
>       without the ability to process structs.  This is probably going to 
>       scupper any attempt to use HTTP-GET for PLASTIC access.  There
>       may be other platforms too that don't lend themselves to an
>       XML-RPC implementation which can decode structs (i.e. don't
>       have langauge features resembling associative arrays/hashes) - 
>       can anyone think of examples of this which might be important (IDL?)
>
>   
>> We might want to keep the application's understood messages out of this 
>> struct though, to allow them to be changed post-registration as per your 
>> earlier email.
>>     
>
> yes, good point.
>
>
>   
>>> Now (v0.5) that you can't register to receive all messages by sending 
>>> an empty supportedMessages array at registration time, we're
>>> missing a way for monitor-type applications to see everything that's
>>> going on in the hub (good for debugging).  This could be addressed
>>> by a new message
>>>
>>>    id:
>>>        ivo://votech.org/hub/event/requestSent
>>>    args:
>>>        String senderId
>>>        String messageId
>>>        List recipientIds
>>>        String requestId
>>>    return type:
>>>        void
>>>
>>> which the hub sends to interested parties whenenever it makes a 
>>> request to listeners (er, with the possible exception of requestSent or 
>>> responseRecieved requests...).  The requestId argument is just a 
>>> string unique to this request.
>>>
>>> We also introduce a second message
>>>
>>>    id:
>>>        ivo://votech.org/hub/event/responseReceived
>>>    args:
>>>        String requestId
>>>        String recipientId
>>>        Object response
>>>    return type:
>>>        void
>>>
>>> Here the requestId is used to match the response up with the
>>> earlier-notified request, for those that care.
>>>   
>>>       
>> I like it.  Let's put the senderId in there as well, just so that the 
>> client doesn't _have_ to match up the two messages to find out who the 
>> response is going to.
>>
>>     
>>> Monitor-type applications can now see all the requests that go out from
>>> and all the responses that come back to the hub, which is good.
>>>
>>> A bonus is that this pair of messages can also be used by applications
>>> which want to use callbacks to get responses from the hub; either
>>> because they're not multithreaded or because response times are
>>> potentially slow.  Instead of calling request() synchronously,
>>> they call requestAsynch() and make sure they're registered to 
>>> receive requestReceived messages (ignoring ones they didn't send
>>> in the first place if necessary).  None of this requires any additional
>>> hub methods.
>>>   
>>>       
>> This is promising, but I have a couple of comments:
>> 1) We could reduce the burden on the client of sifting through all the 
>> responses by having a similar
>>
>> ivo://votech.org/hub/event/myResponseReceived
>>
>> message that only gets sent to the sender of the original message.
>>     
>
> OK.
>
>   
>> 2) It could be a little tricky matching up responses to messages.  If 
>> you're monitoring the requestReceived message you can tie them together 
>> using your proposed requestId, but it's not obvious to me how you tie 
>> together (with certainty) your original call the the hub with the 
>> response message.  Maybe it doesn't matter.  
>>     
>
> Hadn't thought of that.  However, I can think of a good solution:
> get the hub requestAsynch() methods to return the responseId for the
> request rather than void.
>
> Thinking about it, with this in place you can achieve anything using
> the asynchronous hub request methods that you could before using 
> the synchronous ones (though with more client-side work).  
> The possibility therefore arises of ditching the synchronous 
> hub request methods.  Is that a bit too radical?
>
> Mark
>
>   

Re: [Plastic-devs] Future API changes

[Mark T. <m.b...@br...>]

John, I'm exploding this to the mailing list, where I probably should
have started off the conversation, hope you don't object.  
I think there's enough context here for other readers to pick up 
what we're talking about.

On Tue, 18 Jul 2006, John Taylor wrote:

> Mark Taylor wrote:
> >
> > One possibility is to replace hub methods getName(), getRegisteredIds(),
> > getUnderstoodMessages() and getMessageRegisteredHubIdsPleaseIfYouDontMind()
> > with a single getRegisteredApplications() which returns an 
> > <array>-of-<struct>s/List-of-Maps giving all the information that
> > the hub knows from registration time about every registered application
> > (name and supported message IDs).  I'm not saying it's a good idea mind.
> >
>
> Actually I quite like this.  Instead of registering with your name, you 
> register with a struct containing your name, ivorn, iconurl, description 
> + whatever else we decide to put in there in the future.  We'll define a 
> few well-known keys to get started, but people can expand the struct as 
> they see fit.  And you have a single method to return the lot, as you 
> just described. 

I hadn't thought of doing it for registration as well, but that probably
makes sense if we're going to do that.  Downsides:

  1.  Different from current practice (but looks like we may be heading
      for a backwardly-incompatible change to the protocol sometime
      soon so may as well make the most of it).

  2.  One call fits all may mean you're transmitting more data than you
      need to - however we're probably not talking about very large
      amounts so this is probably not important.

  3.  Means you can neither register not obtain any application metadata
      without the ability to process structs.  This is probably going to 
      scupper any attempt to use HTTP-GET for PLASTIC access.  There
      may be other platforms too that don't lend themselves to an
      XML-RPC implementation which can decode structs (i.e. don't
      have langauge features resembling associative arrays/hashes) - 
      can anyone think of examples of this which might be important (IDL?)

> We might want to keep the application's understood messages out of this 
> struct though, to allow them to be changed post-registration as per your 
> earlier email.

yes, good point.


> > Now (v0.5) that you can't register to receive all messages by sending 
> > an empty supportedMessages array at registration time, we're
> > missing a way for monitor-type applications to see everything that's
> > going on in the hub (good for debugging).  This could be addressed
> > by a new message
> >
> >    id:
> >        ivo://votech.org/hub/event/requestSent
> >    args:
> >        String senderId
> >        String messageId
> >        List recipientIds
> >        String requestId
> >    return type:
> >        void
> >
> > which the hub sends to interested parties whenenever it makes a 
> > request to listeners (er, with the possible exception of requestSent or 
> > responseRecieved requests...).  The requestId argument is just a 
> > string unique to this request.
> >
> > We also introduce a second message
> >
> >    id:
> >        ivo://votech.org/hub/event/responseReceived
> >    args:
> >        String requestId
> >        String recipientId
> >        Object response
> >    return type:
> >        void
> >
> > Here the requestId is used to match the response up with the
> > earlier-notified request, for those that care.
> >   
> I like it.  Let's put the senderId in there as well, just so that the 
> client doesn't _have_ to match up the two messages to find out who the 
> response is going to.
> 
> > Monitor-type applications can now see all the requests that go out from
> > and all the responses that come back to the hub, which is good.
> >
> > A bonus is that this pair of messages can also be used by applications
> > which want to use callbacks to get responses from the hub; either
> > because they're not multithreaded or because response times are
> > potentially slow.  Instead of calling request() synchronously,
> > they call requestAsynch() and make sure they're registered to 
> > receive requestReceived messages (ignoring ones they didn't send
> > in the first place if necessary).  None of this requires any additional
> > hub methods.
> >   
> This is promising, but I have a couple of comments:
> 1) We could reduce the burden on the client of sifting through all the 
> responses by having a similar
> 
> ivo://votech.org/hub/event/myResponseReceived
> 
> message that only gets sent to the sender of the original message.

OK.

> 2) It could be a little tricky matching up responses to messages.  If 
> you're monitoring the requestReceived message you can tie them together 
> using your proposed requestId, but it's not obvious to me how you tie 
> together (with certainty) your original call the the hub with the 
> response message.  Maybe it doesn't matter.  

Hadn't thought of that.  However, I can think of a good solution:
get the hub requestAsynch() methods to return the responseId for the
request rather than void.

Thinking about it, with this in place you can achieve anything using
the asynchronous hub request methods that you could before using 
the synchronous ones (though with more client-side work).  
The possibility therefore arises of ditching the synchronous 
hub request methods.  Is that a bit too radical?

Mark

-- 
Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
m.b...@br... +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/

[Plastic-devs] Manifesto [was: Re: Temporary files and IDs for loadFromURL

[John T. <jon...@gm...>]

> [assuming you mean "two files obtained from the same source at 
> different times might not be the same"]:
>
> true, but in 99(?)% of cases it probably is the same.  Here's a
> proposal for the manifesto: "A simple solution which does the right
> thing nearly all the time is better than a complicated one which
> does the right thing all the time".
>   
I really like the idea of a manifesto describing what we're about.

Anyone else care to summarize what we aim to do?  Add your contribution to
http://eurovotech.org/twiki/bin/view/VOTech/PlasticManifesto

John

Re: [Plastic-devs] Temporary files and IDs for loadFromURL

[John T. <jd...@ro...>]

My own view is that this is something we should seriously look at, but 
not rush into.  At the moment we're dealing with files that are not 
going to exceed the user's disk space, not even close.  So, apart from 
the processing overhead, it's not really a big deal if each application 
makes its own copy of the data.  With remote URLs, there is the issue of 
multiple downloads of the same data, but the feeling a while back was 
that we can leave the user's cache to deal with this.

If we do decide to do it, then I think it would have to be an optional 
extension of a hub, and accessed by messaging.  In fact, it needn't be 
bundled with a hub - we can define a set of messages for a third-party 
application to "adopt" a file, do the reference counting and clean-up.  
Perhaps this "cache" will be bundled with a particular hub impl, perhaps 
it won't.  The client application will have to deal with its presence or 
absence.

I think we should proceed with some caution (contrary to my usual 
act-first-think-later behaviour!).  As Tony pointed out during our 
meeting (Richard), this sort of thing has been done before and we should 
probably do some investigating.

John


>
> Your ideas are quite reasonable ones, but the reason I'm not very
> enthusiastic about them is that part of the unwritten philosophy behind
> PLASTIC (at least in my understanding) is that it's simple in 
> order to be difficult to break.  For instance, reference counting
> to keep track of temporary resources is clearly the Right Thing 
> to do if you have a controlled environment with reliable object
> destructors etc.  But by its nature a hub is dealing with unreliable
> connections to unreliably implemented clients, and a client that 
> forgets to unregister itself, or its interest in a file, could 
> very easily end up leaving (possibly large) temporary files hanging 
> around for much longer than they ought to be there.  This sort of 
> thing would be fairly easy to track down and fix in a single-process 
> application, but practically impossible when you've got no idea
> what applications you might or might not be talking to and how
> well or badly they might behave.
>
> That's my take on it anyway - by all means other PLASTICkers chip 
> in with your points of view (including, obviously, right of reply
> from Richard); I'm quite open to debating either the general or
> specific points and being persuaded otherwise.  Either way, if we 
> reach a consensus on this it might be a good idea to agree on a 
> kind of explicit Manifesto or Philosophy of PLASTIC document 
> which can be referred to to clarify this kind of debate/suggestion 
> in the future.
>
>   
>   

-- 
------------------------------------------------------------------------
AstroGrid/VOTech
&
Institute for Astronomy, Edinburgh
Skype:johndavidtaylor <skype:johndavidtaylor?chat>

------------------------------------------------------------------------

Re: [Plastic-devs] Temporary files and IDs for loadFromURL

[Noel W. <Noe...@ma...>]

On 17 Jul 2006, at 17:04, John Taylor wrote:

>
>
> Mark Taylor wrote:
>> On Fri, 14 Jul 2006, R Holbrey wrote:
>>
>>> Doesn't "load" have a second arg anyway? This seems to be the  
>>> main difference between two. Help, I'm still confused about this  
>>> one..
>>>
>>
>> the main difference is that in "loadFromURL" you pass the URL.
>> In "load" the text of the votable is sent as the content of the  
>> first argument.  Not very scalable, admittedly.
>>
> FWIW I think this is actually used in the Workbench though...in  
> fact it looks as though they might appreciate a similar "in-line"  
> version of the "load fits" message.  Noel: is this for sending data  
> that comes back by value from a CEA app?
>

yep. that's it - data is already in memory, and if it's got that far  
it's obviously not tooo big.


> John
>

Re: [Plastic-devs] Temporary files and IDs for loadFromURL

[John T. <jon...@gm...>]

>
>   
>>>> For symmetry's sake we could also make the id optional for this message,
>>>> and use the table name if it's missing, but I'd prefer not to given the
>>>> potential for lack of uniqueness.
>>>>         
>> Maybe it's because plastic is so new to me but I don't quite get the logic 
>> here. How many apps are going to break if you deprecate one message and 
>> introduce a few new ones (with clarification). Surely this is the time to 
>> make changes and revisions.
>>     
>
> Well I don't think we should/will make this change.  However, I think
> we currently consider ourselves in an early phase in which we can
> make some backwardly incompatible changes if we consider there are
> sufficient benefits since the developer community is still quite small
> and responsive.
>   
Richard, this was me confusing the issue by thinking aloud.  There's no 
good reason to make this ID optional except to make both loadVOTable 
messages have the same argument signature.  Since (with hindsight) we'd 
actually prefer both messages to have the ID as mandatory, my raising 
the idea was daft. 

Re: [Plastic-devs] Temporary files and IDs for loadFromURL

[John T. <jon...@gm...>]

Mark Taylor wrote:
> On Fri, 14 Jul 2006, R Holbrey wrote:
>   
>   
>> Doesn't "load" have a second arg anyway? This seems to be the main 
>> difference between two. Help, I'm still confused about this one..
>>     
>
> the main difference is that in "loadFromURL" you pass the URL.
> In "load" the text of the votable is sent as the content of the 
> first argument.  Not very scalable, admittedly.
>   
FWIW I think this is actually used in the Workbench though...in fact it 
looks as though they might appreciate a similar "in-line" version of the 
"load fits" message.  Noel: is this for sending data that comes back by 
value from a CEA app?

John

Re: [Plastic-devs] Temporary files and IDs for loadFromURL

[John T. <jon...@gm...>]

Mark Taylor wrote:
>   
>> The solution is to adopt the same argument list as 
>> ivo://.../votable/load and have a second argument specifying the ID.   I 
>> suggest we make this second argument optional (but strongly recommended) 
>> to avoid breaking existing apps, and if it's missing assume that the 
>> id=url as we have now.
>>     
>
> After thinking about it for a while, I agree, but subject to the 
> following comment.  I seem to remember that the, or at least a
> reason we decided to use the URL for the table ID was that 
> it meant if the table had some persistence (i.e. it was a non-temporary
> file in the filesystem, or something on a remote server) then two
> applications could know they were talking about the same thing even
> if they picked it up independently rather than one having got it from
> the other through PLASTIC.  So for a file which is (expected to be) 
> persistent the URL is a good choice for the identifier.
>   
I don't remember the discussion, but the idea seems pretty sensible.
> [snip]

>   
>> On the subject of optional arguments.  I think it might be a good 
>> pattern that any optional args go into a struct tacked on after the 
>> mandatory arguments.  That way they can be referred to by name rather 
>> than position.
>>     
>
> Hmm, interesting thought.  My initial feeling is that for arguments
> which are optional but strongly recommended like the one you've 
> suggested above (i.e. ones we should have put in in the first 
> place but didn't realise until too late) there's not much advantage
> in this.  However, if some commands end up with a forest of options 
> it could be a good plan.
>   
I think the id parameter falls into the category of "should be mandatory 
from now on", so I agree, specifying it by position is easier and 
clearer.  I was really thinking on the lines of (e.g.) if someone wanted 
to do something very app-specific such as send a color=blue parameter in 
with the showObjects message, though my original post isn't very clear.

Re: [Plastic-devs] Temporary files and IDs for loadFromURL

[Mark T. <m.b...@br...>]

On Fri, 14 Jul 2006, R Holbrey wrote:

> On Fri, 14 Jul 2006, Mark Taylor wrote:
> 
> > On Fri, 14 Jul 2006, John Taylor wrote:
> >
> >> Hi All,
> >> We have a slight problem with the current
> >> ivo://votech.org/votable/loadFromURL message in that it uses the URL of
> >> the file as the table's ID.    Here's the problem:
> >> The user is running three apps A, B, C.  Application A creates a subset
> >> and wants to broadcast it to B and C.  It does this by creating a
> >> temporary file and sending it to B and C.  B & C dutifully make their
> >> own copies of it.   The user then starts application D, and shuts down
> >> application A, which clears up its temporary file.  How does the user
> >> then send the file from B to D and ensure that all three remaining apps
> >> think they are referring to the same table?
> >
> > Actually, as per the trouble Richard had a short while ago, it's worse
> > than that - TOPCAT at least doesn't wait until shutdown to clear up
> > such temporary files, it does it as soon as the votable/loadFromURL
> > request has returned.
> 
> Sorry to keep harking on, but I still think things would be better if the 
> hub had more control.

Richard,

sorry for not making a response to your previous message, and (in 
advance) for the nature of the response to this one.

Your ideas are quite reasonable ones, but the reason I'm not very
enthusiastic about them is that part of the unwritten philosophy behind
PLASTIC (at least in my understanding) is that it's simple in 
order to be difficult to break.  For instance, reference counting
to keep track of temporary resources is clearly the Right Thing 
to do if you have a controlled environment with reliable object
destructors etc.  But by its nature a hub is dealing with unreliable
connections to unreliably implemented clients, and a client that 
forgets to unregister itself, or its interest in a file, could 
very easily end up leaving (possibly large) temporary files hanging 
around for much longer than they ought to be there.  This sort of 
thing would be fairly easy to track down and fix in a single-process 
application, but practically impossible when you've got no idea
what applications you might or might not be talking to and how
well or badly they might behave.

That's my take on it anyway - by all means other PLASTICkers chip 
in with your points of view (including, obviously, right of reply
from Richard); I'm quite open to debating either the general or
specific points and being persuaded otherwise.  Either way, if we 
reach a consensus on this it might be a good idea to agree on a 
kind of explicit Manifesto or Philosophy of PLASTIC document 
which can be referred to to clarify this kind of debate/suggestion 
in the future.


> Doesn't "load" have a second arg anyway? This seems to be the main 
> difference between two. Help, I'm still confused about this one..

the main difference is that in "loadFromURL" you pass the URL.
In "load" the text of the votable is sent as the content of the 
first argument.  Not very scalable, admittedly.


> > After thinking about it for a while, I agree, but subject to the
> > following comment.  I seem to remember that the, or at least a
> > reason we decided to use the URL for the table ID was that
> > it meant if the table had some persistence (i.e. it was a non-temporary
> > file in the filesystem, or something on a remote server) then two
> > applications could know they were talking about the same thing even
> > if they picked it up independently rather than one having got it from
> > the other through PLASTIC.  So for a file which is (expected to be)
> > persistent the URL is a good choice for the identifier.
> 
> This seems a bit dicey to me. Two files obtained from different sources at 
> the same time (or vice versa) might not be the same ..??

[assuming you mean "two files obtained from the same source at 
different times might not be the same"]:

true, but in 99(?)% of cases it probably is the same.  Here's a
proposal for the manifesto: "A simple solution which does the right
thing nearly all the time is better than a complicated one which
does the right thing all the time".


> >> For symmetry's sake we could also make the id optional for this message,
> >> and use the table name if it's missing, but I'd prefer not to given the
> >> potential for lack of uniqueness.
>
> Maybe it's because plastic is so new to me but I don't quite get the logic 
> here. How many apps are going to break if you deprecate one message and 
> introduce a few new ones (with clarification). Surely this is the time to 
> make changes and revisions.

Well I don't think we should/will make this change.  However, I think
we currently consider ourselves in an early phase in which we can
make some backwardly incompatible changes if we consider there are
sufficient benefits since the developer community is still quite small
and responsive.

Mark

-- 
Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
m.b...@br... +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/

Re: [Plastic-devs] Temporary files and IDs for loadFromURL

[R H. <ric...@co...>]

On Fri, 14 Jul 2006, Mark Taylor wrote:

> On Fri, 14 Jul 2006, John Taylor wrote:
>
>> Hi All,
>> We have a slight problem with the current
>> ivo://votech.org/votable/loadFromURL message in that it uses the URL of
>> the file as the table's ID.    Here's the problem:
>> The user is running three apps A, B, C.  Application A creates a subset
>> and wants to broadcast it to B and C.  It does this by creating a
>> temporary file and sending it to B and C.  B & C dutifully make their
>> own copies of it.   The user then starts application D, and shuts down
>> application A, which clears up its temporary file.  How does the user
>> then send the file from B to D and ensure that all three remaining apps
>> think they are referring to the same table?
>
> Actually, as per the trouble Richard had a short while ago, it's worse
> than that - TOPCAT at least doesn't wait until shutdown to clear up
> such temporary files, it does it as soon as the votable/loadFromURL
> request has returned.

Sorry to keep harking on, but I still think things would be better if the 
hub had more control.

If "A" registers an interesting file with the hub, the hub can maintain 
the host and file URL (a plasticFileID?) and send messages to third 
parties, counting how many are interested. "A" can unregister and as long 
as other apps are interested and registered -- ie the count remains above 
zero -- and only one temporary file need exist. This still leaves D and 
others who come along later in the dark, unless there is an option to 
request any 'hub-cached' files or unless say "B" specifically sends it. 
Either way, the count would increase by one for "D".

It might also be possible to automate updates of a given file ID, and the 
earlier file could be removed, when all other apps have the new version.
The 'plasticFileID' might (?) even be able to stay the same.

>
>> The solution is to adopt the same argument list as
>> ivo://.../votable/load and have a second argument specifying the ID.   I
>> suggest we make this second argument optional (but strongly recommended)
>> to avoid breaking existing apps, and if it's missing assume that the
>> id=url as we have now.
>

Doesn't "load" have a second arg anyway? This seems to be the main 
difference between two. Help, I'm still confused about this one..

> After thinking about it for a while, I agree, but subject to the
> following comment.  I seem to remember that the, or at least a
> reason we decided to use the URL for the table ID was that
> it meant if the table had some persistence (i.e. it was a non-temporary
> file in the filesystem, or something on a remote server) then two
> applications could know they were talking about the same thing even
> if they picked it up independently rather than one having got it from
> the other through PLASTIC.  So for a file which is (expected to be)
> persistent the URL is a good choice for the identifier.

This seems a bit dicey to me. Two files obtained from different sources at 
the same time (or vice versa) might not be the same ..??

>
> So an application which injects a table into the PLASTIC system by
> sending a loadFromURL should use the URL itself as the ID
> (or, equivalently, supply no ID and let apps assume id=url,
> though I'm happy for clarity's sake to recommend that both arguments
> are supplied).  In this way an application which happens to have
> loaded the same table from a non-PLASTIC source has a chance of
> knowing that fact.  It also solves the problem of how the sending
> application is supposed to generate an ID guaranteed not to
> clash with someone else's.  But if you've acquired a table by
> responding to a loadFromURL, then if you subsequently send a
> loadFromURL you should propagate the ID that came with it.
> This may or may not be the same as the URL you send.
>
>
>> Is anyone actually implementing ivo://.../votable/load?
>
> yes.
>
>> For symmetry's sake we could also make the id optional for this message,
>> and use the table name if it's missing, but I'd prefer not to given the
>> potential for lack of uniqueness.
>

Maybe it's because plastic is so new to me but I don't quite get the logic 
here. How many apps are going to break if you deprecate one message and 
introduce a few new ones (with clarification). Surely this is the time to 
make changes and revisions.


Richard

Re: [Plastic-devs] Temporary files and IDs for loadFromURL

[Mark T. <m.b...@br...>]

On Fri, 14 Jul 2006, John Taylor wrote:

> Hi All,
> We have a slight problem with the current 
> ivo://votech.org/votable/loadFromURL message in that it uses the URL of 
> the file as the table's ID.    Here's the problem:
> The user is running three apps A, B, C.  Application A creates a subset 
> and wants to broadcast it to B and C.  It does this by creating a 
> temporary file and sending it to B and C.  B & C dutifully make their 
> own copies of it.   The user then starts application D, and shuts down 
> application A, which clears up its temporary file.  How does the user 
> then send the file from B to D and ensure that all three remaining apps 
> think they are referring to the same table?

Actually, as per the trouble Richard had a short while ago, it's worse
than that - TOPCAT at least doesn't wait until shutdown to clear up
such temporary files, it does it as soon as the votable/loadFromURL
request has returned.

> The solution is to adopt the same argument list as 
> ivo://.../votable/load and have a second argument specifying the ID.   I 
> suggest we make this second argument optional (but strongly recommended) 
> to avoid breaking existing apps, and if it's missing assume that the 
> id=url as we have now.

After thinking about it for a while, I agree, but subject to the 
following comment.  I seem to remember that the, or at least a
reason we decided to use the URL for the table ID was that 
it meant if the table had some persistence (i.e. it was a non-temporary
file in the filesystem, or something on a remote server) then two
applications could know they were talking about the same thing even
if they picked it up independently rather than one having got it from
the other through PLASTIC.  So for a file which is (expected to be) 
persistent the URL is a good choice for the identifier.

So an application which injects a table into the PLASTIC system by
sending a loadFromURL should use the URL itself as the ID
(or, equivalently, supply no ID and let apps assume id=url,
though I'm happy for clarity's sake to recommend that both arguments 
are supplied).  In this way an application which happens to have 
loaded the same table from a non-PLASTIC source has a chance of 
knowing that fact.  It also solves the problem of how the sending 
application is supposed to generate an ID guaranteed not to 
clash with someone else's.  But if you've acquired a table by 
responding to a loadFromURL, then if you subsequently send a 
loadFromURL you should propagate the ID that came with it.  
This may or may not be the same as the URL you send.


> Is anyone actually implementing ivo://.../votable/load?

yes.

> For symmetry's sake we could also make the id optional for this message, 
> and use the table name if it's missing, but I'd prefer not to given the 
> potential for lack of uniqueness.

I agree there's no reason to change this (also could potentially cause
backward compatibility problems).

> On the subject of optional arguments.  I think it might be a good 
> pattern that any optional args go into a struct tacked on after the 
> mandatory arguments.  That way they can be referred to by name rather 
> than position.

Hmm, interesting thought.  My initial feeling is that for arguments
which are optional but strongly recommended like the one you've 
suggested above (i.e. ones we should have put in in the first 
place but didn't realise until too late) there's not much advantage
in this.  However, if some commands end up with a forest of options 
it could be a good plan.

-- 
Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
m.b...@br... +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/

[Plastic-devs] [Fwd: Temporary files and IDs for loadFromURL]

[John T. <jd...@ro...>]

Not sure this got through first time....

-------- Original Message --------
Subject: 	Temporary files and IDs for loadFromURL
Date: 	Fri, 14 Jul 2006 13:22:08 +0100
From: 	John Taylor <jd...@ro...>
To: 	pla...@li... 
<pla...@li...>



Hi All,
We have a slight problem with the current 
ivo://votech.org/votable/loadFromURL message in that it uses the URL of 
the file as the table's ID.    Here's the problem:
The user is running three apps A, B, C.  Application A creates a subset 
and wants to broadcast it to B and C.  It does this by creating a 
temporary file and sending it to B and C.  B & C dutifully make their 
own copies of it.   The user then starts application D, and shuts down 
application A, which clears up its temporary file.  How does the user 
then send the file from B to D and ensure that all three remaining apps 
think they are referring to the same table?

The solution is to adopt the same argument list as 
ivo://.../votable/load and have a second argument specifying the ID.   I 
suggest we make this second argument optional (but strongly recommended) 
to avoid breaking existing apps, and if it's missing assume that the 
id=url as we have now.

Is anyone actually implementing ivo://.../votable/load?
For symmetry's sake we could also make the id optional for this message, 
and use the table name if it's missing, but I'd prefer not to given the 
potential for lack of uniqueness.

On the subject of optional arguments.  I think it might be a good 
pattern that any optional args go into a struct tacked on after the 
mandatory arguments.  That way they can be referred to by name rather 
than position.

John



-- 
------------------------------------------------------------------------
AstroGrid/VOTech
&
Institute for Astronomy, Edinburgh
Skype:johndavidtaylor <skype:johndavidtaylor?chat>

------------------------------------------------------------------------



-- 
------------------------------------------------------------------------
AstroGrid/VOTech
&
Institute for Astronomy, Edinburgh
Skype:johndavidtaylor <skype:johndavidtaylor?chat>

------------------------------------------------------------------------

[Plastic-devs] Test message - pls ignore

[John T. <jd...@ro...>]

Sorry for the intrusion - just testing.

-- 
------------------------------------------------------------------------
AstroGrid/VOTech
&
Institute for Astronomy, Edinburgh
Skype:johndavidtaylor <skype:johndavidtaylor?chat>

------------------------------------------------------------------------

[Plastic-devs] Temporary files and IDs for loadFromURL

[John T. <jd...@ro...>]

Hi All,
We have a slight problem with the current 
ivo://votech.org/votable/loadFromURL message in that it uses the URL of 
the file as the table's ID.    Here's the problem:
The user is running three apps A, B, C.  Application A creates a subset 
and wants to broadcast it to B and C.  It does this by creating a 
temporary file and sending it to B and C.  B & C dutifully make their 
own copies of it.   The user then starts application D, and shuts down 
application A, which clears up its temporary file.  How does the user 
then send the file from B to D and ensure that all three remaining apps 
think they are referring to the same table?

The solution is to adopt the same argument list as 
ivo://.../votable/load and have a second argument specifying the ID.   I 
suggest we make this second argument optional (but strongly recommended) 
to avoid breaking existing apps, and if it's missing assume that the 
id=url as we have now.

Is anyone actually implementing ivo://.../votable/load?
For symmetry's sake we could also make the id optional for this message, 
and use the table name if it's missing, but I'd prefer not to given the 
potential for lack of uniqueness.

On the subject of optional arguments.  I think it might be a good 
pattern that any optional args go into a struct tacked on after the 
mandatory arguments.  That way they can be referred to by name rather 
than position.

John



-- 
------------------------------------------------------------------------
AstroGrid/VOTech
&
Institute for Astronomy, Edinburgh
Skype:johndavidtaylor <skype:johndavidtaylor?chat>

------------------------------------------------------------------------