Re: [Plastic-devs] Temporary files and IDs for loadFromURL

[Mark T. <m.b...@br...>]

On Fri, 14 Jul 2006, R Holbrey wrote:

> On Fri, 14 Jul 2006, Mark Taylor wrote:
> 
> > On Fri, 14 Jul 2006, John Taylor wrote:
> >
> >> Hi All,
> >> We have a slight problem with the current
> >> ivo://votech.org/votable/loadFromURL message in that it uses the URL of
> >> the file as the table's ID.    Here's the problem:
> >> The user is running three apps A, B, C.  Application A creates a subset
> >> and wants to broadcast it to B and C.  It does this by creating a
> >> temporary file and sending it to B and C.  B & C dutifully make their
> >> own copies of it.   The user then starts application D, and shuts down
> >> application A, which clears up its temporary file.  How does the user
> >> then send the file from B to D and ensure that all three remaining apps
> >> think they are referring to the same table?
> >
> > Actually, as per the trouble Richard had a short while ago, it's worse
> > than that - TOPCAT at least doesn't wait until shutdown to clear up
> > such temporary files, it does it as soon as the votable/loadFromURL
> > request has returned.
> 
> Sorry to keep harking on, but I still think things would be better if the 
> hub had more control.

Richard,

sorry for not making a response to your previous message, and (in 
advance) for the nature of the response to this one.

Your ideas are quite reasonable ones, but the reason I'm not very
enthusiastic about them is that part of the unwritten philosophy behind
PLASTIC (at least in my understanding) is that it's simple in 
order to be difficult to break.  For instance, reference counting
to keep track of temporary resources is clearly the Right Thing 
to do if you have a controlled environment with reliable object
destructors etc.  But by its nature a hub is dealing with unreliable
connections to unreliably implemented clients, and a client that 
forgets to unregister itself, or its interest in a file, could 
very easily end up leaving (possibly large) temporary files hanging 
around for much longer than they ought to be there.  This sort of 
thing would be fairly easy to track down and fix in a single-process 
application, but practically impossible when you've got no idea
what applications you might or might not be talking to and how
well or badly they might behave.

That's my take on it anyway - by all means other PLASTICkers chip 
in with your points of view (including, obviously, right of reply
from Richard); I'm quite open to debating either the general or
specific points and being persuaded otherwise.  Either way, if we 
reach a consensus on this it might be a good idea to agree on a 
kind of explicit Manifesto or Philosophy of PLASTIC document 
which can be referred to to clarify this kind of debate/suggestion 
in the future.


> Doesn't "load" have a second arg anyway? This seems to be the main 
> difference between two. Help, I'm still confused about this one..

the main difference is that in "loadFromURL" you pass the URL.
In "load" the text of the votable is sent as the content of the 
first argument.  Not very scalable, admittedly.


> > After thinking about it for a while, I agree, but subject to the
> > following comment.  I seem to remember that the, or at least a
> > reason we decided to use the URL for the table ID was that
> > it meant if the table had some persistence (i.e. it was a non-temporary
> > file in the filesystem, or something on a remote server) then two
> > applications could know they were talking about the same thing even
> > if they picked it up independently rather than one having got it from
> > the other through PLASTIC.  So for a file which is (expected to be)
> > persistent the URL is a good choice for the identifier.
> 
> This seems a bit dicey to me. Two files obtained from different sources at 
> the same time (or vice versa) might not be the same ..??

[assuming you mean "two files obtained from the same source at 
different times might not be the same"]:

true, but in 99(?)% of cases it probably is the same.  Here's a
proposal for the manifesto: "A simple solution which does the right
thing nearly all the time is better than a complicated one which
does the right thing all the time".


> >> For symmetry's sake we could also make the id optional for this message,
> >> and use the table name if it's missing, but I'd prefer not to given the
> >> potential for lack of uniqueness.
>
> Maybe it's because plastic is so new to me but I don't quite get the logic 
> here. How many apps are going to break if you deprecate one message and 
> introduce a few new ones (with clarification). Surely this is the time to 
> make changes and revisions.

Well I don't think we should/will make this change.  However, I think
we currently consider ourselves in an early phase in which we can
make some backwardly incompatible changes if we consider there are
sufficient benefits since the developer community is still quite small
and responsive.

Mark

-- 
Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
m.b...@br... +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/