Re: [Plastic-devs] Temporary files and IDs for loadFromURL

[R H. <ric...@co...>]

On Fri, 14 Jul 2006, Mark Taylor wrote:

> On Fri, 14 Jul 2006, John Taylor wrote:
>
>> Hi All,
>> We have a slight problem with the current
>> ivo://votech.org/votable/loadFromURL message in that it uses the URL of
>> the file as the table's ID.    Here's the problem:
>> The user is running three apps A, B, C.  Application A creates a subset
>> and wants to broadcast it to B and C.  It does this by creating a
>> temporary file and sending it to B and C.  B & C dutifully make their
>> own copies of it.   The user then starts application D, and shuts down
>> application A, which clears up its temporary file.  How does the user
>> then send the file from B to D and ensure that all three remaining apps
>> think they are referring to the same table?
>
> Actually, as per the trouble Richard had a short while ago, it's worse
> than that - TOPCAT at least doesn't wait until shutdown to clear up
> such temporary files, it does it as soon as the votable/loadFromURL
> request has returned.

Sorry to keep harking on, but I still think things would be better if the 
hub had more control.

If "A" registers an interesting file with the hub, the hub can maintain 
the host and file URL (a plasticFileID?) and send messages to third 
parties, counting how many are interested. "A" can unregister and as long 
as other apps are interested and registered -- ie the count remains above 
zero -- and only one temporary file need exist. This still leaves D and 
others who come along later in the dark, unless there is an option to 
request any 'hub-cached' files or unless say "B" specifically sends it. 
Either way, the count would increase by one for "D".

It might also be possible to automate updates of a given file ID, and the 
earlier file could be removed, when all other apps have the new version.
The 'plasticFileID' might (?) even be able to stay the same.

>
>> The solution is to adopt the same argument list as
>> ivo://.../votable/load and have a second argument specifying the ID.   I
>> suggest we make this second argument optional (but strongly recommended)
>> to avoid breaking existing apps, and if it's missing assume that the
>> id=url as we have now.
>

Doesn't "load" have a second arg anyway? This seems to be the main 
difference between two. Help, I'm still confused about this one..

> After thinking about it for a while, I agree, but subject to the
> following comment.  I seem to remember that the, or at least a
> reason we decided to use the URL for the table ID was that
> it meant if the table had some persistence (i.e. it was a non-temporary
> file in the filesystem, or something on a remote server) then two
> applications could know they were talking about the same thing even
> if they picked it up independently rather than one having got it from
> the other through PLASTIC.  So for a file which is (expected to be)
> persistent the URL is a good choice for the identifier.

This seems a bit dicey to me. Two files obtained from different sources at 
the same time (or vice versa) might not be the same ..??

>
> So an application which injects a table into the PLASTIC system by
> sending a loadFromURL should use the URL itself as the ID
> (or, equivalently, supply no ID and let apps assume id=url,
> though I'm happy for clarity's sake to recommend that both arguments
> are supplied).  In this way an application which happens to have
> loaded the same table from a non-PLASTIC source has a chance of
> knowing that fact.  It also solves the problem of how the sending
> application is supposed to generate an ID guaranteed not to
> clash with someone else's.  But if you've acquired a table by
> responding to a loadFromURL, then if you subsequently send a
> loadFromURL you should propagate the ID that came with it.
> This may or may not be the same as the URL you send.
>
>
>> Is anyone actually implementing ivo://.../votable/load?
>
> yes.
>
>> For symmetry's sake we could also make the id optional for this message,
>> and use the table name if it's missing, but I'd prefer not to given the
>> potential for lack of uniqueness.
>

Maybe it's because plastic is so new to me but I don't quite get the logic 
here. How many apps are going to break if you deprecate one message and 
introduce a few new ones (with clarification). Surely this is the time to 
make changes and revisions.


Richard