RE: [Planetlab-arch] Proposed Changes for Dynamic Slice API

[Timothy R. <tr...@in...>]

Quick response: your answer doesn't address scalability at all (which
is about how the system grows), though it does point out that the
current solution is adequate at the moment.

Thought experiment: suppose we increase the number of slices by a
factor of 10, the number nodes by a factor of 10, and the number of
users by a factor of 100.   Suppose that the file is downloaded
several times an hour by a significant fraction of those users
(because their automated status tools do it).   What is the dollar
cost in bandwidth to Princeton or the Consortium for this traffic? 

Another question: does XMLRPC have a gzippable transport, or are we
defining a new RPC protocol by compressing things?

 -- Mothy


At Tue, 27 Jan 2004 13:55:28 -0500 (EST), Steve Muir <sm...@CS...> wrote:
> quick response to the scalability question: we found that the XML file is
> very amenable to being gzipped - right now those sizes are 800k
> uncompressed, 52k gzipped.  an argument in favour of giving everyone that
> XML repr of the db is that serving a 52k static file probably compares
> very favourably (w.r.t. scalability) to any solution requring even a
> modicum of computation on the server.  and i'll preempt Vivek by saying
> that it's even more scalable if we use something like Codeen[cap?] to
> distribute it.  but that argument doesn't necessarily trump whatever
> security/privacy concerns may exist.
> 
> steve
> 
> 
> 
> On Tue, 27 Jan 2004, Timothy Roscoe wrote:
> 
> > At Tue, 27 Jan 2004 13:28:21 -0500 (EST), Steve Muir
> <sm...@CS...> wrote:
> > > On Tue, 27 Jan 2004, Timothy Roscoe wrote:
> > > i meant that the point i was making was an implementation point - clearly
> > > the architecture of planetlab doesn't dictate that we export the entire
> > > contents of the slice DB as an XML file (albeit in a somewhat NM-specific
> > > format), that's just how it's implemented right now.
> >
> > Fair enough.
> >
> > > i'll generalise my point to the following: since there seem to be several
> > > applications that require access to the slice DB we should consider if
> > > the needs of those applications can be met with a common DB access or
> > > export mechanism.  for example: listSlice could just send you the same
> XML
> > > file that we export to Node Manager.  that may not be ideal but it would
> > > be reasonable.
> > >
> > > >  - list of slices I've created
> > > >  - list of slices I'm authorized for
> > > >  - list of slices created by one of my PIs
> > >
> > > yes, those all seem like reasonable queries.  isn't the last query
> > > equivalent to all slices with a specific site prefix e.g., irb_?  both
> the
> > > last two points can be answered by consulting the XML file - i'll use
> that
> > > as an argument for exporting the DB wholescale and letting you grep it
> > > whichever way you like rather than trying to anticipate all the queries
> > > you might perform and encapsulating them in an API.
> >
> > So I'd make a few additional points here:
> >
> >  1) I think "queries" is not the best way to think of these, even if
> >     they end  up translating  to db queries.   I think it's  better to
> >     view them in the way you suggest above, as different applications.
> >
> >  2) There is a security aspect here as well: we have yet to resolve
> >     whether we want the list of slices to be protected in some way,
> >     but in either case the issue is scoping the view of the database
> >     according to users.
> >
> >  3) There is a scalability aspect here independent of the centralized
> >     aspect of the current implementation.   Returning a list of all
> >     slices obviously won't scale, but limiting the list to one of the
> >     three attributes about does scale quite nicely.
> >
> >  4) About the prefix issue: this opens the whole Pandora's box of
> >     naming issues w.r.t. sites and slices.   I sent out an email to
> >     planetlab-arch yesterday, but sourceforge seems to have dropped
> >     it.  I'll resend.
> >
> > > let me put a (perhaps) simple question out there: is there a reason that
> > > jon q. user should not be able to see the entire list of slices,
> including
> > > node and user assocations and resource specifications?
> >
> > I think there may be security/privacy issues there, but I'm not
> > going to contribute any points yet (!).   But the scaling issue (3) is
> > also important, particularly as we grow.  Right now there are
> > relatively few people exercising this interface, but if it suddenly
> > became the de facto standard way to keep track of one's slice, we'd
> > get a whole lot of traffic very quickly.   It's certainly the way I
> > envisage working in the future.
> >
> >  -- Mothy
> >
> >
> > -------------------------------------------------------
> > The SF.Net email is sponsored by EclipseCon 2004
> > Premiere Conference on Open Tools Development and Integration
> > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
> > http://www.eclipsecon.org/osdn
> > _______________________________________________
> > Planetlab-arch mailing list
> > Pla...@li...
> > https://lists.sourceforge.net/lists/listinfo/planetlab-arch
> >
> 
> 
>