RE: [Planetlab-arch] Proposed Changes for Dynamic Slice API

[Timothy R. <tr...@in...>]

I think we have different definitions of "scalability".  I was going
on the following:

 "How well a solution to some problem work as the size of the
 problem increases."

Thus, systems scale linearly, superlinearly, or whatever in one or
more parameters, and improving the scalability of a system typically
involves changing some exponent of the system.

I think that the cost of distributing slices.xml scales linearly with
the number of nodes, with the number of slices, and with the number of
users.  As we expect (and hope) all these things to increase over
time, this is an O(n^3) scaling problem in the size of the state over
time. 

Compressing a (large) data file consequently doesn't improve
scalability at all - it simply reduces the problem by a constant
factor.   

Since all the nodes at a site download this file every 5 minutes, it
will begin to become a scaling problem.  For example, in the past
InfoSpect downloaded the Ganglia information every 5 minutes, and
eventually hit a scaling limit where the file was simply too big.

CoDeeN buys you nothing here, since even if one node per site talks to
the outside world, again you've only bought a (small) one-time
constant factor.  

CoDeen *might* save aggregate bandwidth at Princeton's access router,
but I doubt it's much.  Most users (me for instance) won't be using a
proxy when they check on their slice status every couple of minutes,
and will be going straight for PlanetLab central.  I suppose you could
try putting a transparent cache at the edge of your network.


Let's step back a bit here, and try to think calmly about the
problem.  PLC offers a service: to PlanetLab nodes which need to know
what slices they should be supporting, to PlanetLab slices which need
to query their current resource allocation, and to PlanetLab users who
need to create, update, query, and destroy slices.   The service is
basically access to a database through a set of business rules. 

Designing the interface to this is an architectural problem.  I have
not seen an architectural approach to this problem, and I would like
to - it's the motivation for some of my queries to this list, and this
is planetlab-arch after all).  Jeff's API is useful first step in this
direction, but distributing an xml file with CoDeeN is not. 

Am I the only person who is concerned here?

Is there an architect in the house?

 -- Mothy

At Tue, 27 Jan 2004 17:02:50 -0500 (EST), Steve Muir <sm...@CS...> wrote:
> my answer addressed two aspects of scalability:
>
> 1) relative scalability - i claim that providing a (compressed) static
> file is in general more scalable than supporting a wide range of database
> queries, particularly if we can use a CDN to distribute that file.  even
> though the database is optimised for supporting a range of queries there
> is a fairly significant overhead in redirecting every XML-RPC request from
> the HTTP(S) server to the PHP (or whatever) entity that performs the db
> query.
> 
> 2) increase in number of users - for a fixed number of slices and nodes, a
> static file can be served repeatedly to a large number of users without
> increasing the load on our server by using some CDN.
> 
> 
> > Thought experiment: suppose we increase the number of slices by a
> > factor of 10, the number nodes by a factor of 10, and the number of
> > users by a factor of 100.   Suppose that the file is downloaded
> > several times an hour by a significant fraction of those users
> > (because their automated status tools do it).   What is the dollar
> > cost in bandwidth to Princeton or the Consortium for this traffic?
> 
> i don't know how Princeton or the Consortium gets billed for traffic
> to/from, say, Codeen proxies, so i can't answer that question.
> 
> 
> > Another question: does XMLRPC have a gzippable transport, or are we
> > defining a new RPC protocol by compressing things?
> 
> i wasn't thinking of this as RPC, just file download.  if you're asking a
> more general question then i defer to david anderson's response.
> 
> steve
> 
> 
>