planetlab-arch Mailing List for PlanetLab
Brought to you by:
alklinga
Menu
▾
▴
planetlab-arch — Technical discussion of the design of PlanetLab
You can subscribe to this list here.
| 2002 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(2) |
Aug
(21) |
Sep
(8) |
Oct
(11) |
Nov
(34) |
Dec
(24) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2003 |
Jan
(2) |
Feb
(10) |
Mar
(1) |
Apr
(43) |
May
(31) |
Jun
(18) |
Jul
(2) |
Aug
(8) |
Sep
(44) |
Oct
(16) |
Nov
(27) |
Dec
(41) |
| 2004 |
Jan
(79) |
Feb
(3) |
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Scott K. <sc...@CS...> - 2004-02-11 03:09:43
|
Hi Mike, FYI, I have an evaluation unit from Baytech on order (2-3 week delivery) that is a 4-port power switch with SSH access. The URL is http://www.baytech.net/ and we are looking at the F4-RPC (base unit), F62 (Ethernet/SSH) module, and F74 (4 x RS-232) module. I'll let you know... As far as remote power for Internet2, keep in mind that the Abilene colo sites use -48VDC Telco power and may already have remote power control. Scott On Tue, Feb 10, 2004 at 04:04:27PM -0800, Mike Howard wrote: > On Tue, Jan 20, 2004 at 02:40:16PM -0600, Richard Murphy wrote: > > > > We obtained a model IP 420 so that our Planet Lab nodes could be > > remotely rebooted. However, the power strip software seems capable > > only of communication on the same subnet and it assumes the default > > netmask. Does anyone actually have one of these deployed in such > > a way that it can be used by Planet Lab over Internet2? Does anyone > > recommend a "smarter" product? Thanks. > > At Berkeley we bought some new APC Masterswitch PDU's. > > You can get the AP7900 for regular 110V outlets, but I prefer the > AP7920, which contrary to the USA APC webpage, is a dual voltage > 110V or 220V unit with international plugs. > > They're very nice, and even have an LCD readout of the electrical > load. > > Not only that, but they're cheaper than the DataProbe's. > > The biggest gripe is that they don't seem to have any secure remote > access methods (https or ssh). And sadly, it's impossible to enable > the ethernet interface without defining the default gateway... so > they're naked on the net. > > > -- > Mike Howard - mh...@cs... > UC Berkeley Computer Science Division > SysAdmin - Clustered Computing Team > (510)643-8046 - 505 Soda Hall |
|
From: Mike H. <mh...@cs...> - 2004-02-11 00:04:48
|
On Tue, Jan 20, 2004 at 02:40:16PM -0600, Richard Murphy wrote: >=20 > We obtained a model IP 420 so that our Planet Lab nodes could be=20 > remotely rebooted. However, the power strip software seems capable > only of communication on the same subnet and it assumes the default=20 > netmask. Does anyone actually have one of these deployed in such > a way that it can be used by Planet Lab over Internet2? Does anyone=20 > recommend a "smarter" product? Thanks. At Berkeley we bought some new APC Masterswitch PDU's. You can get the AP7900 for regular 110V outlets, but I prefer the AP7920, which contrary to the USA APC webpage, is a dual voltage 110V or 220V unit with international plugs. They're very nice, and even have an LCD readout of the electrical load. Not only that, but they're cheaper than the DataProbe's. The biggest gripe is that they don't seem to have any secure remote access methods (https or ssh). And sadly, it's impossible to enable the ethernet interface without defining the default gateway... so they're naked on the net. --=20 Mike Howard - mh...@cs... UC Berkeley Computer Science Division SysAdmin - Clustered Computing Team (510)643-8046 - 505 Soda Hall |
|
From: Scott K. <sc...@CS...> - 2004-02-03 01:32:27
|
sites.xml now contains site-ids and node-ids -- these are simply integers. Will this suit your needs? Scott On Mon, Jan 26, 2004 at 01:59:47PM -0800, Timothy Roscoe wrote: > > Since there seems to be a good discussion going, I thought I'd throw > some more petrol on the fire, as it were. > > Naming of slices and sites. > > PLC needs to name slices and sites. Other brokerage services > presumably will need to do something similar. How do we do it? > > Let's start with sites. Sites are needed because PlanetLab is > site-based - if you don't have a site you don't get any slices. Right > now we allocate short identifiers (gt, irb, etc.) to sites. If it > continues unchecked, this process will eventually lead to long, > expensive, bitter, and obnoxious copyright lawsuits in the future > (it's only a matter of time before the International Rugby Board want > to join PlanetLab and their lawyers start coming after us at > Berkeley). Options include: > > - stay with the current, arbitrary, short names. See above. > - DNS. irb -> berkeley.intel-research.net. Still acrimonious, but > at least we stay out of the immediate crossfire. > - something else? > > Slices: slice names can be scoped to the creating PI's institution, > but this doesn't alter the need to say what slice names are. > > - Readable slice names will presumably be bound "temporarily" to > unique-for-all-time identifiers for slices, which in our case we > have not got. If so, shouldn't these uids be usable with the RPC > API as well? > > - Do readable slice names get allocated by PIs at slice creation > time? > > Flame on... > > -- Mothy > > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Planetlab-arch mailing list > Pla...@li... > https://lists.sourceforge.net/lists/listinfo/planetlab-arch |
|
From: Timothy R. <tr...@in...> - 2004-01-27 23:07:38
|
I think we have different definitions of "scalability". I was going on the following: "How well a solution to some problem work as the size of the problem increases." Thus, systems scale linearly, superlinearly, or whatever in one or more parameters, and improving the scalability of a system typically involves changing some exponent of the system. I think that the cost of distributing slices.xml scales linearly with the number of nodes, with the number of slices, and with the number of users. As we expect (and hope) all these things to increase over time, this is an O(n^3) scaling problem in the size of the state over time. Compressing a (large) data file consequently doesn't improve scalability at all - it simply reduces the problem by a constant factor. Since all the nodes at a site download this file every 5 minutes, it will begin to become a scaling problem. For example, in the past InfoSpect downloaded the Ganglia information every 5 minutes, and eventually hit a scaling limit where the file was simply too big. CoDeeN buys you nothing here, since even if one node per site talks to the outside world, again you've only bought a (small) one-time constant factor. CoDeen *might* save aggregate bandwidth at Princeton's access router, but I doubt it's much. Most users (me for instance) won't be using a proxy when they check on their slice status every couple of minutes, and will be going straight for PlanetLab central. I suppose you could try putting a transparent cache at the edge of your network. Let's step back a bit here, and try to think calmly about the problem. PLC offers a service: to PlanetLab nodes which need to know what slices they should be supporting, to PlanetLab slices which need to query their current resource allocation, and to PlanetLab users who need to create, update, query, and destroy slices. The service is basically access to a database through a set of business rules. Designing the interface to this is an architectural problem. I have not seen an architectural approach to this problem, and I would like to - it's the motivation for some of my queries to this list, and this is planetlab-arch after all). Jeff's API is useful first step in this direction, but distributing an xml file with CoDeeN is not. Am I the only person who is concerned here? Is there an architect in the house? -- Mothy At Tue, 27 Jan 2004 17:02:50 -0500 (EST), Steve Muir <sm...@CS...> wrote: > my answer addressed two aspects of scalability: > > 1) relative scalability - i claim that providing a (compressed) static > file is in general more scalable than supporting a wide range of database > queries, particularly if we can use a CDN to distribute that file. even > though the database is optimised for supporting a range of queries there > is a fairly significant overhead in redirecting every XML-RPC request from > the HTTP(S) server to the PHP (or whatever) entity that performs the db > query. > > 2) increase in number of users - for a fixed number of slices and nodes, a > static file can be served repeatedly to a large number of users without > increasing the load on our server by using some CDN. > > > > Thought experiment: suppose we increase the number of slices by a > > factor of 10, the number nodes by a factor of 10, and the number of > > users by a factor of 100. Suppose that the file is downloaded > > several times an hour by a significant fraction of those users > > (because their automated status tools do it). What is the dollar > > cost in bandwidth to Princeton or the Consortium for this traffic? > > i don't know how Princeton or the Consortium gets billed for traffic > to/from, say, Codeen proxies, so i can't answer that question. > > > > Another question: does XMLRPC have a gzippable transport, or are we > > defining a new RPC protocol by compressing things? > > i wasn't thinking of this as RPC, just file download. if you're asking a > more general question then i defer to david anderson's response. > > steve > > > |
|
From: Bowman, M. <mic...@in...> - 2004-01-27 23:01:18
|
Just a comment on relative scalability... this discussion is very similar to one we had on ganglia about a year ago. Ganglia was transferring huge amounts of data (consistently one of the top five users of bw on planetlab) to PLC. To get around the problem, we started compressing the data that was sent from gmond to gmetad. It helped a lot.... initially. At 100 nodes, the reduction in bw was sufficient to keep ganglia running. However, by the time we got to 350 nodes, the amount of data, the amount of time required to process the data, the number of connections, and any number of other scalability concerns started to bite us hard. Point is that simply compressing the data helps us get through some of the bw concerns for now. But as the number of nodes & number of users increase, it will quickly fail if it is the only way we address scalability. The size of the file is the product of the increase in # of users and # of nodes which is multiplied again by the number of nodes that have to retrieve it.=20 BTW... on thought experiments... if we assume a planetlab with 10,000 nodes and 1000 slices, that's still a very small database. Simple implementations like an index array of <nodeid,sliceid> tuples could easily fit in about $50 worth of memory and response time for searching would be measured in milliseconds. Sorry if I'm rambling... --Mic=20 -----Original Message----- From: pla...@li... [mailto:pla...@li...] On Behalf Of Steve Muir Sent: Tuesday, January 27, 2004 02:03 PM To: Timothy Roscoe Cc: pla...@li... Subject: RE: [Planetlab-arch] Proposed Changes for Dynamic Slice API On Tue, 27 Jan 2004, Timothy Roscoe wrote: > > Quick response: your answer doesn't address scalability at all (which=20 > is about how the system grows), though it does point out that the=20 > current solution is adequate at the moment. my answer addressed two aspects of scalability: 1) relative scalability - i claim that providing a (compressed) static file is in general more scalable than supporting a wide range of database queries, particularly if we can use a CDN to distribute that file. even though the database is optimised for supporting a range of queries there is a fairly significant overhead in redirecting every XML-RPC request from the HTTP(S) server to the PHP (or whatever) entity that performs the db query. 2) increase in number of users - for a fixed number of slices and nodes, a static file can be served repeatedly to a large number of users without increasing the load on our server by using some CDN. > Thought experiment: suppose we increase the number of slices by a=20 > factor of 10, the number nodes by a factor of 10, and the number of > users by a factor of 100. Suppose that the file is downloaded > several times an hour by a significant fraction of those users > (because their automated status tools do it). What is the dollar > cost in bandwidth to Princeton or the Consortium for this traffic? i don't know how Princeton or the Consortium gets billed for traffic to/from, say, Codeen proxies, so i can't answer that question. > Another question: does XMLRPC have a gzippable transport, or are we=20 > defining a new RPC protocol by compressing things? i wasn't thinking of this as RPC, just file download. if you're asking a more general question then i defer to david anderson's response. steve ------------------------------------------------------- The SF.Net email is sponsored by EclipseCon 2004 Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. http://www.eclipsecon.org/osdn _______________________________________________ Planetlab-arch mailing list Pla...@li... https://lists.sourceforge.net/lists/listinfo/planetlab-arch |
|
From: Steve M. <sm...@CS...> - 2004-01-27 22:02:50
|
On Tue, 27 Jan 2004, Timothy Roscoe wrote: > > Quick response: your answer doesn't address scalability at all (which > is about how the system grows), though it does point out that the > current solution is adequate at the moment. my answer addressed two aspects of scalability: 1) relative scalability - i claim that providing a (compressed) static file is in general more scalable than supporting a wide range of database queries, particularly if we can use a CDN to distribute that file. even though the database is optimised for supporting a range of queries there is a fairly significant overhead in redirecting every XML-RPC request from the HTTP(S) server to the PHP (or whatever) entity that performs the db query. 2) increase in number of users - for a fixed number of slices and nodes, a static file can be served repeatedly to a large number of users without increasing the load on our server by using some CDN. > Thought experiment: suppose we increase the number of slices by a > factor of 10, the number nodes by a factor of 10, and the number of > users by a factor of 100. Suppose that the file is downloaded > several times an hour by a significant fraction of those users > (because their automated status tools do it). What is the dollar > cost in bandwidth to Princeton or the Consortium for this traffic? i don't know how Princeton or the Consortium gets billed for traffic to/from, say, Codeen proxies, so i can't answer that question. > Another question: does XMLRPC have a gzippable transport, or are we > defining a new RPC protocol by compressing things? i wasn't thinking of this as RPC, just file download. if you're asking a more general question then i defer to david anderson's response. steve |
|
From: David G. A. <dg...@lc...> - 2004-01-27 21:12:24
|
On Tue, Jan 27, 2004 at 01:00:08PM -0800, Timothy Roscoe scribed:
>
> Another question: does XMLRPC have a gzippable transport, or are we
> defining a new RPC protocol by compressing things?
XMLRPC uses HTTP. HTTP specifies content-encoding: gzip
and accept-encoding: gzip
mod_gzip supplies both dynamic and static encoding of
content to gzip-capable clients. i'm a big fan of it.
don't know if your clients send accept-encoding gzip,
but they should if they have any kind of feature bloat
at all. :)
-dave
--
work: dg...@lc... me: dg...@po...
MIT Laboratory for Computer Science http://www.angio.net/
|
|
From: Timothy R. <tr...@in...> - 2004-01-27 21:00:50
|
Quick response: your answer doesn't address scalability at all (which is about how the system grows), though it does point out that the current solution is adequate at the moment. Thought experiment: suppose we increase the number of slices by a factor of 10, the number nodes by a factor of 10, and the number of users by a factor of 100. Suppose that the file is downloaded several times an hour by a significant fraction of those users (because their automated status tools do it). What is the dollar cost in bandwidth to Princeton or the Consortium for this traffic? Another question: does XMLRPC have a gzippable transport, or are we defining a new RPC protocol by compressing things? -- Mothy At Tue, 27 Jan 2004 13:55:28 -0500 (EST), Steve Muir <sm...@CS...> wrote: > quick response to the scalability question: we found that the XML file is > very amenable to being gzipped - right now those sizes are 800k > uncompressed, 52k gzipped. an argument in favour of giving everyone that > XML repr of the db is that serving a 52k static file probably compares > very favourably (w.r.t. scalability) to any solution requring even a > modicum of computation on the server. and i'll preempt Vivek by saying > that it's even more scalable if we use something like Codeen[cap?] to > distribute it. but that argument doesn't necessarily trump whatever > security/privacy concerns may exist. > > steve > > > > On Tue, 27 Jan 2004, Timothy Roscoe wrote: > > > At Tue, 27 Jan 2004 13:28:21 -0500 (EST), Steve Muir > <sm...@CS...> wrote: > > > On Tue, 27 Jan 2004, Timothy Roscoe wrote: > > > i meant that the point i was making was an implementation point - clearly > > > the architecture of planetlab doesn't dictate that we export the entire > > > contents of the slice DB as an XML file (albeit in a somewhat NM-specific > > > format), that's just how it's implemented right now. > > > > Fair enough. > > > > > i'll generalise my point to the following: since there seem to be several > > > applications that require access to the slice DB we should consider if > > > the needs of those applications can be met with a common DB access or > > > export mechanism. for example: listSlice could just send you the same > XML > > > file that we export to Node Manager. that may not be ideal but it would > > > be reasonable. > > > > > > > - list of slices I've created > > > > - list of slices I'm authorized for > > > > - list of slices created by one of my PIs > > > > > > yes, those all seem like reasonable queries. isn't the last query > > > equivalent to all slices with a specific site prefix e.g., irb_? both > the > > > last two points can be answered by consulting the XML file - i'll use > that > > > as an argument for exporting the DB wholescale and letting you grep it > > > whichever way you like rather than trying to anticipate all the queries > > > you might perform and encapsulating them in an API. > > > > So I'd make a few additional points here: > > > > 1) I think "queries" is not the best way to think of these, even if > > they end up translating to db queries. I think it's better to > > view them in the way you suggest above, as different applications. > > > > 2) There is a security aspect here as well: we have yet to resolve > > whether we want the list of slices to be protected in some way, > > but in either case the issue is scoping the view of the database > > according to users. > > > > 3) There is a scalability aspect here independent of the centralized > > aspect of the current implementation. Returning a list of all > > slices obviously won't scale, but limiting the list to one of the > > three attributes about does scale quite nicely. > > > > 4) About the prefix issue: this opens the whole Pandora's box of > > naming issues w.r.t. sites and slices. I sent out an email to > > planetlab-arch yesterday, but sourceforge seems to have dropped > > it. I'll resend. > > > > > let me put a (perhaps) simple question out there: is there a reason that > > > jon q. user should not be able to see the entire list of slices, > including > > > node and user assocations and resource specifications? > > > > I think there may be security/privacy issues there, but I'm not > > going to contribute any points yet (!). But the scaling issue (3) is > > also important, particularly as we grow. Right now there are > > relatively few people exercising this interface, but if it suddenly > > became the de facto standard way to keep track of one's slice, we'd > > get a whole lot of traffic very quickly. It's certainly the way I > > envisage working in the future. > > > > -- Mothy > > > > > > ------------------------------------------------------- > > The SF.Net email is sponsored by EclipseCon 2004 > > Premiere Conference on Open Tools Development and Integration > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > http://www.eclipsecon.org/osdn > > _______________________________________________ > > Planetlab-arch mailing list > > Pla...@li... > > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > > > |
|
From: Steve M. <sm...@CS...> - 2004-01-27 18:55:28
|
quick response to the scalability question: we found that the XML file is very amenable to being gzipped - right now those sizes are 800k uncompressed, 52k gzipped. an argument in favour of giving everyone that XML repr of the db is that serving a 52k static file probably compares very favourably (w.r.t. scalability) to any solution requring even a modicum of computation on the server. and i'll preempt Vivek by saying that it's even more scalable if we use something like Codeen[cap?] to distribute it. but that argument doesn't necessarily trump whatever security/privacy concerns may exist. steve On Tue, 27 Jan 2004, Timothy Roscoe wrote: > At Tue, 27 Jan 2004 13:28:21 -0500 (EST), Steve Muir <sm...@CS...> wrote: > > On Tue, 27 Jan 2004, Timothy Roscoe wrote: > > i meant that the point i was making was an implementation point - clearly > > the architecture of planetlab doesn't dictate that we export the entire > > contents of the slice DB as an XML file (albeit in a somewhat NM-specific > > format), that's just how it's implemented right now. > > Fair enough. > > > i'll generalise my point to the following: since there seem to be several > > applications that require access to the slice DB we should consider if > > the needs of those applications can be met with a common DB access or > > export mechanism. for example: listSlice could just send you the same XML > > file that we export to Node Manager. that may not be ideal but it would > > be reasonable. > > > > > - list of slices I've created > > > - list of slices I'm authorized for > > > - list of slices created by one of my PIs > > > > yes, those all seem like reasonable queries. isn't the last query > > equivalent to all slices with a specific site prefix e.g., irb_? both the > > last two points can be answered by consulting the XML file - i'll use that > > as an argument for exporting the DB wholescale and letting you grep it > > whichever way you like rather than trying to anticipate all the queries > > you might perform and encapsulating them in an API. > > So I'd make a few additional points here: > > 1) I think "queries" is not the best way to think of these, even if > they end up translating to db queries. I think it's better to > view them in the way you suggest above, as different applications. > > 2) There is a security aspect here as well: we have yet to resolve > whether we want the list of slices to be protected in some way, > but in either case the issue is scoping the view of the database > according to users. > > 3) There is a scalability aspect here independent of the centralized > aspect of the current implementation. Returning a list of all > slices obviously won't scale, but limiting the list to one of the > three attributes about does scale quite nicely. > > 4) About the prefix issue: this opens the whole Pandora's box of > naming issues w.r.t. sites and slices. I sent out an email to > planetlab-arch yesterday, but sourceforge seems to have dropped > it. I'll resend. > > > let me put a (perhaps) simple question out there: is there a reason that > > jon q. user should not be able to see the entire list of slices, including > > node and user assocations and resource specifications? > > I think there may be security/privacy issues there, but I'm not > going to contribute any points yet (!). But the scaling issue (3) is > also important, particularly as we grow. Right now there are > relatively few people exercising this interface, but if it suddenly > became the de facto standard way to keep track of one's slice, we'd > get a whole lot of traffic very quickly. It's certainly the way I > envisage working in the future. > > -- Mothy > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Planetlab-arch mailing list > Pla...@li... > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > |
|
From: Bowman, M. <mic...@in...> - 2004-01-27 18:51:44
|
I vote for LDAP directory entries. :-) --Mic=20 -----Original Message----- From: pla...@li... [mailto:pla...@li...] On Behalf Of Timothy Roscoe Sent: Tuesday, January 27, 2004 10:44 AM To: pla...@li... Subject: [Planetlab-arch] Naming of parts [Resending] Since there seems to be a good discussion going, I thought I'd throw some more petrol on the fire, as it were. =20 Naming of slices and sites. =20 PLC needs to name slices and sites. Other brokerage services presumably will need to do something similar. How do we do it? =20 Let's start with sites. Sites are needed because PlanetLab is site-based - if you don't have a site you don't get any slices. Right now we allocate short identifiers (gt, irb, etc.) to sites. If it continues unchecked, this process will eventually lead to long, expensive, bitter, and obnoxious copyright lawsuits in the future (it's only a matter of time before the International Rugby Board want to join PlanetLab and their lawyers start coming after us at Berkeley). Options include: =20 - stay with the current, arbitrary, short names. See above. - DNS. irb -> berkeley.intel-research.net. Still acrimonious, but at least we stay out of the immediate crossfire. - something else? =20 Slices: slice names can be scoped to the creating PI's institution, but this doesn't alter the need to say what slice names are. =20 - Readable slice names will presumably be bound "temporarily" to unique-for-all-time identifiers for slices, which in our case we have not got. If so, shouldn't these uids be usable with the RPC API as well? =20 - Do readable slice names get allocated by PIs at slice creation time? =20 Flame on... =20 -- Mothy =20 =20 ------------------------------------------------------- The SF.Net email is sponsored by EclipseCon 2004 Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. http://www.eclipsecon.org/osdn _______________________________________________ Planetlab-arch mailing list Pla...@li... https://lists.sourceforge.net/lists/listinfo/planetlab-arch |
|
From: Timothy R. <tr...@in...> - 2004-01-27 18:44:51
|
[Resending] Since there seems to be a good discussion going, I thought I'd throw some more petrol on the fire, as it were. Naming of slices and sites. PLC needs to name slices and sites. Other brokerage services presumably will need to do something similar. How do we do it? Let's start with sites. Sites are needed because PlanetLab is site-based - if you don't have a site you don't get any slices. Right now we allocate short identifiers (gt, irb, etc.) to sites. If it continues unchecked, this process will eventually lead to long, expensive, bitter, and obnoxious copyright lawsuits in the future (it's only a matter of time before the International Rugby Board want to join PlanetLab and their lawyers start coming after us at Berkeley). Options include: - stay with the current, arbitrary, short names. See above. - DNS. irb -> berkeley.intel-research.net. Still acrimonious, but at least we stay out of the immediate crossfire. - something else? Slices: slice names can be scoped to the creating PI's institution, but this doesn't alter the need to say what slice names are. - Readable slice names will presumably be bound "temporarily" to unique-for-all-time identifiers for slices, which in our case we have not got. If so, shouldn't these uids be usable with the RPC API as well? - Do readable slice names get allocated by PIs at slice creation time? Flame on... -- Mothy |
|
From: Timothy R. <tr...@in...> - 2004-01-27 18:43:51
|
At Tue, 27 Jan 2004 13:28:21 -0500 (EST), Steve Muir <sm...@CS...> wrote:
> On Tue, 27 Jan 2004, Timothy Roscoe wrote:
> i meant that the point i was making was an implementation point - clearly
> the architecture of planetlab doesn't dictate that we export the entire
> contents of the slice DB as an XML file (albeit in a somewhat NM-specific
> format), that's just how it's implemented right now.
Fair enough.
> i'll generalise my point to the following: since there seem to be several
> applications that require access to the slice DB we should consider if
> the needs of those applications can be met with a common DB access or
> export mechanism. for example: listSlice could just send you the same XML
> file that we export to Node Manager. that may not be ideal but it would
> be reasonable.
>
> > - list of slices I've created
> > - list of slices I'm authorized for
> > - list of slices created by one of my PIs
>
> yes, those all seem like reasonable queries. isn't the last query
> equivalent to all slices with a specific site prefix e.g., irb_? both the
> last two points can be answered by consulting the XML file - i'll use that
> as an argument for exporting the DB wholescale and letting you grep it
> whichever way you like rather than trying to anticipate all the queries
> you might perform and encapsulating them in an API.
So I'd make a few additional points here:
1) I think "queries" is not the best way to think of these, even if
they end up translating to db queries. I think it's better to
view them in the way you suggest above, as different applications.
2) There is a security aspect here as well: we have yet to resolve
whether we want the list of slices to be protected in some way,
but in either case the issue is scoping the view of the database
according to users.
3) There is a scalability aspect here independent of the centralized
aspect of the current implementation. Returning a list of all
slices obviously won't scale, but limiting the list to one of the
three attributes about does scale quite nicely.
4) About the prefix issue: this opens the whole Pandora's box of
naming issues w.r.t. sites and slices. I sent out an email to
planetlab-arch yesterday, but sourceforge seems to have dropped
it. I'll resend.
> let me put a (perhaps) simple question out there: is there a reason that
> jon q. user should not be able to see the entire list of slices, including
> node and user assocations and resource specifications?
I think there may be security/privacy issues there, but I'm not
going to contribute any points yet (!). But the scaling issue (3) is
also important, particularly as we grow. Right now there are
relatively few people exercising this interface, but if it suddenly
became the de facto standard way to keep track of one's slice, we'd
get a whole lot of traffic very quickly. It's certainly the way I
envisage working in the future.
-- Mothy
|
|
From: Steve M. <sm...@CS...> - 2004-01-27 18:28:21
|
On Tue, 27 Jan 2004, Timothy Roscoe wrote: > > It's not purely an implementation point. Regexps aside, I would like > to know: i meant that the point i was making was an implementation point - clearly the architecture of planetlab doesn't dictate that we export the entire contents of the slice DB as an XML file (albeit in a somewhat NM-specific format), that's just how it's implemented right now. i'll generalise my point to the following: since there seem to be several applications that require access to the slice DB we should consider if the needs of those applications can be met with a common DB access or export mechanism. for example: listSlice could just send you the same XML file that we export to Node Manager. that may not be ideal but it would be reasonable. > - list of slices I've created > - list of slices I'm authorized for > - list of slices created by one of my PIs yes, those all seem like reasonable queries. isn't the last query equivalent to all slices with a specific site prefix e.g., irb_? both the last two points can be answered by consulting the XML file - i'll use that as an argument for exporting the DB wholescale and letting you grep it whichever way you like rather than trying to anticipate all the queries you might perform and encapsulating them in an API. let me put a (perhaps) simple question out there: is there a reason that jon q. user should not be able to see the entire list of slices, including node and user assocations and resource specifications? |
|
From: Timothy R. <tr...@in...> - 2004-01-27 18:12:39
|
It's not purely an implementation point. Regexps aside, I would like to know: - list of slices I've created - list of slices I'm authorized for - list of slices created by one of my PIs This information is currently not available through the programmatic API, and I think it should be. -- Mothy At Tue, 27 Jan 2004 13:01:30 -0500 (EST), Steve Muir <sm...@CS...> wrote: > notwithstanding the fact that this is an implementation point rather than > architecture, we shouldn't worry too much about exposing the list of all > slices to arbitrary users given that any old joe can suck it down as an > xml file. when we change how the DB is exported to Node Manager we can > start to think about whether we need to keep slice info compartmentalised. > > steve > > > > On Tue, 27 Jan 2004, Timothy Roscoe wrote: > > > > > Seconded (as long as the performance is OK). > > > > -- Mothy > > > > > > At Mon, 26 Jan 2004 15:41:50 -0800, "Adams, Robert" > <rob...@in...> wrote: > > > I thought the original philosophy was to not put query like operations > > > into the foundation APIs -- these operations are best handled by some > > > 'service' which sits on top. > > > > > > Rather than wedging regexps into the Dynamic Slice API, it seems better > > > to let them return what they return and to add filtering capability to > > > the command line invokable programs that interface to the API. > > > > > > The 'auth' capability should be added to the API since that is not > > > something that an external service can figure out and that we wish to > > > control. > > > > > > -- RA > > > > > > -----Original Message----- > > > From: pla...@li... > > > [mailto:pla...@li...] On Behalf Of Timothy > > > Roscoe > > > Sent: Monday, January 26, 2004 11:27 AM > > > To: pla...@li... > > > Subject: Re: [Planetlab-arch] Proposed Changes for Dynamic Slice API > > > > > > > > > At Mon, 26 Jan 2004 14:18:24 -0500, Vivek Pai <vi...@CS...> > > > wrote: > > > > Sedayao, Jeff wrote: > > > > > The parameters are optional parameters, so that scripts depending on > > > the > > > > current > > > > > behavior are not broken. The purpose of these changes is to simply > > > the > > > > writing > > > > > of scripts. For the listSlice options, we also get a tradeoff of > > > less > > > > network > > > > > bandwidth and less CPU on the querying machine for more CPU usage on > > > the > > > > > PlanetLab web server. > > > > > > > > Just my two cents, and I'll admit that the relevance may be remote, > > > but > > > > I would be very wary of anything that pushes work onto the server > > > side. > > > > In particular, when I was at iMimic and we were playing around with > > > some > > > > of the regexp libraries on Linux, performance ranged from mediocre to > > > > dismal. In particular, I recall that some libraries could only do > > > about > > > > 50-100 matches/second on a 1 GHz machine. > > > > > > > > In terms of bandwidth versus CPU, it would be interesting to see how > > > > compressible this data is using gzip. That way, you could compress the > > > > file, send it over, and have the client do the regexp processing. This > > > > way, whoever is consuming the resources gets to "pay" for it. > > > > > > > > Just a random opinion - feel free to ignore it. > > > > > > Good point. We should also look at how this tradeoff scales, > > > as the number of slices/PI, users/slice, and nodes/slice goes up. But > > > in general, the principle of offloading processing to clients if it > > > means no loss of security and the bandwidth is available is an > > > important one. > > > > > > -- Mothy > > > > > > > > > ------------------------------------------------------- > > > The SF.Net email is sponsored by EclipseCon 2004 > > > Premiere Conference on Open Tools Development and Integration > > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > > http://www.eclipsecon.org/osdn > > > _______________________________________________ > > > Planetlab-arch mailing list > > > Pla...@li... > > > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > > > > > > > > > > > > > ------------------------------------------------------- > > The SF.Net email is sponsored by EclipseCon 2004 > > Premiere Conference on Open Tools Development and Integration > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > http://www.eclipsecon.org/osdn > > _______________________________________________ > > Planetlab-arch mailing list > > Pla...@li... > > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > > > |
|
From: Steve M. <sm...@CS...> - 2004-01-27 18:01:30
|
notwithstanding the fact that this is an implementation point rather than architecture, we shouldn't worry too much about exposing the list of all slices to arbitrary users given that any old joe can suck it down as an xml file. when we change how the DB is exported to Node Manager we can start to think about whether we need to keep slice info compartmentalised. steve On Tue, 27 Jan 2004, Timothy Roscoe wrote: > > Seconded (as long as the performance is OK). > > -- Mothy > > > At Mon, 26 Jan 2004 15:41:50 -0800, "Adams, Robert" <rob...@in...> wrote: > > I thought the original philosophy was to not put query like operations > > into the foundation APIs -- these operations are best handled by some > > 'service' which sits on top. > > > > Rather than wedging regexps into the Dynamic Slice API, it seems better > > to let them return what they return and to add filtering capability to > > the command line invokable programs that interface to the API. > > > > The 'auth' capability should be added to the API since that is not > > something that an external service can figure out and that we wish to > > control. > > > > -- RA > > > > -----Original Message----- > > From: pla...@li... > > [mailto:pla...@li...] On Behalf Of Timothy > > Roscoe > > Sent: Monday, January 26, 2004 11:27 AM > > To: pla...@li... > > Subject: Re: [Planetlab-arch] Proposed Changes for Dynamic Slice API > > > > > > At Mon, 26 Jan 2004 14:18:24 -0500, Vivek Pai <vi...@CS...> > > wrote: > > > Sedayao, Jeff wrote: > > > > The parameters are optional parameters, so that scripts depending on > > the > > > current > > > > behavior are not broken. The purpose of these changes is to simply > > the > > > writing > > > > of scripts. For the listSlice options, we also get a tradeoff of > > less > > > network > > > > bandwidth and less CPU on the querying machine for more CPU usage on > > the > > > > PlanetLab web server. > > > > > > Just my two cents, and I'll admit that the relevance may be remote, > > but > > > I would be very wary of anything that pushes work onto the server > > side. > > > In particular, when I was at iMimic and we were playing around with > > some > > > of the regexp libraries on Linux, performance ranged from mediocre to > > > dismal. In particular, I recall that some libraries could only do > > about > > > 50-100 matches/second on a 1 GHz machine. > > > > > > In terms of bandwidth versus CPU, it would be interesting to see how > > > compressible this data is using gzip. That way, you could compress the > > > file, send it over, and have the client do the regexp processing. This > > > way, whoever is consuming the resources gets to "pay" for it. > > > > > > Just a random opinion - feel free to ignore it. > > > > Good point. We should also look at how this tradeoff scales, > > as the number of slices/PI, users/slice, and nodes/slice goes up. But > > in general, the principle of offloading processing to clients if it > > means no loss of security and the bandwidth is available is an > > important one. > > > > -- Mothy > > > > > > ------------------------------------------------------- > > The SF.Net email is sponsored by EclipseCon 2004 > > Premiere Conference on Open Tools Development and Integration > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > http://www.eclipsecon.org/osdn > > _______________________________________________ > > Planetlab-arch mailing list > > Pla...@li... > > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > > > > > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Planetlab-arch mailing list > Pla...@li... > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > |
|
From: Timothy R. <tr...@in...> - 2004-01-27 17:53:26
|
At Mon, 26 Jan 2004 13:57:12 -0500, Frank Dabek <fd...@MI...> wrote: > > > > > Of course, that doesn't address the short-term issue of programs > > hanging or crashing on PlanetLab nodes. I assume C or C++ users can > > statically link their binaries (I do, and it's pretty trouble-free), > > but it's not ideal. > > > > Static linking doesn't actually help in my case. The libc that gets > pulled in during the static link is the same "TLS" enabled libc that > issues the unimplemented system calls. At least that is what happens > when I build on a RH9 box... maybe building a static binary on an older > distribution of RedHat would work? Is this what you've done with > success? Good point - perhaps I've just been lucky (I grew out of using threads, and haven't yet resorted to BerkeleyDB or some of the other packages that tickle the problem). -- Mothy |
|
From: Timothy R. <tr...@in...> - 2004-01-27 17:44:22
|
So Google is your friend: http://www.w3.org/TR/NOTE-datetime defines a date format as a particular instance of ISO8601. Various subsets are allowed, but the full monty looks like: 2004-01-27T09:38:03.456-08:00 The timezone can instead by 'Z' (for UTC/GMT/Zulu). This is (reasonably) human readable, includes a precise notion of time zone, and allows a decimal fraction for seconds. It also avoids the problem of specifying the accuracy of the timestamp (microseconds? nanoseconds?) since it's just a fraction. Finally, it's a well-known XML standard (xsd:dateTime): http://books.xmlschemata/org/relaxng/ch17-77031.html I vote for this. It's also easy to parse into a time_t. -- Mothy At Mon, 26 Jan 2004 13:27:06 -0800, Timothy Roscoe <tr...@in...> wrote: > > Hang on a moment. > > Since we're already using XML-RPC (presumably because IIOP is too > standardized, XDR is too easy, and and ASN.1/BER is too compact), > SURELY the Web Services people have standardized an accurate date > format? > > Is there a web services expert on this list? Feel free to post > anonymously if you want to keep quiet about it. > > -- Mothy > > > At Mon, 26 Jan 2004 16:11:52 -0500 (EST), Steve Muir <sm...@CS...> > wrote: > > in what sense is it harder to debug? since the functions for manipulating > > epoch-relative values are all part of the standard C library i would > > expect them to be well debugged already. and why is this well-defined > > value less portable than some string for which we have to point people to > > a complex spec. for example, do you know how many spaces go between the > > month and date strings if the date string is only a single digit? > > > > yes, there will be programmer error when 123.456 gets interpreted as 456 > > microsecs, but there will also be a ton of programmers who get their > > regexps wrong when, say, the date or hour is a single digit. > > > > i'm assuming that there will always be a program between the output of the > > API and the end user (isn't that why it's called the programmatic API?), > > so i'd rather make it easy for the program to parse the input and use > > existing functions to output a value in whatever the format the user > > wants. if you have to output anything other than UTC values you're going > > to be converting to an internal epoch-relative value anyway. > > > > steve > > > > > > > > On Mon, 26 Jan 2004, Aaron Klingaman wrote: > > > > > You may be right about that, but my instinct tells me not to use a > > timestamp > > > like that. Harder to debug, a bit less portable than a generic string. > > It'll > > > cost a bit more to parse and ensure the timestamp is correct, but I think > > > it'd be worth it in the long run. > > > > > > Aaron > > > > > > On Monday 26 January 2004 03:47 pm, Steve Muir wrote: > > > > i don't think it makes using the API any more difficult on non-UNIX > > boxes. > > > > python, and i imagine perl and most other scripting languages, can > > > > manipulate epoch-relative values directly, as can any C program (the > > ctime > > > > function and its relatives are part of C89). or am i missing > something? > > > > > > > > steve > > > > > > > > On Mon, 26 Jan 2004, Aaron Klingaman wrote: > > > > > How much more difficult will it make using the api on non-linux type > > > > > machines, like a windows box? > > > > > > > > > > I would vote for a more readable string. > > > > > > > > > > ak > > > > > > > > > > On Monday 26 January 2004 03:35 pm, Steve Muir wrote: > > > > > > yes, i meant exactly what you wrote. i wrote 'UTC epoch' since i > > > > > > wasn't sure if the definition of Unix epoch specified that it was > > > > > > relative to UTC. > > > > > > > > > > > > we could presumably support sub-second resolution using floating > > point > > > > > > numbers (or fixed point if there are accuracy issues with floating > > > > > > point), but i haven't thought through whether that has > complications. > > > > > > > > > > > > i agree that this makes it less straightforward to check timestamps > > > > > > visually, but on the other hand i expect that it will eliminate a > > bunch > > > > > > of parsing errors; parsing an integer/float is much more 'uniform' > > than > > > > > > parsing a string like "Mon Jan 26 15:32:46 UTC 2004" in the sense > > that > > > > > > i don't have to deal with special cases like single-digit values. > > > > > > > > > > > > steve > > > > > > > > > > > > On Mon, 26 Jan 2004, Timothy Roscoe wrote: > > > > > > > At Mon, 26 Jan 2004 15:06:18 -0500 (EST), Steve Muir > > > > > > > > > > <sm...@CS...> wrote: > > > > > > > > how about seconds since UTC epoch? it's easiest to parse and > > meets > > > > > > > > all of your requirements. > > > > > > > > > > > > > > > > steve > > > > > > > > > > > > > > I don't think there is a UTC epoch. But I'm assuming you mean > the > > > > > > > Unix epoch, which is "seconds since 00:00:00 UTC, January 1, > 1970". > > > > > > > That works for me too, but is harder to debug. > > > > > > > > > > > > > > -- Mothy > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------- > > > > > > > The SF.Net email is sponsored by EclipseCon 2004 > > > > > > > Premiere Conference on Open Tools Development and Integration > > > > > > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > > > > > > http://www.eclipsecon.org/osdn > > > > > > > _______________________________________________ > > > > > > > Planetlab-arch mailing list > > > > > > > Pla...@li... > > > > > > > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > > > > > > > > > > ------------------------------------------------------- > > > > > > The SF.Net email is sponsored by EclipseCon 2004 > > > > > > Premiere Conference on Open Tools Development and Integration > > > > > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > > > > > http://www.eclipsecon.org/osdn > > > > > > _______________________________________________ > > > > > > Planetlab-arch mailing list > > > > > > Pla...@li... > > > > > > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > > > > > > ------------------------------------------------------- > > > > The SF.Net email is sponsored by EclipseCon 2004 > > > > Premiere Conference on Open Tools Development and Integration > > > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > > > http://www.eclipsecon.org/osdn > > > > _______________________________________________ > > > > Planetlab-arch mailing list > > > > Pla...@li... > > > > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > > > > > > > > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Planetlab-arch mailing list > Pla...@li... > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > |
|
From: Timothy R. <tr...@in...> - 2004-01-27 17:20:07
|
Seconded (as long as the performance is OK). -- Mothy At Mon, 26 Jan 2004 15:41:50 -0800, "Adams, Robert" <rob...@in...> wrote: > I thought the original philosophy was to not put query like operations > into the foundation APIs -- these operations are best handled by some > 'service' which sits on top. > > Rather than wedging regexps into the Dynamic Slice API, it seems better > to let them return what they return and to add filtering capability to > the command line invokable programs that interface to the API. > > The 'auth' capability should be added to the API since that is not > something that an external service can figure out and that we wish to > control. > > -- RA > > -----Original Message----- > From: pla...@li... > [mailto:pla...@li...] On Behalf Of Timothy > Roscoe > Sent: Monday, January 26, 2004 11:27 AM > To: pla...@li... > Subject: Re: [Planetlab-arch] Proposed Changes for Dynamic Slice API > > > At Mon, 26 Jan 2004 14:18:24 -0500, Vivek Pai <vi...@CS...> > wrote: > > Sedayao, Jeff wrote: > > > The parameters are optional parameters, so that scripts depending on > the > > current > > > behavior are not broken. The purpose of these changes is to simply > the > > writing > > > of scripts. For the listSlice options, we also get a tradeoff of > less > > network > > > bandwidth and less CPU on the querying machine for more CPU usage on > the > > > PlanetLab web server. > > > > Just my two cents, and I'll admit that the relevance may be remote, > but > > I would be very wary of anything that pushes work onto the server > side. > > In particular, when I was at iMimic and we were playing around with > some > > of the regexp libraries on Linux, performance ranged from mediocre to > > dismal. In particular, I recall that some libraries could only do > about > > 50-100 matches/second on a 1 GHz machine. > > > > In terms of bandwidth versus CPU, it would be interesting to see how > > compressible this data is using gzip. That way, you could compress the > > file, send it over, and have the client do the regexp processing. This > > way, whoever is consuming the resources gets to "pay" for it. > > > > Just a random opinion - feel free to ignore it. > > Good point. We should also look at how this tradeoff scales, > as the number of slices/PI, users/slice, and nodes/slice goes up. But > in general, the principle of offloading processing to clients if it > means no loss of security and the bandwidth is available is an > important one. > > -- Mothy > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Planetlab-arch mailing list > Pla...@li... > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > |
|
From: Adams, R. <rob...@in...> - 2004-01-26 23:42:09
|
I thought the original philosophy was to not put query like operations into the foundation APIs -- these operations are best handled by some 'service' which sits on top. Rather than wedging regexps into the Dynamic Slice API, it seems better to let them return what they return and to add filtering capability to the command line invokable programs that interface to the API. The 'auth' capability should be added to the API since that is not something that an external service can figure out and that we wish to control. -- RA -----Original Message----- From: pla...@li... [mailto:pla...@li...] On Behalf Of Timothy Roscoe Sent: Monday, January 26, 2004 11:27 AM To: pla...@li... Subject: Re: [Planetlab-arch] Proposed Changes for Dynamic Slice API At Mon, 26 Jan 2004 14:18:24 -0500, Vivek Pai <vi...@CS...> wrote: > Sedayao, Jeff wrote: > > The parameters are optional parameters, so that scripts depending on the > current > > behavior are not broken. The purpose of these changes is to simply the > writing > > of scripts. For the listSlice options, we also get a tradeoff of less > network > > bandwidth and less CPU on the querying machine for more CPU usage on the > > PlanetLab web server. >=20 > Just my two cents, and I'll admit that the relevance may be remote, but > I would be very wary of anything that pushes work onto the server side. > In particular, when I was at iMimic and we were playing around with some > of the regexp libraries on Linux, performance ranged from mediocre to > dismal. In particular, I recall that some libraries could only do about > 50-100 matches/second on a 1 GHz machine. >=20 > In terms of bandwidth versus CPU, it would be interesting to see how > compressible this data is using gzip. That way, you could compress the > file, send it over, and have the client do the regexp processing. This > way, whoever is consuming the resources gets to "pay" for it. >=20 > Just a random opinion - feel free to ignore it. Good point. We should also look at how this tradeoff scales, as the number of slices/PI, users/slice, and nodes/slice goes up. But in general, the principle of offloading processing to clients if it means no loss of security and the bandwidth is available is an important one.=20 -- Mothy ------------------------------------------------------- The SF.Net email is sponsored by EclipseCon 2004 Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. http://www.eclipsecon.org/osdn _______________________________________________ Planetlab-arch mailing list Pla...@li... https://lists.sourceforge.net/lists/listinfo/planetlab-arch |
|
From: Timothy R. <tr...@in...> - 2004-01-26 22:00:19
|
Since there seems to be a good discussion going, I thought I'd throw some more petrol on the fire, as it were. Naming of slices and sites. PLC needs to name slices and sites. Other brokerage services presumably will need to do something similar. How do we do it? Let's start with sites. Sites are needed because PlanetLab is site-based - if you don't have a site you don't get any slices. Right now we allocate short identifiers (gt, irb, etc.) to sites. If it continues unchecked, this process will eventually lead to long, expensive, bitter, and obnoxious copyright lawsuits in the future (it's only a matter of time before the International Rugby Board want to join PlanetLab and their lawyers start coming after us at Berkeley). Options include: - stay with the current, arbitrary, short names. See above. - DNS. irb -> berkeley.intel-research.net. Still acrimonious, but at least we stay out of the immediate crossfire. - something else? Slices: slice names can be scoped to the creating PI's institution, but this doesn't alter the need to say what slice names are. - Readable slice names will presumably be bound "temporarily" to unique-for-all-time identifiers for slices, which in our case we have not got. If so, shouldn't these uids be usable with the RPC API as well? - Do readable slice names get allocated by PIs at slice creation time? Flame on... -- Mothy |
|
From: Timothy R. <tr...@in...> - 2004-01-26 21:27:40
|
Hang on a moment. Since we're already using XML-RPC (presumably because IIOP is too standardized, XDR is too easy, and and ASN.1/BER is too compact), SURELY the Web Services people have standardized an accurate date format? Is there a web services expert on this list? Feel free to post anonymously if you want to keep quiet about it. -- Mothy At Mon, 26 Jan 2004 16:11:52 -0500 (EST), Steve Muir <sm...@CS...> wrote: > in what sense is it harder to debug? since the functions for manipulating > epoch-relative values are all part of the standard C library i would > expect them to be well debugged already. and why is this well-defined > value less portable than some string for which we have to point people to > a complex spec. for example, do you know how many spaces go between the > month and date strings if the date string is only a single digit? > > yes, there will be programmer error when 123.456 gets interpreted as 456 > microsecs, but there will also be a ton of programmers who get their > regexps wrong when, say, the date or hour is a single digit. > > i'm assuming that there will always be a program between the output of the > API and the end user (isn't that why it's called the programmatic API?), > so i'd rather make it easy for the program to parse the input and use > existing functions to output a value in whatever the format the user > wants. if you have to output anything other than UTC values you're going > to be converting to an internal epoch-relative value anyway. > > steve > > > > On Mon, 26 Jan 2004, Aaron Klingaman wrote: > > > You may be right about that, but my instinct tells me not to use a > timestamp > > like that. Harder to debug, a bit less portable than a generic string. > It'll > > cost a bit more to parse and ensure the timestamp is correct, but I think > > it'd be worth it in the long run. > > > > Aaron > > > > On Monday 26 January 2004 03:47 pm, Steve Muir wrote: > > > i don't think it makes using the API any more difficult on non-UNIX > boxes. > > > python, and i imagine perl and most other scripting languages, can > > > manipulate epoch-relative values directly, as can any C program (the > ctime > > > function and its relatives are part of C89). or am i missing something? > > > > > > steve > > > > > > On Mon, 26 Jan 2004, Aaron Klingaman wrote: > > > > How much more difficult will it make using the api on non-linux type > > > > machines, like a windows box? > > > > > > > > I would vote for a more readable string. > > > > > > > > ak > > > > > > > > On Monday 26 January 2004 03:35 pm, Steve Muir wrote: > > > > > yes, i meant exactly what you wrote. i wrote 'UTC epoch' since i > > > > > wasn't sure if the definition of Unix epoch specified that it was > > > > > relative to UTC. > > > > > > > > > > we could presumably support sub-second resolution using floating > point > > > > > numbers (or fixed point if there are accuracy issues with floating > > > > > point), but i haven't thought through whether that has complications. > > > > > > > > > > i agree that this makes it less straightforward to check timestamps > > > > > visually, but on the other hand i expect that it will eliminate a > bunch > > > > > of parsing errors; parsing an integer/float is much more 'uniform' > than > > > > > parsing a string like "Mon Jan 26 15:32:46 UTC 2004" in the sense > that > > > > > i don't have to deal with special cases like single-digit values. > > > > > > > > > > steve > > > > > > > > > > On Mon, 26 Jan 2004, Timothy Roscoe wrote: > > > > > > At Mon, 26 Jan 2004 15:06:18 -0500 (EST), Steve Muir > > > > > > > > <sm...@CS...> wrote: > > > > > > > how about seconds since UTC epoch? it's easiest to parse and > meets > > > > > > > all of your requirements. > > > > > > > > > > > > > > steve > > > > > > > > > > > > I don't think there is a UTC epoch. But I'm assuming you mean the > > > > > > Unix epoch, which is "seconds since 00:00:00 UTC, January 1, 1970". > > > > > > That works for me too, but is harder to debug. > > > > > > > > > > > > -- Mothy > > > > > > > > > > > > > > > > > > ------------------------------------------------------- > > > > > > The SF.Net email is sponsored by EclipseCon 2004 > > > > > > Premiere Conference on Open Tools Development and Integration > > > > > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > > > > > http://www.eclipsecon.org/osdn > > > > > > _______________________________________________ > > > > > > Planetlab-arch mailing list > > > > > > Pla...@li... > > > > > > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > > > > > > > > ------------------------------------------------------- > > > > > The SF.Net email is sponsored by EclipseCon 2004 > > > > > Premiere Conference on Open Tools Development and Integration > > > > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > > > > http://www.eclipsecon.org/osdn > > > > > _______________________________________________ > > > > > Planetlab-arch mailing list > > > > > Pla...@li... > > > > > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > > > > ------------------------------------------------------- > > > The SF.Net email is sponsored by EclipseCon 2004 > > > Premiere Conference on Open Tools Development and Integration > > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > > http://www.eclipsecon.org/osdn > > > _______________________________________________ > > > Planetlab-arch mailing list > > > Pla...@li... > > > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > > > |
|
From: Timothy R. <tr...@in...> - 2004-01-26 21:20:52
|
At Mon, 26 Jan 2004 16:11:52 -0500 (EST), Steve Muir <sm...@CS...> wrote: > in what sense is it harder to debug? since the functions for manipulating > epoch-relative values are all part of the standard C library i would > expect them to be well debugged already. and why is this well-defined > value less portable than some string for which we have to point people to > a complex spec. for example, do you know how many spaces go between the > month and date strings if the date string is only a single digit? I think the argument Aaron's making is similar to the reason that 'Date:' headers in HTTP and RFC2822 are textual rather than numeric - it does make things much easier, even though the explicit assumption in these standards is that the values will be hidden by a web browser or MHA. It does make a difference. Yes, there will be programmers who get their regexps wrong, but it'll be easier for them to debug. I speak from experience, having written a bunch of date parsing functions, and it would have been more painful without a set of human-readable input datasets. Parsing rfc2822 dates now is sufficiently widespread that everyone will just use the relevant library functions. Actually, there's even a somewhat cheeky argument that something that is (a) hard to parse, and (b) supported by library functions in all common programming languages, is better than a format that is more likely to tempt programmers to write their own (buggy) parser... Either way, I'm not terribly religious about this. I hope this won't develop into a flame ware... -- Mothy > yes, there will be programmer error when 123.456 gets interpreted as 456 > microsecs, but there will also be a ton of programmers who get their > regexps wrong when, say, the date or hour is a single digit. > > i'm assuming that there will always be a program between the output of the > API and the end user (isn't that why it's called the programmatic API?), > so i'd rather make it easy for the program to parse the input and use > existing functions to output a value in whatever the format the user > wants. if you have to output anything other than UTC values you're going > to be converting to an internal epoch-relative value anyway. > > steve > > > > On Mon, 26 Jan 2004, Aaron Klingaman wrote: > > > You may be right about that, but my instinct tells me not to use a > timestamp > > like that. Harder to debug, a bit less portable than a generic string. > It'll > > cost a bit more to parse and ensure the timestamp is correct, but I think > > it'd be worth it in the long run. > > > > Aaron > > > > On Monday 26 January 2004 03:47 pm, Steve Muir wrote: > > > i don't think it makes using the API any more difficult on non-UNIX > boxes. > > > python, and i imagine perl and most other scripting languages, can > > > manipulate epoch-relative values directly, as can any C program (the > ctime > > > function and its relatives are part of C89). or am i missing something? > > > > > > steve > > > > > > On Mon, 26 Jan 2004, Aaron Klingaman wrote: > > > > How much more difficult will it make using the api on non-linux type > > > > machines, like a windows box? > > > > > > > > I would vote for a more readable string. > > > > > > > > ak > > > > > > > > On Monday 26 January 2004 03:35 pm, Steve Muir wrote: > > > > > yes, i meant exactly what you wrote. i wrote 'UTC epoch' since i > > > > > wasn't sure if the definition of Unix epoch specified that it was > > > > > relative to UTC. > > > > > > > > > > we could presumably support sub-second resolution using floating > point > > > > > numbers (or fixed point if there are accuracy issues with floating > > > > > point), but i haven't thought through whether that has complications. > > > > > > > > > > i agree that this makes it less straightforward to check timestamps > > > > > visually, but on the other hand i expect that it will eliminate a > bunch > > > > > of parsing errors; parsing an integer/float is much more 'uniform' > than > > > > > parsing a string like "Mon Jan 26 15:32:46 UTC 2004" in the sense > that > > > > > i don't have to deal with special cases like single-digit values. > > > > > > > > > > steve > > > > > > > > > > On Mon, 26 Jan 2004, Timothy Roscoe wrote: > > > > > > At Mon, 26 Jan 2004 15:06:18 -0500 (EST), Steve Muir > > > > > > > > <sm...@CS...> wrote: > > > > > > > how about seconds since UTC epoch? it's easiest to parse and > meets > > > > > > > all of your requirements. > > > > > > > > > > > > > > steve > > > > > > > > > > > > I don't think there is a UTC epoch. But I'm assuming you mean the > > > > > > Unix epoch, which is "seconds since 00:00:00 UTC, January 1, 1970". > > > > > > That works for me too, but is harder to debug. > > > > > > > > > > > > -- Mothy > > > > > > > > > > > > > > > > > > ------------------------------------------------------- > > > > > > The SF.Net email is sponsored by EclipseCon 2004 > > > > > > Premiere Conference on Open Tools Development and Integration > > > > > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > > > > > http://www.eclipsecon.org/osdn > > > > > > _______________________________________________ > > > > > > Planetlab-arch mailing list > > > > > > Pla...@li... > > > > > > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > > > > > > > > ------------------------------------------------------- > > > > > The SF.Net email is sponsored by EclipseCon 2004 > > > > > Premiere Conference on Open Tools Development and Integration > > > > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > > > > http://www.eclipsecon.org/osdn > > > > > _______________________________________________ > > > > > Planetlab-arch mailing list > > > > > Pla...@li... > > > > > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > > > > ------------------------------------------------------- > > > The SF.Net email is sponsored by EclipseCon 2004 > > > Premiere Conference on Open Tools Development and Integration > > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > > http://www.eclipsecon.org/osdn > > > _______________________________________________ > > > Planetlab-arch mailing list > > > Pla...@li... > > > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > > > |
|
From: Steve M. <sm...@CS...> - 2004-01-26 21:11:52
|
in what sense is it harder to debug? since the functions for manipulating epoch-relative values are all part of the standard C library i would expect them to be well debugged already. and why is this well-defined value less portable than some string for which we have to point people to a complex spec. for example, do you know how many spaces go between the month and date strings if the date string is only a single digit? yes, there will be programmer error when 123.456 gets interpreted as 456 microsecs, but there will also be a ton of programmers who get their regexps wrong when, say, the date or hour is a single digit. i'm assuming that there will always be a program between the output of the API and the end user (isn't that why it's called the programmatic API?), so i'd rather make it easy for the program to parse the input and use existing functions to output a value in whatever the format the user wants. if you have to output anything other than UTC values you're going to be converting to an internal epoch-relative value anyway. steve On Mon, 26 Jan 2004, Aaron Klingaman wrote: > You may be right about that, but my instinct tells me not to use a timestamp > like that. Harder to debug, a bit less portable than a generic string. It'll > cost a bit more to parse and ensure the timestamp is correct, but I think > it'd be worth it in the long run. > > Aaron > > On Monday 26 January 2004 03:47 pm, Steve Muir wrote: > > i don't think it makes using the API any more difficult on non-UNIX boxes. > > python, and i imagine perl and most other scripting languages, can > > manipulate epoch-relative values directly, as can any C program (the ctime > > function and its relatives are part of C89). or am i missing something? > > > > steve > > > > On Mon, 26 Jan 2004, Aaron Klingaman wrote: > > > How much more difficult will it make using the api on non-linux type > > > machines, like a windows box? > > > > > > I would vote for a more readable string. > > > > > > ak > > > > > > On Monday 26 January 2004 03:35 pm, Steve Muir wrote: > > > > yes, i meant exactly what you wrote. i wrote 'UTC epoch' since i > > > > wasn't sure if the definition of Unix epoch specified that it was > > > > relative to UTC. > > > > > > > > we could presumably support sub-second resolution using floating point > > > > numbers (or fixed point if there are accuracy issues with floating > > > > point), but i haven't thought through whether that has complications. > > > > > > > > i agree that this makes it less straightforward to check timestamps > > > > visually, but on the other hand i expect that it will eliminate a bunch > > > > of parsing errors; parsing an integer/float is much more 'uniform' than > > > > parsing a string like "Mon Jan 26 15:32:46 UTC 2004" in the sense that > > > > i don't have to deal with special cases like single-digit values. > > > > > > > > steve > > > > > > > > On Mon, 26 Jan 2004, Timothy Roscoe wrote: > > > > > At Mon, 26 Jan 2004 15:06:18 -0500 (EST), Steve Muir > > > > > > <sm...@CS...> wrote: > > > > > > how about seconds since UTC epoch? it's easiest to parse and meets > > > > > > all of your requirements. > > > > > > > > > > > > steve > > > > > > > > > > I don't think there is a UTC epoch. But I'm assuming you mean the > > > > > Unix epoch, which is "seconds since 00:00:00 UTC, January 1, 1970". > > > > > That works for me too, but is harder to debug. > > > > > > > > > > -- Mothy > > > > > > > > > > > > > > > ------------------------------------------------------- > > > > > The SF.Net email is sponsored by EclipseCon 2004 > > > > > Premiere Conference on Open Tools Development and Integration > > > > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > > > > http://www.eclipsecon.org/osdn > > > > > _______________________________________________ > > > > > Planetlab-arch mailing list > > > > > Pla...@li... > > > > > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > > > > > > ------------------------------------------------------- > > > > The SF.Net email is sponsored by EclipseCon 2004 > > > > Premiere Conference on Open Tools Development and Integration > > > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > > > http://www.eclipsecon.org/osdn > > > > _______________________________________________ > > > > Planetlab-arch mailing list > > > > Pla...@li... > > > > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > > ------------------------------------------------------- > > The SF.Net email is sponsored by EclipseCon 2004 > > Premiere Conference on Open Tools Development and Integration > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > http://www.eclipsecon.org/osdn > > _______________________________________________ > > Planetlab-arch mailing list > > Pla...@li... > > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > |
|
From: Timothy R. <tr...@in...> - 2004-01-26 21:03:20
|
At Mon, 26 Jan 2004 12:51:16 -0800, "David L. Oppenheimer" <dav...@cs...> wrote: > I was assuming the value could have two separate pieces (as in the timeval). > There are advantages and disadvantages to doing this compared to (for > example) a single floating point number. Not least ambiguity - while it's well-specified, I've seen a number of programmer errors where people have assumed that 123456.1 refers to either 123456.1 seconds, OR 123456.000001 seconds, OR 123456.000000000001 seconds, depending on the format, and got it wrong. I'm in favour of human-readable representations, but only if there's a nice standard. If we're going to break a standard, or such things don't exist, then I vote for a single floating point value represented in ASCII as the number of seconds from the epoch. This also happens to be Python's default representation. -- Mothy > > > -----Original Message----- > > From: pla...@li... > > [mailto:pla...@li...] On Behalf > > Of David L. Oppenheimer > > Sent: Monday, January 26, 2004 12:41 PM > > To: 'Steve Muir' > > Cc: pla...@li... > > Subject: RE: [Planetlab-arch] Proposed Changes for Dynamic Slice API > > > > How about just using number of microseconds for subsecond > > resolution, as is > > done in the Unix timeval struct? > > > > David > > > > > -----Original Message----- > > > From: pla...@li... > > > [mailto:pla...@li...] On Behalf > > > Of Steve Muir > > > Sent: Monday, January 26, 2004 12:35 PM > > > To: Timothy Roscoe > > > Cc: pla...@li... > > > Subject: Re: [Planetlab-arch] Proposed Changes for Dynamic Slice API > > > > > > yes, i meant exactly what you wrote. i wrote 'UTC epoch' > > > since i wasn't > > > sure if the definition of Unix epoch specified that it was > > relative to > > > UTC. > > > > > > we could presumably support sub-second resolution using > > floating point > > > numbers (or fixed point if there are accuracy issues with > > > floating point), > > > but i haven't thought through whether that has complications. > > > > > > i agree that this makes it less straightforward to check timestamps > > > visually, but on the other hand i expect that it will > > > eliminate a bunch of > > > parsing errors; parsing an integer/float is much more 'uniform' than > > > parsing a string like "Mon Jan 26 15:32:46 UTC 2004" in the > > > sense that i > > > don't have to deal with special cases like single-digit values. > > > > > > steve > > > > > > > > > > > > On Mon, 26 Jan 2004, Timothy Roscoe wrote: > > > > > > > At Mon, 26 Jan 2004 15:06:18 -0500 (EST), Steve Muir > > > <sm...@CS...> wrote: > > > > > how about seconds since UTC epoch? it's easiest to parse > > > and meets > > > > > all of your requirements. > > > > > > > > > > steve > > > > > > > > I don't think there is a UTC epoch. But I'm assuming you mean the > > > > Unix epoch, which is "seconds since 00:00:00 UTC, January > > 1, 1970". > > > > That works for me too, but is harder to debug. > > > > > > > > -- Mothy > > > > > > > > > > > > ------------------------------------------------------- > > > > The SF.Net email is sponsored by EclipseCon 2004 > > > > Premiere Conference on Open Tools Development and Integration > > > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > > > http://www.eclipsecon.org/osdn > > > > _______________________________________________ > > > > Planetlab-arch mailing list > > > > Pla...@li... > > > > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > > > > > > > > > > > ------------------------------------------------------- > > > The SF.Net email is sponsored by EclipseCon 2004 > > > Premiere Conference on Open Tools Development and Integration > > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > > http://www.eclipsecon.org/osdn > > > _______________________________________________ > > > Planetlab-arch mailing list > > > Pla...@li... > > > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > > > > > > > > > ------------------------------------------------------- > > The SF.Net email is sponsored by EclipseCon 2004 > > Premiere Conference on Open Tools Development and Integration > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > http://www.eclipsecon.org/osdn > > _______________________________________________ > > Planetlab-arch mailing list > > Pla...@li... > > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Planetlab-arch mailing list > Pla...@li... > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > |
|
From: Aaron K. <al...@CS...> - 2004-01-26 21:01:00
|
You may be right about that, but my instinct tells me not to use a timestamp like that. Harder to debug, a bit less portable than a generic string. It'll cost a bit more to parse and ensure the timestamp is correct, but I think it'd be worth it in the long run. Aaron On Monday 26 January 2004 03:47 pm, Steve Muir wrote: > i don't think it makes using the API any more difficult on non-UNIX boxes. > python, and i imagine perl and most other scripting languages, can > manipulate epoch-relative values directly, as can any C program (the ctime > function and its relatives are part of C89). or am i missing something? > > steve > > On Mon, 26 Jan 2004, Aaron Klingaman wrote: > > How much more difficult will it make using the api on non-linux type > > machines, like a windows box? > > > > I would vote for a more readable string. > > > > ak > > > > On Monday 26 January 2004 03:35 pm, Steve Muir wrote: > > > yes, i meant exactly what you wrote. i wrote 'UTC epoch' since i > > > wasn't sure if the definition of Unix epoch specified that it was > > > relative to UTC. > > > > > > we could presumably support sub-second resolution using floating point > > > numbers (or fixed point if there are accuracy issues with floating > > > point), but i haven't thought through whether that has complications. > > > > > > i agree that this makes it less straightforward to check timestamps > > > visually, but on the other hand i expect that it will eliminate a bunch > > > of parsing errors; parsing an integer/float is much more 'uniform' than > > > parsing a string like "Mon Jan 26 15:32:46 UTC 2004" in the sense that > > > i don't have to deal with special cases like single-digit values. > > > > > > steve > > > > > > On Mon, 26 Jan 2004, Timothy Roscoe wrote: > > > > At Mon, 26 Jan 2004 15:06:18 -0500 (EST), Steve Muir > > > > <sm...@CS...> wrote: > > > > > how about seconds since UTC epoch? it's easiest to parse and meets > > > > > all of your requirements. > > > > > > > > > > steve > > > > > > > > I don't think there is a UTC epoch. But I'm assuming you mean the > > > > Unix epoch, which is "seconds since 00:00:00 UTC, January 1, 1970". > > > > That works for me too, but is harder to debug. > > > > > > > > -- Mothy > > > > > > > > > > > > ------------------------------------------------------- > > > > The SF.Net email is sponsored by EclipseCon 2004 > > > > Premiere Conference on Open Tools Development and Integration > > > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > > > http://www.eclipsecon.org/osdn > > > > _______________________________________________ > > > > Planetlab-arch mailing list > > > > Pla...@li... > > > > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > > > > > ------------------------------------------------------- > > > The SF.Net email is sponsored by EclipseCon 2004 > > > Premiere Conference on Open Tools Development and Integration > > > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > > > http://www.eclipsecon.org/osdn > > > _______________________________________________ > > > Planetlab-arch mailing list > > > Pla...@li... > > > https://lists.sourceforge.net/lists/listinfo/planetlab-arch > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Planetlab-arch mailing list > Pla...@li... > https://lists.sourceforge.net/lists/listinfo/planetlab-arch |
6 messages has been excluded from this view by a project administrator.
