From: Luck, T. <ton...@in...> - 2002-08-27 23:56:00
|
The SGI IA-64 machine isn't a what-if ... it really has multiple blocks of memory on a mode, and the nodes are arranged in a hierarchical way (some nodes are closer together than other nodes). -Tony -----Original Message----- From: Niels Christiansen [mailto:nc...@ej...] Sent: Tuesday, August 27, 2002 4:42 PM To: lse-tech Subject: Re: [Lse-tech] [patch][rfc] Topology API v0.4 (1/2) Nah, not really! Those were what-ifs and percussions of other design decisions. Hierarchical nodes is an awkward way of handling hyper-threading. This Topology API was grafted on the root of wild rose but the graft doesn't seem to have taken. I think y'all need to go back to the garage and reinvent this (and the SGI cpumemsets, while you are at it). And I still think it should be syscall based (in addition to an interface through driverfs, which needs a major overhaul as well, incidentally). -nc- ----- Original Message ----- From: "Martin J. Bligh" <Mar...@us...> To: "Niels Christiansen" <nc...@ej...>; "lse-tech" <lse...@li...> Sent: Tuesday, August 27, 2002 6:13 PM Subject: Re: [Lse-tech] [patch][rfc] Topology API v0.4 (1/2) > > Michael, looks like you are proposing a tool with more capabilities than > > what is required for the job... Making a tool for some "conceivable > > requirement" is not what you used to advocate! Actually. memblks are way > > too abstract for my cup of tea (as apparently for the tea of others too). > > Ummm ... we just gave two specific examples of machines that don't have a > 1-1 mapping between memblocks and nodes. So those capabilities surely > are required? > > M. > > ------------------------------------------------------- This sf.net email is sponsored by: Jabber - The world's fastest growing real-time communications platform! Don't just IM. Build it in! http://www.jabber.com/osdn/xim _______________________________________________ Lse-tech mailing list Lse...@li... https://lists.sourceforge.net/lists/listinfo/lse-tech |
From: Gerrit H. <gh...@us...> - 2002-08-28 00:06:03
|
Do you have a practical coded example which works on several architectures which you are willing to counter-propose? Based on the architectures I've worked with before and those that the NUMA folks are using, this seems like it is small enough and general enough to cover the wide range of NUMA machines that are extant today. gerrit In message <016701c24e23$5941c6a0$fad...@ri...>, > : "Niels Christiansen " writes: > > Nah, not really! Those were what-ifs and percussions of other design > decisions. > > Hierarchical nodes is an awkward way of handling hyper-threading. This > Topology API was grafted on the root of wild rose but the graft doesn't seem > to have taken. I think y'all need to go back to the garage and reinvent > this (and the SGI cpumemsets, while you are at it). And I still think it > should be syscall based (in addition to an interface through driverfs, which > needs a major overhaul as well, incidentally). > > -nc- > > ----- Original Message ----- > From: "Martin J. Bligh" <Mar...@us...> > To: "Niels Christiansen" <nc...@ej...>; "lse-tech" > <lse...@li...> > Sent: Tuesday, August 27, 2002 6:13 PM > Subject: Re: [Lse-tech] [patch][rfc] Topology API v0.4 (1/2) > > > > > Michael, looks like you are proposing a tool with more capabilities than > > > what is required for the job... Making a tool for some "conceivable > > > requirement" is not what you used to advocate! Actually. memblks are > way > > > too abstract for my cup of tea (as apparently for the tea of others > too). > > > > Ummm ... we just gave two specific examples of machines that don't have a > > 1-1 mapping between memblocks and nodes. So those capabilities surely > > are required? > > > > M. > > > > > > > > ------------------------------------------------------- > This sf.net email is sponsored by: Jabber - The world's fastest growing > real-time communications platform! Don't just IM. Build it in! > http://www.jabber.com/osdn/xim > _______________________________________________ > Lse-tech mailing list > Lse...@li... > https://lists.sourceforge.net/lists/listinfo/lse-tech > > |
From: Niels C. <nc...@ej...> - 2002-08-28 03:49:40
|
Actually, I did, but proposing it got me sacked as you guys know. -nc- > Do you have a practical coded example which works on several > architectures which you are willing to counter-propose? Based > on the architectures I've worked with before and those that > the NUMA folks are using, this seems like it is small enough > and general enough to cover the wide range of NUMA machines > that are extant today. > > gerrit |
From: Paul M. <Pau...@us...> - 2002-08-28 15:14:26
|
It is quite true that one can represent NUMA properties without reference to hierarchy. The main reason for favoring a hierarchy is the electronics properties underlying Moore's Law. These properties have motivated hierarchical system architectures that take advantage of locality. The largish ratios between CPU and bus clocks, the multi-threaded CPUs, the multi-CPU cores, and the memory hierarchies are all evidence of this hierarchy. Perhaps someone will manage to come up with a killer architecture that is not basically hierarchical in structure, but the trends over the past couple of decades are most definitely in favor of hierarchy. So, it is not entirely unreasonable to reflect this hierarchy in the software structure, so that software can easily and efficiently take advantage of the performance benefits of maintaining good locality. You asked!!! ;-) Thanx, Paul > Tony, I did not mean to imply that your box doesn't exist. What I do mean > is that difference in distance does not necessarily need to be expressed as > a hierarchy in software. I do not know the details of your hardware so can > not tell if there are other reasons to look at the nodes as a hierarchy of > nodes (such as what you can disable/enable or what have you). If so, a > hierarchy makes sense, of course, but creating a hierarchy just to express > difference in latency doesn't. > > As for multiple memory blocks on a node, that is something I failed to see > in any of the designs I had access to. Multiple memory slots, of course, > but all of those were hidden from the software in the designs I have seen > (which were of IBM boxes). I am happy to learn that on the SGI machine, you > can actually see each memory block. I hope you can also relate it to memory > slots and that you have feasible ways of reconfiguring memory if somebody > throws a 1GB memory dimm in a slot that had a 512MB dimm before. With such > a design, memblks just may make sense but I have yet to see a practical > application (although SGI may have many such). > > Anyway, enough is enough. To all who responded to my comments only this: I > believe the designs you are integrating (Topology API, driverfs, cpumemsets, > ...) are each designed to do some isolated stuff and that the final product > would have benefited from those designs having been integrated from start > or -- as I said -- reinvented together. > > All of which you are free to ignore, as always. > > -nc- > > > The SGI IA-64 machine isn't a what-if ... it really has > > multiple blocks of memory on a mode, and the nodes are > > arranged in a hierarchical way (some nodes are closer > > together than other nodes). > > > > -Tony/listinfo/lse-tech |
From: Niels C. <nc...@ej...> - 2002-08-28 04:10:37
|
Tony, I did not mean to imply that your box doesn't exist. What I do mean is that difference in distance does not necessarily need to be expressed as a hierarchy in software. I do not know the details of your hardware so can not tell if there are other reasons to look at the nodes as a hierarchy of nodes (such as what you can disable/enable or what have you). If so, a hierarchy makes sense, of course, but creating a hierarchy just to express difference in latency doesn't. As for multiple memory blocks on a node, that is something I failed to see in any of the designs I had access to. Multiple memory slots, of course, but all of those were hidden from the software in the designs I have seen (which were of IBM boxes). I am happy to learn that on the SGI machine, you can actually see each memory block. I hope you can also relate it to memory slots and that you have feasible ways of reconfiguring memory if somebody throws a 1GB memory dimm in a slot that had a 512MB dimm before. With such a design, memblks just may make sense but I have yet to see a practical application (although SGI may have many such). Anyway, enough is enough. To all who responded to my comments only this: I believe the designs you are integrating (Topology API, driverfs, cpumemsets, ...) are each designed to do some isolated stuff and that the final product would have benefited from those designs having been integrated from start or -- as I said -- reinvented together. All of which you are free to ignore, as always. -nc- > The SGI IA-64 machine isn't a what-if ... it really has > multiple blocks of memory on a mode, and the nodes are > arranged in a hierarchical way (some nodes are closer > together than other nodes). > > -Tony/listinfo/lse-tech |