Re[2]: [Fwd: [SSI-users] moving data around]
Brought to you by:
brucewalker,
rogertsang
Menu
▾
▴
Re[2]: [Fwd: [SSI-users] moving data around]
|
From: Ron C. <ro...@de...> - 2004-07-03 13:30:05
|
Hi Bruce, thanks. for the info. I figured out later that my performance problem was because of one node not being in 100Mbs-FD even though it said it was. (I used a 3com utility to "hardcode" the card to autonegotiation instead of setting it to "100Mbs-FD". ) thanks, Ron >Ron, > If all the processes are writing into the same file (at different > offsets), that could explain a performance problem because they would > be thrashing the file (we don't have cache coherency at the page > level, only the total file level). If this is the case, try writing > to different files and have the master guy put it together as needed. > > bruce > -> >> Unfortunately, my CFS performance changes are in the development >> branch. I would say that a pipe is something to try. Another thing >> would be to have every node write to a local file and after everything >> is done the master just gathers up the data by reading each node's >> information. Finally, you could use standard shared memory IPC. >> Currently, you need to have a separate shared memory segment for each >> node. Otherwise, the performance will be really bad because we don't >> independently lock individual pages yet. >> >> David >> >> On Jul 2, 2004, at 3:44 PM, Brian J. Watson wrote: >> >> > Dave- >> > >> > Can you answer his question? I considered doing so, but I don't >> > understand your CFS implementation well enough to know exactly what's >> > killing his performance. >> > >> > A suggestion I considered is just using a pipe for each child to >> > report its calculations to the master, since I don't think the >> > children need to interact with each other. >> > >> > Thanks, >> > >> > Brian >> > >> > >> > -------- Original Message -------- >> > Subject: [SSI-users] moving data around >> > Date: Mon, 28 Jun 2004 08:59:52 CT >> > From: Ron Croonenberg <ro...@de...> >> > To: ssi...@li... >> > >> > Hello all, >> > >> > I am writing something that calculates a fractal on a 13 node cluster. >> > >> > I have everything working (something that executes programs on all the >> > slave >> > nodes that do the iterations and a program that runs on the main node >> > to >> > "collect" the data and display the picture.) >> > >> > I have all the slave nodes dump their data in a file. The calculations >> > seem to >> > be pretty quick however writing the data seems to be very slow. >> > >> > Is there a way to do things differently ? For example can I access an >> > allocated >> > chunk of memory on a different node ? >> > >> > thanks, >> > >> > Ron >> > >> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3 D >> > 1879: >> > Thomas Edison gets an idea, and his brother Timmy says, >> > "Hey, what's that thing over your head? >> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3 D >> > Ron Croonenberg | Phone: 1 765 658 4761 >> > Technology Coordinator | Fax: 1 765 658 4732 >> > | >> > Department of ComputerScience | e-mail : ronc@DePauw.edu >> > DePauw University | >> > Julian Science & Math Center | >> > 602 South College Ave. | >> > Greencastle, IN 46135 | >> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3 D >> > http://www.depauw.edu/acad/computer/RonCroonenberg.asp >> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3 D >> > >> > >> > ------------------------------------------------------- >> > This SF.Net email sponsored by Black Hat Briefings & Training. >> > Attend Black Hat Briefings & Training, Las Vegas July 24-29 - >> > digital self defense, top technical experts, no vendor pitches, >> > unmatched networking opportunities. Visit www.blackhat.com >> > _______________________________________________ >> > Ssic-linux-users mailing list >> > Ssi...@li... >> > https://lists.sourceforge.net/lists/listinfo/ssic-linux-users >> > >> > >> > >> > >> > >> David B. Zafman >> mailto:da...@za... >> If Bill Gates had a dime for every time a Windows box crashed... >> ..Oh, wait a minute, he already does. > > >------------------------------------------------------- >This SF.Net email sponsored by Black Hat Briefings & Training. >Attend Black Hat Briefings & Training, Las Vegas July 24-29 - >digital self defense, top technical experts, no vendor pitches, >unmatched networking opportunities. Visit www.blackhat.com >_______________________________________________ >Ssic-linux-users mailing list >Ssi...@li... >https://lists.sourceforge.net/lists/listinfo/ssic-linux-users > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 1879: Thomas Edison gets an idea, and his brother Timmy says, "Hey, what's that thing over your head? =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Ron Croonenberg | Phone: 1 765 658 4761 Technology Coordinator | Fax: 1 765 658 4732 | Department of ComputerScience | e-mail : ronc@DePauw.edu DePauw University | Julian Science & Math Center | 602 South College Ave. | Greencastle, IN 46135 | =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D http://www.depauw.edu/acad/computer/RonCroonenberg.asp =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D |