chromium-dev — For developers of the Chromium framework or SPU's

[Chromium-dev] ADDRINFO use [Was: Windows breakage with cr_net.h]

[Samuel T. <sam...@en...>]

Hi

Le lun 21 jun 2004 =E0 17:23:28 -0700, Mike Houston a tapot=E9 sur son cl=
avier :
> The other oddity is that the changes to support IPv6 seem to always kic=
k=20
> us into using ADDRINFO on recent versions of Linux.  This works,

Of course it does, to support IPv6 whenever available. getaddrinfo()'s
interface was specially design to always work: the dns lookup may
report ipv4 and ipv6 addresses, getaddrinfo() gives them all. If an
ipv6 address is given but doesn't work either because lack of support
in the kernel (ENOAFSUPPORT) or for network reasons, the connection
will revert back to ipv4.

> but moves all streams to PF instead of AF.

PF / AF ??
That means Protocol Family / Address Family.

There's actually no difference between them, except maybe some minor
differences in the numbering (there is none under linux for instance),
and the *good* one to use for getaddrinfo in the hints structure *is*
PF_something, since we tell the Protocol Family we'd like to use for
conencting (here, unspecified). Then getaddrinfo may report
AF_something to tell which address family it found. I just checked
this once again in Gis=E8le Cizault's book "IPv6 Th=E9orie et pratique"

> Things work, I just want to point out that this probably shouldn't
> be default behavior.  We probably only really want to kick across
> to PF streams when actually using IPv6 to connect.

Currently, we actually *precisely* use PF_INET when not using IPv6 to
connect.

Regards,
Samuel Thibault

[Chromium-dev] Windows breakage with cr_net.h

[Mike H. <mho...@gr...>]

I noticed that we are trying to include sys/socket.h on ALL platforms.  
We shouldn't be doing this under Windows.  I'll be checking in a fix 
shortly.

The other oddity is that the changes to support IPv6 seem to always kick 
us into using ADDRINFO on recent versions of Linux.  This works, but 
moves all streams to PF instead of AF.  Things work, I just want to 
point out that this probably shouldn't be default behavior.  We probably 
only really want to kick across to PF streams when actually using IPv6 
to connect.

-Mike

Re: [Chromium-dev] The tcpip layer is really frustrating me

[Mike H. <mho...@gr...>]

The main issue here is time.  I'll be at ATI this summer, so I don't 
have the time to deal with a network layer rewrite just now.

The main issue with the tcpip layer is how we handle mothership 
connections.  We really should use special calls for mothership 
connections so that we don't pollute the rest of the network layer.  The 
mothership connections do not stay live, but the tcpip layer still 
checks them in a tight loop.  This isn't a huge hit, but is does have an 
impact on other layers.

At the moment, I have some really hacky code that forces the tcpip layer 
to bypass if the only connections are mothership connections.  This 
helps to speed things up a little.  However, I quickly become CPU bound 
in the packer/tilesorter on our 2.4GHz P4 Xeons.  SDP over IP is ~5X 
GigE, but running through the packer (crdemo.conf), I only see ~2X 
performance increase.  48% of the time is spent in the SDP layer, almost 
all of it doing actual data transfer, and 52% is spent in the packer, a 
good chuck of that in crPackCanHoldOpcode, which gets hit alot.  Through 
tilesort, we are REALLY CPU bound, and I see an even smaller performance 
gain over GigE.

Switching to non-blocking sends and/or threading the network layer is 
needed to better make use of highspeed layers.  If we can overlap the 
packing/sort and the network send(s), we should be able to almost double 
the performance of Chromium, at least over highspeed interconnects like 
IB/Quadrics/Myrinet.  Obviously, this is a non-trivial exercise.  It's 
pretty easy to setup non-blocking sends in the tcpip and sdp layers 
using pthreads, but it's difficult to convert spus to make good use of it.

-Mike

Alan Hourihane wrote:

>On Wed, Jun 16, 2004 at 02:57:06PM -0700, Mike Houston wrote:
>  
>
>>I have SDP tuned to run ~600MB/s through the netperf programs in 
>>Chromium (npserver/npclient), as long as you don't send messages larger 
>>than the MTU.  This is excellent news overall.
>>
>>However, I only run 10% faster than GigE (105MB/s) because I spend lots 
>>time in crTCPIPRecv even though there is ZERO traffic over tcp.  Hell, 
>>the only connections are mothership connections.  We are spending most 
>>of the time in the giant block around line 782 in tpcip.c which 
>>literally has KLUDGE written all over it.  We really need to be able to 
>>bail out faster.  Any thoughts/ideas/suggestions before I start ripping 
>>code out?
>>    
>>
>
>I wrote that code to bypass problems in a multithreaded environment. And
>as the comments explain exactly what's going on, it's easier to be able
>to rework things.
>
>I certainly agree with Brian, that any work around this should be done
>on a branch and thoroughly tested in a multithreaded enviroment too.
>
>Alan.
>
>
>-------------------------------------------------------
>This SF.Net email is sponsored by The 2004 JavaOne(SM) Conference
>Learn from the experts at JavaOne(SM), Sun's Worldwide Java Developer
>Conference, June 28 - July 1 at the Moscone Center in San Francisco, CA
>REGISTER AND SAVE! http://java.sun.com/javaone/sf Priority Code NWMGYKND
>_______________________________________________
>Chromium-dev mailing list
>Chr...@li...
>https://lists.sourceforge.net/lists/listinfo/chromium-dev
>  
>

Re: [Chromium-dev] some per Vertex Commands dont have the pervertex props

[Keith W. <ke...@tu...>]

chr...@tm... wrote:
> Hallo
> I'm wondering why some commands like glMaterial and glColorMaterial 
> witch I can use per Vertex, don't have a pervertex attribute in the props 
> fields in the file APIspec.txt. Is there a reason?

I can't answer your question, but will point out that while glMaterial is 
allowed between begin/end, glColorMaterial is not.

Keith

Re: [Chromium-dev] The tcpip layer is really frustrating me

[Alan H. <al...@tu...>]

On Wed, Jun 16, 2004 at 02:57:06PM -0700, Mike Houston wrote:
> I have SDP tuned to run ~600MB/s through the netperf programs in 
> Chromium (npserver/npclient), as long as you don't send messages larger 
> than the MTU.  This is excellent news overall.
> 
> However, I only run 10% faster than GigE (105MB/s) because I spend lots 
> time in crTCPIPRecv even though there is ZERO traffic over tcp.  Hell, 
> the only connections are mothership connections.  We are spending most 
> of the time in the giant block around line 782 in tpcip.c which 
> literally has KLUDGE written all over it.  We really need to be able to 
> bail out faster.  Any thoughts/ideas/suggestions before I start ripping 
> code out?

I wrote that code to bypass problems in a multithreaded environment. And
as the comments explain exactly what's going on, it's easier to be able
to rework things.

I certainly agree with Brian, that any work around this should be done
on a branch and thoroughly tested in a multithreaded enviroment too.

Alan.

[Chromium-dev] some per Vertex Commands dont have the pervertex props

[<chr...@tm...>]

Hallo
I'm wondering why some commands like glMaterial and glColorMaterial 
witch I can use per Vertex, don't have a pervertex attribute in the props 
fields in the file APIspec.txt. Is there a reason?

Tobias

Re: [Chromium-dev] Packer conflicting types

[Brian P. <bri...@tu...>]

Christopher Waters wrote:
> It seems that if there is a function in glapi_parser/APIspec.txt that 
> has a non-void return value and does not specify a "get" in props, its 
> packer function definition will be different from its declaration.  I 
> found where this difference occurs in both pack_header.py and packer.py:
> 
> pack_header.py (60):
>         if return_type != 'void' or "get" in apiutil.Properties(func_name):
>             args.append(("writeback", "int *", 0))
> 
> packer.py (242):
>     if "get" in apiutil.Properties(func_name):
>         pointers_ok = 1
>         params.append(("writeback", "int *", 0))
> 
> Was this done intentionally?  If not, adding the check for the void 
> return type in packer.py seems to have worked.
> 

Any function which returns non-void should have the "get" property 
(since it returns/gets some state).  So, the test for 'return_type != 
'void' in pack_header.py should not be needed.

I just tried making that change and found the glMapBufferARB and 
glUnmapBufferARB functions were missing the 'get' property.  I'm 
fixing it now.

Thanks.

-Brian

[Chromium-dev] Packer conflicting types

[Christopher W. <cr...@ms...>]

It seems that if there is a function in glapi_parser/APIspec.txt that 
has a non-void return value and does not specify a "get" in props, its 
packer function definition will be different from its declaration.  I 
found where this difference occurs in both pack_header.py and 
packer.py:

pack_header.py (60):
		if return_type != 'void' or "get" in apiutil.Properties(func_name):
			args.append(("writeback", "int *", 0))

packer.py (242):
	if "get" in apiutil.Properties(func_name):
		pointers_ok = 1
		params.append(("writeback", "int *", 0))

Was this done intentionally?  If not, adding the check for the void 
return type in packer.py seems to have worked.

  -Chris

[Chromium-dev] Re: [Chromium-users] tilesort-->readback trouble.

[Brian P. <bri...@tu...>]

Brian Paul wrote:
> Eric Mueller wrote:
> 
>> I have a config script set up to handle a tilesort (as in 
>> psubmit_first.conf)
>> except i replaced one (originally both) of the render spus with a 
>> readback spu and a packer, and attempted to display the result on 
>> another render spu.
>>
>> psubmit1  -> tilesorta   ----->    readback(left) > pack ---> render
>>                          /\/\/      psubmit2  -> tilesortb   ----->    
>> render (right)
>>
>>
>> the trouble is that this setup only works if the tile that is being 
>> readback and rendered again is based at the origin.
>> if it is not at the origin, then the readback spu still displays 
>> properly, but the render spu after it does not.
>>
>> as far as i can follow the code, this is intended to work.. (but im 
>> not much of a c programmer) Is there a configuration option i need 
>> that i dont have or anything?
> 
> 
> 
> Actually, the last render SPU is rendering what it's supposed to. It's 
> just that your window isn't wide enough to see it.  If you change:
> 
> rendspu.Conf('window_geometry',[20+W,20+H,W,H])
> 
> to:
> 
> rendspu.Conf('window_geometry',[20+W,20+H,W*2,H])
> 
> you'll see the rendering.
> 
> When the Readback SPU sends images to the downstream SPU, it positions 
> them according to the server's tiling information.  This is what makes 
> the reassemble.conf demo work properly.
> 
> This probably isn't the behaviour you want though.  To hack in what you 
> want, edit readbackspu.c and change this line (around 665) from:
> 
>     CompositeTile(window, w, h, readx, ready, drawx, drawy);
> 
> to:
>     CompositeTile(window, w, h, readx, ready, 0, 0);
> 
> 
> In principle, the AddTile line you have on the testnode:
> 
> testnode.AddTile(W,0,W,H)
> 
> Should position the rendering at (0,0) but the raster position code 
> which would interpret that is actually in the tilesort SPU (not the 
> readback SPU or crserver).  Maybe that should be changed.

I've been looking at this problem in more detail and I think I've come 
up with a nice fix that also cleans up some of the ugliness involved 
in dealing with glRasterPos and tilesorting.

Basically, I'm intercepting glWindowPos() commands in the crserver and 
biasing the coordinates by the current tile's position.  In the 
tilesort SPU, glRasterPos, glWindowPos and glBitmap commands update 
the current raster pos in the state tracker.  When it's time to send 
the new raster pos to the servers, we use glWindowPos with a bias to 
do the job.  I'm basically just moving the biasing to the server side.

If I also update the Readback and Binaryswap SPUs to use glWindowPos 
(instead of glRasterPos), the raster position can be correctly 
adjusted by the server - and Eric's config runs as it should.

I'll check in my changes after a bit more testing.

-Brian

[Chromium-dev] It's time to put together an FAQ ?

[Alan M. <al...@re...>]

It seems that since everybody seems to be answering the same questions
on the list over and over again it's time to put together an FAQ.

I'm going to be poring over my mail over the last couple of years and
seeing what I can put in. If anybody has any suggestions then please
post them to the list and I'll start including them.

 
-- 
Alan Matsuoka
Global Professional Services
Red Hat Canada, Ltd
mailto:al...@re...

Re: [Chromium-dev] Some profiling info

[Alan M. <al...@re...>]

On Thu, 2004-06-17 at 03:43, Keith Whitwell wrote:
> Mike Houston wrote:
> > I've just started playing with this thanks to Alan, so bare with me.
> > 
> > It seems that the hangup is on the appside in libcrutil, 87% of the time 
> > spent there.  Does anyone know how I can get the function breakdown from 
> > within the libs?  optstack doesn't seem to be doing the right thing for 
> > me...  I've also set the kernel image to the correct place, but I still 
> > get a listing for no-vmlinux.
> > 
> 
> One problem with profiling is that none of these tools will count programs 
> which are waiting, and that's what you're looking for.  Unless it's in a tight 
> poll loop on a non-blocking ioctl or similar, the network suckage you're 
> looking for is going to look just like an idle task as far as a profiler is 
> concerned.
> 
I'd have to agree with that. Profiling can find your CPU bound
bottlenecks but there's no easy way to account for any wait time. Maybe
you should look at strace and write some scripts to process the output.
You should be able to get a timestamped output of when you are making
system calls.
> What's the difference between your task (presumably) sitting waiting on a 
> network event and an xterm sitting waiting for a keystroke?
> 
> Keith
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by The 2004 JavaOne(SM) Conference
> Learn from the experts at JavaOne(SM), Sun's Worldwide Java Developer
> Conference, June 28 - July 1 at the Moscone Center in San Francisco, CA
> REGISTER AND SAVE! http://java.sun.com/javaone/sf Priority Code NWMGYKND
> _______________________________________________
> Chromium-dev mailing list
> Chr...@li...
> https://lists.sourceforge.net/lists/listinfo/chromium-dev
-- 
Alan Matsuoka
Global Professional Services
Red Hat Canada, Ltd
mailto:al...@re...

Re: [Chromium-dev] It's the calls to FD_ZERO sucking performance

[Brian P. <bri...@tu...>]

Mike Houston wrote:
> It's packspu_context that calls crNetRecv()...  Apparently, we call 
> packspu_CreateContext repeatedly and do:
> 
> while(writeback)
>   crNetRecv();
> 
> Here is where time is getting sucked away.  I fear that we probably do 
> something similar in the tilesort spu...

packspu_CreateContext() is called only when a new rendering context is 
created.  I don't see it being repeatedly called.

-Brian

Re: [Chromium-dev] It's the calls to FD_ZERO sucking performance

[Martins D I. <mi...@en...>]

> > 
> If you are running with a 2.6 kernel with oprofile enabled you can run
> opstack to get gprof like output.
> see http://oprofile.sourceforge.net/doc/opstack.html
> 
> You have to apply a kernel patch to make it work though.The web page I'm
> referring to has a link to it.

Yeah, we're still running a 2.4 kernel and I don't see an upgrade in
the near future. Thanks.

Martins

Re: [Chromium-dev] It's the calls to FD_ZERO sucking performance

[Alan M. <al...@re...>]

On Thu, 2004-06-17 at 08:57, Martins D Innus wrote:
> Mike,
> 	I found the same thing when I was profiling tilesort last week.
> Seems to be doing Recv due to Sync on Swap. I guess the FD_ZERO just takes
> the most time in that loop.  The problem I found with oprofile was that I
> could not get it to tell me how many times a function was called, only how
> much time was spent in a function.  I was able to get gmon.out and bb.out
> on the crserver side by compiling with "-pg -a" but never got a gmon.out
> with any chromium calls in it on the app side (only function calls from my
> application), even when doing the symlink trick and not using crappfaker.
> bb.out has some decent information in it but some of the files I get are
> corrupted so I don't know how much they can be trusted.
> 
If you are running with a 2.6 kernel with oprofile enabled you can run
opstack to get gprof like output.
see http://oprofile.sourceforge.net/doc/opstack.html

You have to apply a kernel patch to make it work though.The web page I'm
referring to has a link to it.


> Oh, also the "make profile" target is slightly broken( at least in the cr
> version I was using).  In cr.mk PROFILE_LAGS should be PROFILE_FLAGS and
> in Linux.mk PROFILEFLAGS should be PROFILE_FLAGS.
> 
> Let me know if you come up with any profiling tricks.
> 
> Martins
> 
> On Wed, 16 Jun 2004, Mike Houston wrote:
> 
> > It's packspu_context that calls crNetRecv()...  Apparently, we call 
> > packspu_CreateContext repeatedly and do:
> > 
> > while(writeback)
> >    crNetRecv();
> > 
> > Here is where time is getting sucked away.  I fear that we probably do 
> > something similar in the tilesort spu...
> > 
> > -Mike
> > 
> > Mike Houston wrote:
> > 
> > > According to the code annotations, thanks again Alan!, the calls to 
> > > FD_ZERO in the recv functions for both SDP and TCP are quite 
> > > expensive.  What's odd is that it's the appfaker side that is getting 
> > > creamed by the FD_ZERO in recv.  What the heck is the appfaker doing 
> > > recv's for?!
> > >
> > > -Mike
> > >
> > >
> > > -------------------------------------------------------
> > > This SF.Net email is sponsored by The 2004 JavaOne(SM) Conference
> > > Learn from the experts at JavaOne(SM), Sun's Worldwide Java Developer
> > > Conference, June 28 - July 1 at the Moscone Center in San Francisco, CA
> > > REGISTER AND SAVE! http://java.sun.com/javaone/sf Priority Code NWMGYKND
> > > _______________________________________________
> > > Chromium-dev mailing list
> > > Chr...@li...
> > > https://lists.sourceforge.net/lists/listinfo/chromium-dev
> > 
> > 
> > 
> > 
> > -------------------------------------------------------
> > This SF.Net email is sponsored by The 2004 JavaOne(SM) Conference
> > Learn from the experts at JavaOne(SM), Sun's Worldwide Java Developer
> > Conference, June 28 - July 1 at the Moscone Center in San Francisco, CA
> > REGISTER AND SAVE! http://java.sun.com/javaone/sf Priority Code NWMGYKND
> > _______________________________________________
> > Chromium-dev mailing list
> > Chr...@li...
> > https://lists.sourceforge.net/lists/listinfo/chromium-dev
> > 
> 
> -----------------------------------------------------------------------
> Martins Innus				(716) 645-6500 x534
> Scientific Visualization Specialist	mi...@cc...
> Center for Computational Research	www.ccr.buffalo.edu/martins.htm
> University at Buffalo
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by The 2004 JavaOne(SM) Conference
> Learn from the experts at JavaOne(SM), Sun's Worldwide Java Developer
> Conference, June 28 - July 1 at the Moscone Center in San Francisco, CA
> REGISTER AND SAVE! http://java.sun.com/javaone/sf Priority Code NWMGYKND
> _______________________________________________
> Chromium-dev mailing list
> Chr...@li...
> https://lists.sourceforge.net/lists/listinfo/chromium-dev
-- 
Alan Matsuoka
Global Professional Services
Red Hat Canada, Ltd
mailto:al...@re...

Re: [Chromium-dev] It's the calls to FD_ZERO sucking performance

[Martins D I. <mi...@en...>]

Mike,
	I found the same thing when I was profiling tilesort last week.
Seems to be doing Recv due to Sync on Swap. I guess the FD_ZERO just takes
the most time in that loop.  The problem I found with oprofile was that I
could not get it to tell me how many times a function was called, only how
much time was spent in a function.  I was able to get gmon.out and bb.out
on the crserver side by compiling with "-pg -a" but never got a gmon.out
with any chromium calls in it on the app side (only function calls from my
application), even when doing the symlink trick and not using crappfaker.
bb.out has some decent information in it but some of the files I get are
corrupted so I don't know how much they can be trusted.

Oh, also the "make profile" target is slightly broken( at least in the cr
version I was using).  In cr.mk PROFILE_LAGS should be PROFILE_FLAGS and
in Linux.mk PROFILEFLAGS should be PROFILE_FLAGS.

Let me know if you come up with any profiling tricks.

Martins

On Wed, 16 Jun 2004, Mike Houston wrote:

> It's packspu_context that calls crNetRecv()...  Apparently, we call 
> packspu_CreateContext repeatedly and do:
> 
> while(writeback)
>    crNetRecv();
> 
> Here is where time is getting sucked away.  I fear that we probably do 
> something similar in the tilesort spu...
> 
> -Mike
> 
> Mike Houston wrote:
> 
> > According to the code annotations, thanks again Alan!, the calls to 
> > FD_ZERO in the recv functions for both SDP and TCP are quite 
> > expensive.  What's odd is that it's the appfaker side that is getting 
> > creamed by the FD_ZERO in recv.  What the heck is the appfaker doing 
> > recv's for?!
> >
> > -Mike
> >
> >
> > -------------------------------------------------------
> > This SF.Net email is sponsored by The 2004 JavaOne(SM) Conference
> > Learn from the experts at JavaOne(SM), Sun's Worldwide Java Developer
> > Conference, June 28 - July 1 at the Moscone Center in San Francisco, CA
> > REGISTER AND SAVE! http://java.sun.com/javaone/sf Priority Code NWMGYKND
> > _______________________________________________
> > Chromium-dev mailing list
> > Chr...@li...
> > https://lists.sourceforge.net/lists/listinfo/chromium-dev
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by The 2004 JavaOne(SM) Conference
> Learn from the experts at JavaOne(SM), Sun's Worldwide Java Developer
> Conference, June 28 - July 1 at the Moscone Center in San Francisco, CA
> REGISTER AND SAVE! http://java.sun.com/javaone/sf Priority Code NWMGYKND
> _______________________________________________
> Chromium-dev mailing list
> Chr...@li...
> https://lists.sourceforge.net/lists/listinfo/chromium-dev
> 

-----------------------------------------------------------------------
Martins Innus				(716) 645-6500 x534
Scientific Visualization Specialist	mi...@cc...
Center for Computational Research	www.ccr.buffalo.edu/martins.htm
University at Buffalo

Re: [Chromium-dev] Some profiling info

[Keith W. <ke...@tu...>]

Mike Houston wrote:
> I've just started playing with this thanks to Alan, so bare with me.
> 
> It seems that the hangup is on the appside in libcrutil, 87% of the time 
> spent there.  Does anyone know how I can get the function breakdown from 
> within the libs?  optstack doesn't seem to be doing the right thing for 
> me...  I've also set the kernel image to the correct place, but I still 
> get a listing for no-vmlinux.
> 

One problem with profiling is that none of these tools will count programs 
which are waiting, and that's what you're looking for.  Unless it's in a tight 
poll loop on a non-blocking ioctl or similar, the network suckage you're 
looking for is going to look just like an idle task as far as a profiler is 
concerned.

What's the difference between your task (presumably) sitting waiting on a 
network event and an xterm sitting waiting for a keystroke?

Keith

Re: [Chromium-dev] It's the calls to FD_ZERO sucking performance

[Mike H. <mho...@gr...>]

SDP layer choked.  FD_ZERO is not the problem....  grumble.  The 
appfaker does like to do recvs which is still concerning to me.

-Mike

Mike Houston wrote:

> According to the code annotations, thanks again Alan!, the calls to 
> FD_ZERO in the recv functions for both SDP and TCP are quite 
> expensive.  What's odd is that it's the appfaker side that is getting 
> creamed by the FD_ZERO in recv.  What the heck is the appfaker doing 
> recv's for?!
>
> -Mike
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by The 2004 JavaOne(SM) Conference
> Learn from the experts at JavaOne(SM), Sun's Worldwide Java Developer
> Conference, June 28 - July 1 at the Moscone Center in San Francisco, CA
> REGISTER AND SAVE! http://java.sun.com/javaone/sf Priority Code NWMGYKND
> _______________________________________________
> Chromium-dev mailing list
> Chr...@li...
> https://lists.sourceforge.net/lists/listinfo/chromium-dev

Re: [Chromium-dev] It's the calls to FD_ZERO sucking performance

[Mike H. <mho...@gr...>]

SDP layer choked.  FD_ZERO is not the problem....  grumble.  The 
appfaker does like to do recvs which is still concerning to me.

-Mike

Mike Houston wrote:

> It's packspu_context that calls crNetRecv()...  Apparently, we call 
> packspu_CreateContext repeatedly and do:
>
> while(writeback)
>   crNetRecv();
>
> Here is where time is getting sucked away.  I fear that we probably do 
> something similar in the tilesort spu...
>
> -Mike
>
> Mike Houston wrote:
>
>> According to the code annotations, thanks again Alan!, the calls to 
>> FD_ZERO in the recv functions for both SDP and TCP are quite 
>> expensive.  What's odd is that it's the appfaker side that is getting 
>> creamed by the FD_ZERO in recv.  What the heck is the appfaker doing 
>> recv's for?!
>>
>> -Mike
>>
>>
>> -------------------------------------------------------
>> This SF.Net email is sponsored by The 2004 JavaOne(SM) Conference
>> Learn from the experts at JavaOne(SM), Sun's Worldwide Java Developer
>> Conference, June 28 - July 1 at the Moscone Center in San Francisco, CA
>> REGISTER AND SAVE! http://java.sun.com/javaone/sf Priority Code NWMGYKND
>> _______________________________________________
>> Chromium-dev mailing list
>> Chr...@li...
>> https://lists.sourceforge.net/lists/listinfo/chromium-dev
>
>
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by The 2004 JavaOne(SM) Conference
> Learn from the experts at JavaOne(SM), Sun's Worldwide Java Developer
> Conference, June 28 - July 1 at the Moscone Center in San Francisco, CA
> REGISTER AND SAVE! http://java.sun.com/javaone/sf Priority Code NWMGYKND
> _______________________________________________
> Chromium-dev mailing list
> Chr...@li...
> https://lists.sourceforge.net/lists/listinfo/chromium-dev

Re: [Chromium-dev] It's the calls to FD_ZERO sucking performance

[Mike H. <mho...@gr...>]

It's packspu_context that calls crNetRecv()...  Apparently, we call 
packspu_CreateContext repeatedly and do:

while(writeback)
   crNetRecv();

Here is where time is getting sucked away.  I fear that we probably do 
something similar in the tilesort spu...

-Mike

Mike Houston wrote:

> According to the code annotations, thanks again Alan!, the calls to 
> FD_ZERO in the recv functions for both SDP and TCP are quite 
> expensive.  What's odd is that it's the appfaker side that is getting 
> creamed by the FD_ZERO in recv.  What the heck is the appfaker doing 
> recv's for?!
>
> -Mike
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by The 2004 JavaOne(SM) Conference
> Learn from the experts at JavaOne(SM), Sun's Worldwide Java Developer
> Conference, June 28 - July 1 at the Moscone Center in San Francisco, CA
> REGISTER AND SAVE! http://java.sun.com/javaone/sf Priority Code NWMGYKND
> _______________________________________________
> Chromium-dev mailing list
> Chr...@li...
> https://lists.sourceforge.net/lists/listinfo/chromium-dev

[Chromium-dev] It's the calls to FD_ZERO sucking performance

[Mike H. <mho...@gr...>]

According to the code annotations, thanks again Alan!, the calls to 
FD_ZERO in the recv functions for both SDP and TCP are quite expensive.  
What's odd is that it's the appfaker side that is getting creamed by the 
FD_ZERO in recv.  What the heck is the appfaker doing recv's for?!

-Mike

Re: [Chromium-dev] Some profiling info

[Alan M. <al...@re...>]

On Wed, 2004-06-16 at 20:52, Mike Houston wrote:
> I've just started playing with this thanks to Alan, so bare with me.
> 
> It seems that the hangup is on the appside in libcrutil, 87% of the time 
> spent there.  Does anyone know how I can get the function breakdown from 
> within the libs? 
There are options for opannotate to do that. See
http://oprofile.sourceforge.net/doc/results.html



 optstack doesn't seem to be doing the right thing for 
> me...  I've also set the kernel image to the correct place, but I still 
> get a listing for no-vmlinux.
> 
> Running SDP
>  From the crserver:
> 
> CPU: P4 / Xeon, speed 2399.55 MHz (estimated)
> Counted GLOBAL_POWER_EVENTS events (time during which processor is not 
> stopped)
> with a unit mask of 0x01 (count cycles when processor is active) count 
> 100000
> GLOBAL_POWER_E...|
>   samples|      %|
> ------------------
>    397093 37.5357 no-vmlinux
>    185893 17.5718 libcrserver_crunpacker_copy.so
>    133904 12.6574 crserver
>    102459  9.6851 vmlinux
>     62978  5.9531 libnopspu.so
>     37783  3.5715 mod_vapi
>     33447  3.1616 ib_sdp
>     26554  2.5100 libcrutil.so
>     23777  2.2475 mod_thh
>     14750  1.3943 mod_vapi_common
>     12952  1.2243 libm-2.3.3.so
>      6974  0.6592 libc-2.3.3.so
>      6511  0.6155 oprofiled
>      3013  0.2848 ib_tavor
>      2060  0.1947 mod_vipkl
>      1541  0.1457 ib_core
>      1329  0.1256 mosal
>      1159  0.1096 oprofile
>       761  0.0719 bash
>       559  0.0528 ld-2.3.3.so
>       504  0.0476 ib_poll
>       458  0.0433 libpthread-0.61.so
>       241  0.0228 e1000
>       208  0.0197 gmond
>       144  0.0136 libsensors.so.3.0.4
>       120  0.0113 jbd
>       103  0.0097 sunrpc
>        97  0.0092 ext3
>        67  0.0063 libspuload.so
>        65  0.0061 nfs
>        60  0.0057 tcsh
>        55  0.0052 ehci_hcd
>        50  0.0047 uhci_hcd
>        49  0.0046 libcrypto.so.0.9.7a
>        34  0.0032 sshd
>        22  0.0021 grep
>        17  0.0016 ntpd
>        14  0.0013 insmod
>        11  0.0010 w83627hf
>  
> And the appfaker:
> 
> CPU: P4 / Xeon, speed 2399.62 MHz (estimated)
> Counted GLOBAL_POWER_EVENTS events (time during which processor is not 
> stopped)
> with a unit mask of 0x01 (count cycles when processor is active) count 
> 100000
> GLOBAL_POWER_E...|
>   samples|      %|
> ------------------
>   5591272 88.4411 libcrutil.so
>    141833  2.2435 no-vmlinux
>    140482  2.2221 libpackspu.so
>    124020  1.9617 libpackspu_crpacker_copy.so
>    101485  1.6053 vmlinux
>     81807  1.2940 atlantis
>     28508  0.4509 ib_sdp
>     18818  0.2977 mod_vapi
>     17494  0.2767 oprofiled
>     15970  0.2526 libcrfaker.so
>     10969  0.1735 libc-2.3.3.so
>      9650  0.1526 mod_thh
>      6511  0.1030 libm-2.3.3.so
>      4721  0.0747 Xorg
>      4642  0.0734 mod_vapi_common
>      4154  0.0657 libz.so.1.2.1.1
>      2403  0.0380 ib_tavor
>      2380  0.0376 libglut.so.3.7
>      2045  0.0323 ld-2.3.3.so
>      1939  0.0307 libX11.so.6.2
>      1266  0.0200 mod_vipkl
>      1263  0.0200 oprofile
>      1034  0.0164 bash
>       859  0.0136 ib_core
>       749  0.0118 mosal
>       744  0.0118 libGLU.so.1.3
>       522  0.0083 xkbcomp
>       507  0.0080 ximcp.so.2
>       462  0.0073 ib_poll
>       458  0.0072 tcsh
>       392  0.0062 insmod
>       286  0.0045 libsensors.so.3.0.4
>       277  0.0044 libpthread-0.61.so
>       226  0.0036 gmond
>       192  0.0030 jbd
>       180  0.0028 e1000
>       138  0.0022 cc1
>       138  0.0022 opensm
>       137  0.0022 libnss_ldap-2.3.3.so
>       123  0.0019 ext3
>       103  0.0016 libspuload.so
>        92  0.0015 sunrpc
>        77  0.0012 nfs
>        67  0.0011 libcomplib.so.0.0
>        61 9.6e-04 uhci_hcd
>        55 8.7e-04 ehci_hcd
>        44 7.0e-04 twm
>        43 6.8e-04 ntpd
>        36 5.7e-04 libcrypto.so.0.9.7a
>        31 4.9e-04 cpp
>        29 4.6e-04 sshd
>        26 4.1e-04 grep
>        26 4.1e-04 libGL.so.1.2
>        22 3.5e-04 irqbalance
>        21 3.3e-04 libXt.so.6.0
>        18 2.8e-04 gawk
>        17 2.7e-04 libtermcap.so.2.0.8
> 
> 
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by The 2004 JavaOne(SM) Conference
> Learn from the experts at JavaOne(SM), Sun's Worldwide Java Developer
> Conference, June 28 - July 1 at the Moscone Center in San Francisco, CA
> REGISTER AND SAVE! http://java.sun.com/javaone/sf Priority Code NWMGYKND
> _______________________________________________
> Chromium-dev mailing list
> Chr...@li...
> https://lists.sourceforge.net/lists/listinfo/chromium-dev
-- 
Alan Matsuoka
Global Professional Services
Red Hat Canada, Ltd
mailto:al...@re...

[Chromium-dev] SDP vs GigE Performance

[Mike H. <mho...@gr...>]

Here are some simple breakdowns with the nop spu and crdemo.conf.  This 
is with some early bailout hacks I've made to the tcp layer.  It turns 
out that we aren't getting burned there as much as I thought...  More on 
that when I get deeper into profiling...

App         Network       FPS
Atlantis        TCP          1560
Atlantis        SDP          3350
City             TCP          872
City             SDP          2080

The netperf app reports 105MB/s for our GigE implementation and ~550MB/s 
for SDP (it depends on MTU and message size).

-Mike

[Chromium-dev] Some profiling info

[Mike H. <mho...@gr...>]

I've just started playing with this thanks to Alan, so bare with me.

It seems that the hangup is on the appside in libcrutil, 87% of the time 
spent there.  Does anyone know how I can get the function breakdown from 
within the libs?  optstack doesn't seem to be doing the right thing for 
me...  I've also set the kernel image to the correct place, but I still 
get a listing for no-vmlinux.

Running SDP
 From the crserver:

CPU: P4 / Xeon, speed 2399.55 MHz (estimated)
Counted GLOBAL_POWER_EVENTS events (time during which processor is not 
stopped)
with a unit mask of 0x01 (count cycles when processor is active) count 
100000
GLOBAL_POWER_E...|
  samples|      %|
------------------
   397093 37.5357 no-vmlinux
   185893 17.5718 libcrserver_crunpacker_copy.so
   133904 12.6574 crserver
   102459  9.6851 vmlinux
    62978  5.9531 libnopspu.so
    37783  3.5715 mod_vapi
    33447  3.1616 ib_sdp
    26554  2.5100 libcrutil.so
    23777  2.2475 mod_thh
    14750  1.3943 mod_vapi_common
    12952  1.2243 libm-2.3.3.so
     6974  0.6592 libc-2.3.3.so
     6511  0.6155 oprofiled
     3013  0.2848 ib_tavor
     2060  0.1947 mod_vipkl
     1541  0.1457 ib_core
     1329  0.1256 mosal
     1159  0.1096 oprofile
      761  0.0719 bash
      559  0.0528 ld-2.3.3.so
      504  0.0476 ib_poll
      458  0.0433 libpthread-0.61.so
      241  0.0228 e1000
      208  0.0197 gmond
      144  0.0136 libsensors.so.3.0.4
      120  0.0113 jbd
      103  0.0097 sunrpc
       97  0.0092 ext3
       67  0.0063 libspuload.so
       65  0.0061 nfs
       60  0.0057 tcsh
       55  0.0052 ehci_hcd
       50  0.0047 uhci_hcd
       49  0.0046 libcrypto.so.0.9.7a
       34  0.0032 sshd
       22  0.0021 grep
       17  0.0016 ntpd
       14  0.0013 insmod
       11  0.0010 w83627hf
 
And the appfaker:

CPU: P4 / Xeon, speed 2399.62 MHz (estimated)
Counted GLOBAL_POWER_EVENTS events (time during which processor is not 
stopped)
with a unit mask of 0x01 (count cycles when processor is active) count 
100000
GLOBAL_POWER_E...|
  samples|      %|
------------------
  5591272 88.4411 libcrutil.so
   141833  2.2435 no-vmlinux
   140482  2.2221 libpackspu.so
   124020  1.9617 libpackspu_crpacker_copy.so
   101485  1.6053 vmlinux
    81807  1.2940 atlantis
    28508  0.4509 ib_sdp
    18818  0.2977 mod_vapi
    17494  0.2767 oprofiled
    15970  0.2526 libcrfaker.so
    10969  0.1735 libc-2.3.3.so
     9650  0.1526 mod_thh
     6511  0.1030 libm-2.3.3.so
     4721  0.0747 Xorg
     4642  0.0734 mod_vapi_common
     4154  0.0657 libz.so.1.2.1.1
     2403  0.0380 ib_tavor
     2380  0.0376 libglut.so.3.7
     2045  0.0323 ld-2.3.3.so
     1939  0.0307 libX11.so.6.2
     1266  0.0200 mod_vipkl
     1263  0.0200 oprofile
     1034  0.0164 bash
      859  0.0136 ib_core
      749  0.0118 mosal
      744  0.0118 libGLU.so.1.3
      522  0.0083 xkbcomp
      507  0.0080 ximcp.so.2
      462  0.0073 ib_poll
      458  0.0072 tcsh
      392  0.0062 insmod
      286  0.0045 libsensors.so.3.0.4
      277  0.0044 libpthread-0.61.so
      226  0.0036 gmond
      192  0.0030 jbd
      180  0.0028 e1000
      138  0.0022 cc1
      138  0.0022 opensm
      137  0.0022 libnss_ldap-2.3.3.so
      123  0.0019 ext3
      103  0.0016 libspuload.so
       92  0.0015 sunrpc
       77  0.0012 nfs
       67  0.0011 libcomplib.so.0.0
       61 9.6e-04 uhci_hcd
       55 8.7e-04 ehci_hcd
       44 7.0e-04 twm
       43 6.8e-04 ntpd
       36 5.7e-04 libcrypto.so.0.9.7a
       31 4.9e-04 cpp
       29 4.6e-04 sshd
       26 4.1e-04 grep
       26 4.1e-04 libGL.so.1.2
       22 3.5e-04 irqbalance
       21 3.3e-04 libXt.so.6.0
       18 2.8e-04 gawk
       17 2.7e-04 libtermcap.so.2.0.8

Re: [Chromium-dev] Profiling Chromium?

[Alan M. <al...@re...>]

On Wed, 2004-06-16 at 18:48, David C. Thompson wrote:
> On Wed, 2004-06-16 at 15:43, Alan Matsuoka wrote:
> > On Wed, 2004-06-16 at 18:44, Mike Houston wrote:
> > > I built with CFLAGS and LDFLAGS including "-g and -pg".  However, I'm 
> > > not getting the gmon.out files I'm used to seeing, and that I get with 
> > > other apps.  What's the trick here?
> > > 
> > You have to use oprofile. Make sure your kernel is enabled for it.
> 
> Why? Is it because the profiling version of libc is static?
> 
Nope. It's just happens to be the profiler that we use the most around
here. It's not as intrusive as gprof and gives me better information. I
can profile all of the bits and pieces of Chromium at once and I can get
a far more accurate set of numbers.


> 		David
> 
> 
-- 
Alan Matsuoka
Global Professional Services
Red Hat Canada, Ltd
mailto:al...@re...

Re: [Chromium-dev] ByteSwapping and MPI/MPT

[Alan M. <al...@re...>]

On Wed, 2004-06-16 at 18:11, David M Chilongo wrote:
> Hello,
> I started working on chromium only recently, and this issue might have
> already been addressed, but since I haven't seen anything recent on the
> mailing lists....
> The pack spu that ships with cr-1.7 doesn't seem to perform byteswapping
> for glEnable and glDisable calls, which results in unexpected coloring. I
> looked through the files and added the following temporary fix in
> <chromium-dir>/spu/pack/pack_spuclient.c:
> 
> - at the end of pack_Enable() , I replaced crPackEnable(cap) with
>         if(pack_spu.swap) crPackEnableSWAP(cap);
>         else crPackEnable(cap);
> 

> and similarly with pack_Disable. I'm not sure whether that's the best
> place to put the fix, but so far it works ok (i'm running apps between a
> little-endian linux pc and an irix...)
> 
>
It's fixed in CVS.
-- 
Alan Matsuoka
Global Professional Services
Red Hat Canada, Ltd
mailto:al...@re...