gamedevlists-general — List for discussing general game development issues

Re: [GD-General] FreeType

[Andras B. <and...@gm...>]

Sunday, January 1, 2006, 10:58:59 PM, you wrote:

>> I'm not sure exactly where your problem is. Do the generated glyphs
>> look bad themselves? Or do they look bad only when rendered on
>> screen?

> They look bad, period.  This is a 128x256 texture of Courier New
> rendered with anti-aliasing.

> http://bookofhook.com/shittyfonts.png

I believe the problem you are seeing is that the fonts are small _and_
antialiased. If you look at how Windows renders small fonts, you will
notice that it either uses ClearType, which is basically subpixel
rendering, or, if ClearType is disabled, then it renders without
antialiasing. The problem is, that antialiasing works great for thick
fonts, because it smoothes the edges, but for very small fonts, it
will only make the curves appear more or less translucent. ClearType
on the other hand does not have this problem, because it can make
lines thinner without making them translucent.

So, if subpixel font rendering is not an option, then I would
recommend turning off antialiasing for small fonts.



Andras

Re: [GD-General] FreeType

[Ash H. <hen...@gm...>]

I'm not completely sure how PyGame works but this snippet of OpenGL Red Boo=
k
text was helpful to me when trying to get better 2D rasterisation.

" If exact two-dimensional rasterization is desired, you must carefully
specify both the orthographic projection and the vertices of primitives tha=
t
are to be rasterized. The orthographic projection should be specified with
integer coordinates, as shown in the following example:

gluOrtho2D(0, width, 0, height);

where width and height are the dimensions of the viewport. Given this
projection matrix, polygon vertices and pixel image positions should be
placed at integer coordinates to rasterize predictably. For example,
glRecti(0, 0, 1, 1) reliably fills the lower left pixel of the viewport, an=
d
glRasterPos2i(0, 0) reliably positions an unzoomed image at the lower left
of the viewport. Point vertices, line vertices, and bitmap positions should
be placed at half-integer locations, however. For example, a line drawn fro=
m
(x1, 0.5) to (x2, 0.5) will be reliably rendered along the bottom row of
pixels int the viewport, and a point drawn at (0.5, 0.5) will reliably fill
the same pixel as glRecti(0, 0, 1, 1).

An optimum compromise that allows all primitives to be specified at integer
positions, while still ensuring predictable rasterization, is to translate =
x
and y by 0.375, as shown in the following code fragment. Such a translation
keeps polygon and pixel image edges safely away from the centers of pixels,
while moving line vertices close enough to the pixel centers."

Can't say as I found a completely satisfactory solution yet though. After
watching a few interviews with Bill Hill at Microsoft on ClearView it galls
me that my FT2 fonts look so dodgy. I contemplated the possibility of a
fragment program based subpixel antialiaser, but don't know enough about ho=
w
TrueType works (to account correctly for hinting for example) to proceed at
the moment.

Another hassle I had was what to do if FSAA is forced (as in a driver
control panel).

Sorry if that's not much use to you Brian, a quick search turned up there
was an OpenGL renderer for PyGame (I think?) but wasn't sure if it was an
exclusive or standard thing.

As an aside you can check what your font smoothing mode in windows in
Display Properties->Appearance->Effects and there's a powertoy from MS to
tweak ClearType to your display which is handy.

On 1/2/06, Brian Hook <ho...@bo...> wrote:
>
> Thread necromancy!
>
> I've started using FT2 via PyGame, and the font rendering...sucks.
> I'm sticking with Courier New on the Windows platform and generating
> glyphs, anti-aliased, into
>
> At least two problems have surfaced.  The first is that the fonts just
> don't look that good compared to the built-in Windows renderer, which
> presumably uses ClearType or something similar (using zoomin the
> difference is fairly obvious).
>
> The second problem, which is not FT2 specific, is that it seems that
> where I need to index my texels differs from driver to driver.  On
> some drivers I need to go from 0,0 to glyph.width,glyph.height, and on
> other drivers I need to offset those values by 0.5 texels.  Anyone
> else seen this?
>
> Brian
>
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through log
> files
> for problems?  Stop!  Download the new AJAX search engine that makes
> searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
> http://ads.osdn.com/?ad_idv37&alloc_id=16865&opclick
> _______________________________________________
> Gamedevlists-general mailing list
> Gam...@li...
> https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_idU7
>

Re: [GD-General] FreeType

[Ivan-Assen I. <iva...@gm...>]

PiBBbHNvLCBJJ20gbm90IHF1aXRlIHN1cmUgd2hhdCBKb24gd2FzIHJlY29tbWVuZGluZyBpbiB0
aGF0IHRocmVhZCAtLQo+IGV2ZXJ5IHRpbWUgYSBzdHJpbmcgbmVlZHMgdG8gYmUgcmVuZGVyZWQg
b24gc2NyZWVuLCByZW5kZXIgaXQgaW4gaXRzCj4gZW50aXJldHkgaW50byBhIHRleHR1cmU/CgpZ
b3UgY2FuIGxpdmUgd2l0aCBnbHlwaC1iYXNlZCBmb250cyBpZiB5b3Ugb25seSBuZWVkIEV1cm9w
ZWFuCmxhbmd1YWdlcyAtIGUuZy4gTGF0aW4sIExhdGluLXdpdGgtZnVubnktZ3V5cy1vbi10b3As
IEN5cmlsbGljIGFuZApHcmVlayAtIHRoYXQgaXMsIGlmIHlvdSBzdGljayB3aXRoIDEwMHMgb2Yg
Y2hhcmFjdGVycy4gVGhlcmUncyBubyB3YXkKaW4gaGVsbCB5b3UgY2FuIGZpdCBnbHlwaHMgZm9y
IGEgQ0pLIGxhbmd1YWdlICgxMDAwMHMpIGludG8gYW55IHNhbmUKYW1vdW50IG9mIHRleHR1cmUg
bWVtb3J5LiBTbyB3aGF0IHlvdSBkbyBpcyBhbGxvY2F0ZSBvbmUgb3Igc2V2ZXJhbAp0ZXh0dXJl
cyBhbmQgdXNlIHRoZW0gYXMgYSBjYWNoZSBmb3Igc3RvcmluZyByZW5kZXJlZCBlbnRpcmUgd29y
ZHMsIG9yCnBocmFzZXMsIG9yIHdoYXRldmVyIG1ha2VzIHNlbnNlIGluIHlvdXIgY2FzZS4gVXNl
IHNvbWUgc29ydCBvZiAyRAphbGxvY2F0b3IgdG8ga2VlcCB0cmFjayBvZiBmcmVlIHN1YnJlY3Rz
LCByZXRpcmUgdGhlIGxlYXN0IHJlY2VudGx5CnVzZWQgc3RyaW5ncyBvciB3aXBlIHRoZSBlbnRp
cmUgdGV4dHVyZSBjbGVhbiwgcmVidWlsZGluZyB3aGF0J3MKbmVlZGVkIGluIHRoaXMgZnJhbWUu
IFR3ZWFrIGFsbCBjb25zdGFudHMgaW52b2x2ZWQgdW50aWwgeW91IGRvbid0CnRyYXNoIHRoZSBj
YWNoZSB0b28gbXVjaCA6LSkKCkFuIGFkZGl0aW9uYWwgYmVuZWZpdCBpcyB0aGF0IHlvdSBnZXQg
c3VicGl4ZWwga2VybmluZyBnbHlwaC10by1nbHlwaCwKd2hpY2ggaXMgcmVhbGx5IGltcG9ydGFu
dCBmb3Igc21hbGwgdHlwZSBzaXplcy4K

Re: [GD-General] FreeType

[Brian H. <ho...@bo...>]

> I'm not sure exactly where your problem is. Do the generated glyphs
> look bad themselves? Or do they look bad only when rendered on
> screen?

They look bad, period.  This is a 128x256 texture of Courier New 
rendered with anti-aliasing.

http://bookofhook.com/shittyfonts.png

It just looks grim.  Maybe I'mm doing something horrible (I'm using 
Pygame.font to render, which uses FreeType2).

> bb=3Dget_topic;f=3D3;t=3D009649 Look for Jon's (jwatte) posts, the truth
> is with him. ;)

Unfortunately there is no clean way in Python to call into Windows and 
do all GDI stuff and call back out unless I write that myself.  Which 
I'm not sure I'm eager to do on the off chance it does look better =3D)

That said, I'd still mightily consider it if it gets me away from this 
asstasticness I'm dealing with right now.

Also, I'm not quite sure what Jon was recommending in that thread -- 
every time a string needs to be rendered on screen, render it in its 
entirety into a texture?

Brian

Re: [GD-General] FreeType

[Andras B. <and...@gm...>]

Brian,

I'm not sure exactly where your problem is. Do the generated glyphs
look bad themselves? Or do they look bad only when rendered on screen?
Anyways, I used to have a pretty bad font renderer too, here are some
links that helped me perfect it:

About ClearType font rendering:
http://grc.com/ctwhat.htm
ClearType is absolutely great, but it only works in 2D pixel perfect
rendering, as soon as you get some perspective, or stretch, you are
gonna miss pixels, and it ruins the whole idea of ClearType. Another
problem is, that the Windows text renderer creates a true color image
for cleartype, and there's no alpha information. And it's not obvious
how to combine those 1/3 pixel parts when blending it on a non-solid
background..

About texture fonts:
http://www.opengl.org/discussion_boards/cgi_directory/ultimatebb.cgi?ubb=get_topic;f=3;t=009649
Look for Jon's (jwatte) posts, the truth is with him. ;) I also
thought that glyph based rendering was the way to go, until I've found
this thread. Rendering entire words is definitely a superior approach.
Especially, if you need support for arabic, chinese or cirillic
languages. Windows text rendering is very fast, and async texture
uploads (via PBO) are very efficient.

About texture filtering:
http://www.mindcontrol.org/~hplus/graphics/opengl-pixel-perfect.html
And so far, I haven't seen any drivers that would not conform to this..

Oh, and here's a great utility to check that everything is pixel
perfect (much better than Windows' built-in magnifier):
http://magnifier.sourceforge.net/

Hmm, one more thing: when rendering text on top of a 3D scene, then
you'll need shadows to make the font readable in every situation. I
prefer one pixel thick crisp hard shadow around the text, but you can
also compute soft shadows easily during the texture generation. Now,
do this with a glyph based renderer. :)


Hope this helps,



Andras



Sunday, January 1, 2006, 9:03:41 PM, you wrote:

> Thread necromancy!

> I've started using FT2 via PyGame, and the font rendering...sucks.
> I'm sticking with Courier New on the Windows platform and generating
> glyphs, anti-aliased, into

> At least two problems have surfaced.  The first is that the fonts just
> don't look that good compared to the built-in Windows renderer, which
> presumably uses ClearType or something similar (using zoomin the
> difference is fairly obvious).

> The second problem, which is not FT2 specific, is that it seems that
> where I need to index my texels differs from driver to driver.  On
> some drivers I need to go from 0,0 to glyph.width,glyph.height, and on
> other drivers I need to offset those values by 0.5 texels.  Anyone
> else seen this?

> Brian

Re: [GD-General] FreeType

[Brian H. <ho...@bo...>]

Thread necromancy!

I've started using FT2 via PyGame, and the font rendering...sucks.  
I'm sticking with Courier New on the Windows platform and generating 
glyphs, anti-aliased, into 

At least two problems have surfaced.  The first is that the fonts just 
don't look that good compared to the built-in Windows renderer, which 
presumably uses ClearType or something similar (using zoomin the 
difference is fairly obvious).  

The second problem, which is not FT2 specific, is that it seems that 
where I need to index my texels differs from driver to driver.  On 
some drivers I need to go from 0,0 to glyph.width,glyph.height, and on 
other drivers I need to offset those values by 0.5 texels.  Anyone 
else seen this?

Brian

Re: [GD-General] Is anybody using CgFX?

[Mike W. <mi...@ge...>]

We've looked into it a bit, but haven't actually tried integrating it 
into the engine yet, basically it's on the list of sdk's that I have 
downloaded & am evaluating. Would be interested to hear how things go 
with it.

Mike W
GDGi
www.gdgi.ca

Noel Llopis wrote:

>On Sunday 02 October 2005 17:24, Ignacio Castaño wrote:
>
>  
>
>>We are using CgFX on FX Composer 2 and NVSG, so it's definitely usable,
>>but your needs might be different from ours, so I'd recommend (and
>>encourage) you to report us the problems that you have found.
>>    
>>
>
>It's good to hear that at least NVidia is using their own product :-) But 
>clearly, your needs and ours (as game developers) are quite different. I'll 
>follow up with you privately about some questions and support.
>
>In the meanwhile, is anybody else (a game developer preferably) actually 
>using CgFX? With some of the issues we've found so far, it really feels 
>like we're the first ones :-(
>
>
>--Noel
>Games from Within
>http://www.gamesfromwithin.com
>
>
>-------------------------------------------------------
>This SF.Net email is sponsored by:
>Power Architecture Resource Center: Free content, downloads, discussions,
>and more. http://solutions.newsforge.com/ibmarch.tmpl
>_______________________________________________
>Gamedevlists-general mailing list
>Gam...@li...
>https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
>Archives:
>http://sourceforge.net/mailarchive/forum.php?forum_idU7
>
>
>  
>

Re: [GD-General] Is anybody using CgFX?

[Noel L. <nl...@co...>]

On Sunday 02 October 2005 17:24, Ignacio Casta=F1o wrote:

> We are using CgFX on FX Composer 2 and NVSG, so it's definitely usable,
> but your needs might be different from ours, so I'd recommend (and
> encourage) you to report us the problems that you have found.

It's good to hear that at least NVidia is using their own product :-) But=20
clearly, your needs and ours (as game developers) are quite different. I'll=
=20
follow up with you privately about some questions and support.

In the meanwhile, is anybody else (a game developer preferably) actually=20
using CgFX? With some of the issues we've found so far, it really feels=20
like we're the first ones :-(


=2D-Noel
Games from Within
http://www.gamesfromwithin.com

Re: [GD-General] Is anybody using CgFX?

[Ignacio <ica...@nv...>]

On Sunday 02 October 2005 08:59 am, Noel Llopis wrote:
> We've been trying to use CgFX with OpenGL and it simply doesn't feel like
> it's usable. The lack of documentation, not being able to compile effects
> offline, lack of good error reporting, and bugs in general render it almo=
st
> useless.
>
> Am I missing something? As much as I hate to say this, HLSL and DirectX
> seem like a much better way to go for PC development.
>
> Is anybody using CgFx at all in their development?

We are using CgFX on FX Composer 2 and NVSG, so it's definitely usable, but=
=20
your needs might be different from ours, so I'd recommend (and encourage) y=
ou=20
to report us the problems that you have found.

Documentation is obviously lacking, but we are currently working on that an=
d=20
it will hopefully be more complete for Cg 1.5. In the meantime we are here =
to=20
help, so feel free to send me any question and I'll do my best to explain i=
t.

Offline compilation is on our radar, but I think it has not been requested =
by=20
any developer. If more people request it, it will be prioritized.

I'm not aware of any problem in the error reporting mechanism, so please,=20
report us your findings.

I know there are still many problems remaining, but CgFX is in better shape=
=20
now than ever and it's getting better. If you are a registered developer, w=
e=20
can give you access to the latest builds to try the latest fixes.

Hope that helps!

=2D-=20
Ignacio Casta=F1o
ica...@nv...

[GD-General] Is anybody using CgFX?

[Noel L. <nl...@co...>]

We've been trying to use CgFX with OpenGL and it simply doesn't feel like 
it's usable. The lack of documentation, not being able to compile effects 
offline, lack of good error reporting, and bugs in general render it almost 
useless. 

Am I missing something? As much as I hate to say this, HLSL and DirectX seem 
like a much better way to go for PC development.

Is anybody using CgFx at all in their development?

 
--Noel
Games from Within
http://www.gamesfromwithin.com

[GD-General] gamma ramp

[<bac...@ke...>]

Hi

I'm trying to set gamma ramp, but every time I switch task, and switch 
back, the settings are gone, even if I call every time 
SetDeviceGammaRamp with the same parameters. But if I change the gamma 
level a little bit (for example 1 byte) and call SetDeviceGammaRamp 
again, it work. Is there an other solution for realtime setting the 
gamma, or do I have to do it with the old way, rendering a blended quad 
over the full screen?

Thanks for your help.

Peter Balogh

Re: [GD-General] responsive threading

[Andras B. <and...@gm...>]

It turns out that all the spikes were my fault. I'm deeply ashamed.. I 
have failed to follow the most important rule of all, that says: profile 
first! I was pretty sure my application was not the one causing the 
stalls, since when I turned the other thread off, I got no spikes at all..

What happened is that I also called another small 3rd party library from 
my main thread, but only when the other thread was also running. This 
library converts coordinates between different projections (this is a 
mapping software), and it seems like that sometimes it takes a pretty 
long time to do this operation, that I assumed to be basically free. Now 
I'm going to dive into this library, to see what's going on.

I think I've learned my lesson:
PROFILE, PROFILE, PROFILE! DO NOT MAKE ASSUMPTIONS! EVER!

Thanks,


Andras


Andras Balogh wrote:
> I'm developing a graphics application, that I want to be very 
> responsive, very smooth, all the time! The rendering is pretty simple, 
> the framerate is high and stable, so I just lock to vsync, and 
> everything moves beautifully smooth! The CPU usage for this thread is 
> always below 10%.
> 
> However, I also have another thread, that has to do some heavy work. 
> This thread does tons of IO operations, and also makes the CPU sweat!
> 
> There is no synchronization between the two threads, so they should be 
> able to run independently. Still, I'm experienceing sudden spikes in the 
> framerate, when the second thread is really hammering. What could be the 
> reason for this? Is it possible to do something about it, to regain the 
> continuous framerate? I'm really not interested in the performance of 
> the second thread, I just want my first thread to run as smooth as 
> possible! I've tried lowering the priority for the second thread, and it 
> does help, but there are still some spikes. Is it possible that it's 
> Windows itself doing some operations at higher priority, like memory 
> management, IO stuff, and the likes?
> 
> Thanks,
> 
> 
> Andras

Re: [GD-General] responsive threading

[Andras B. <and...@gm...>]

No, not really.. :(

I could write something myself, that hammers the CPU, the memory and the 
disk, together and separately, but I don't think I'll have a timeslice 
for that now. I'm in a busy loop. :)


Ivan-Assen Ivanov wrote:
> Andras, can you experiment with varying the processing thread's workload? 
> E.g. change it to computations only, or I/O only, e.g. feed it uncompressed 
> data etc.? That could give you an insight in what's causing it to receive 
> more CPU time.
> 

Re: [GD-General] responsive threading

[Andras B. <and...@gm...>]

Hmm, but when I don't call into that library, then the CPU usage for my 
rendering thread is below 10%! I would think that if I have a busy loop, 
then the CPU would be at 100%, no?

Anyway, I've tried disabling vsync, and surprisingly, there's not much 
difference (except from the tearing). I have no idea why, because now 
the CPU is really at 100% all the time... Although I have to admit that 
it's a bit hard to see the improvement/worsening, because this is not 
something I can measure, so it's totally subjective, and with the 
tearing on screen it's even harder to tell what's going on.

Just to repeat, these spikes are not too bad, only happens like every 
couple seconds, and only maybe for an eight of a second. I guess if my 
app would run at 10fps constant, you'd never even notice.

Hmm, now I've put in some extra sleeps, just for kicks, and it does feel 
a tiny bit more smooth. But again, it's very subjective :)

BTW: I don't think I've ever seen any app handling this amount of data 
without any hiccups (if you happen to know of one, then please let me 
know!), so I'm pretty pleased with the results already. Of course, if 
it's possible to do better, then I'd love to make it better! :P

As for the unknown variables, and black boxes, I'm afraid there'll 
always be some. The driver and Windows itself are just two examples. And 
as we will develop more and more complex applications, we will have to 
integrate more and more off the shelf solutions into our product. I 
think this is something we just have to learn to deal with. Hmm, this 
reminds me of a library I've seen recently, which, when encounters an 
error, just calls exit(-1)!! ==:-O

cheers,


Andras


Brian Hook wrote:
> On Sun, 31 Jul 2005 19:49:46 -0000, Alen Ladavac wrote:
> 
>>Are you sure that waiting for vsync gives out time? 
> 
> 
> That's what I'm thinking as well -- Andras, maybe disable vsync and 
> see how it performs first.  The problem right now is that you haven't 
> really eliminated enough variables for people to work with.  Disabling 
> vsync is one, and unfortunately the black box nature of the library 
> you're using is another.
> 
> Brian

Re: [GD-General] responsive threading

[Ivan-Assen I. <iva...@gm...>]

Andras, can you experiment with varying the processing thread's workload?=
=20
E.g. change it to computations only, or I/O only, e.g. feed it uncompressed=
=20
data etc.? That could give you an insight in what's causing it to receive=
=20
more CPU time.

Re: [GD-General] responsive threading

[Brian H. <ho...@bo...>]

On Sun, 31 Jul 2005 19:49:46 -0000, Alen Ladavac wrote:
> Are you sure that waiting for vsync gives out time? 

That's what I'm thinking as well -- Andras, maybe disable vsync and 
see how it performs first.  The problem right now is that you haven't 
really eliminated enough variables for people to work with.  Disabling 
vsync is one, and unfortunately the black box nature of the library 
you're using is another.

Brian

Re: [GD-General] responsive threading

[Alen L. <ale...@cr...>]

Are you sure that waiting for vsync gives out time? This depends on driver 
and API used, but usually, it is just a busy waiting loop. IIRC, we had to 
jump through some nasty hoops to make our swaps actually yield time to our 
loading thread. YMMV.

Alen


----- Original Message ----- 
From: "Andras Balogh" <and...@gm...>
To: "Alen Ladavac" <gam...@li...>
Sent: Sunday, July 31, 2005 14:51
Subject: Re: [GD-General] responsive threading


>I do wait in the higher priority thread. Actually, this thread spends
> 90% of the frame time waiting (for vsync)! So the idle thread has
> plenty of time, and despite the lower priority, it gets a lot more
> CPU, than my primary thread!
>
>
> Saturday, July 30, 2005, 6:40:15 AM, you wrote:
>
>> If the higher priority thread never ever waits, windows go out of their 
>> way
>> and give some time to the lower priority thread in that case. Search
>> "priority inversion" in MSDN. Perhaps sleeping a bit once in a while in 
>> the
>> rendering thread to give a slice to the processing thread, might help you
>> keep the situation under control.
>
>> HTH,
>> Alen
>
>> ----- Original Message ----- 
>> From: "Andras Balogh" <and...@gm...>
>> To: <gam...@li...>
>> Sent: Friday, July 29, 2005 21:22
>> Subject: Re: [GD-General] responsive threading
>
>
>>> I've tried setting the primary thread to time critical and the other
>>> thread to idle (and I also disabled it's priority boost), and it does 
>>> help
>>> quite a bit, but I'm still seeing some small glitches. I'm not saying 
>>> it's
>>> bad, it's pretty good actually, and people are so used to bad framerates
>>> nowadays that I'm sure they wouldn't complain a tiny bit. I do admit 
>>> that
>>> I'm a bit on the perfectionist side on this one :)
>>>
>>> I have tried setting the process priority class to real-time too, but in
>>> that case my "idle" priority thread won't be really idle anymore,
>>> actually, the hiccups get much worse. I guess I could run in a separate
>>> process even, but that's really overkill.
>>>
>>>
>>> Andras
>>>
>>> Brian Hook wrote:
>>>> At the risk of asking the obvious, have you also set your primary 
>>>> thread
>>>> to run at the highest priority (REAL_TIME or whatever it's called on
>>>> Windows)?
>>>>
>>>> I run audio mixing in a separate thread and by setting things
>>>> appropriately (and sleeping when necessary) I never get hiccups.
>>>>
>>>> Brian
>>>>
>>>> On Fri, 29 Jul 2005 14:35:21 -0600, Andras Balogh wrote:
>>>>
>>>>>I'm loading lots of "small" (avg. 1k x 1k) regions from lots of
>>>>>huge (5GB+) wavelet compressed images. I'm using a 3rd party
>>>>>library to do this (ECW), so I don't know the implementation
>>>>>details, but I could look at the code, if I knew what to look for..
>>>>>
>>>>>
>>>>>Andras
>>>>>
>>>>>
>>>>>Mat Noguchi (BUNGIE) wrote:
>>>>>
>>>>>>What else are you doing on the IO thread to make it eat up so
>>>>>>much CPU time?
>>>>>>
>>>>>>MSN
>>>>>>-----Original Message-----
>>>>>>From: gam...@li...
>>>>>>[mailto:gam...@li...] On
>>>>>>Behalf Of Andras Balogh Sent: Friday, July 29, 2005 10:44 AM To:
>>>>>>gam...@li... Subject: [GD-General]
>>>>>>responsive threading
>>>>>>
>>>>>>I'm developing a graphics application, that I want to be very
>>>>>>responsive, very smooth, all the time! The rendering is pretty
>>>>>>simple, the framerate is high and stable, so I just lock to
>>>>>>vsync, and everything moves beautifully smooth! The CPU usage for
>>>>>>this thread is always below 10%.
>>>>>>
>>>>>>However, I also have another thread, that has to do some heavy
>>>>>>work. This thread does tons of IO operations, and also makes the
>>>>>>CPU sweat!
>>>>>>
>>>>>>There is no synchronization between the two threads, so they
>>>>>>should be able to run independently. Still, I'm experienceing
>>>>>>sudden spikes in the
>>>>>>
>>>>>>framerate, when the second thread is really hammering. What could
>>>>>>be the
>>>>>>
>>>>>>reason for this? Is it possible to do something about it, to
>>>>>>regain the continuous framerate? I'm really not interested in the
>>>>>>performance of the second thread, I just want my first thread to
>>>>>>run as smooth as possible! I've tried lowering the priority for
>>>>>>the second thread, and it
>>>>>>
>>>>>>does help, but there are still some spikes. Is it possible that
>>>>>>it's Windows itself doing some operations at higher priority,
>>>>>>like memory management, IO stuff, and the likes?
>>>>>>
>>>>>>Thanks,
>>>>>>
>>>>>>
>>>>>>Andras
>
>
>
> -------------------------------------------------------
> SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
> from IBM. Find simple to follow Roadmaps, straightforward articles,
> informative Webcasts and more! Get everything you need to get up to
> speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
> _______________________________________________
> Gamedevlists-general mailing list
> Gam...@li...
> https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_id=557
> 

Re: [GD-General] responsive threading

[Andras B. <and...@gm...>]

Could you be a bit more specific? Should I try to decrease the number
of memory allocations? Should I try using a custom memory manager?
BTW: this is not "huge" app by any means, and it only uses two
threads, that's all..

Friday, July 29, 2005, 7:48:48 PM, you wrote:

> Mat makes a good point about MT memory allocations (and CRT).  We have
> ran into issues that stemmed from MT heap allocations in "huge" MT 
> apps.  Probably a good place to start.

> -Beau

> Andras Balogh wrote:

>> Yeah, that's what I was thinking too, that this decompressing library
>> makes some Win32 API calls, that result in the Windows kernel doing
>> some system maintanence at a higher priority..
>>
>> Mat Noguchi (BUNGIE) wrote:
>>
>>> Unfortunately, there are many function calls that can cause stalls under
>>> multithreading: mixing blocking and non-blocking file i/o, allocating
>>> memory with malloc/free, CRT functions that manipulate global state,
>>> etc.

Re: [GD-General] responsive threading

[Andras B. <and...@gm...>]

I do wait in the higher priority thread. Actually, this thread spends
90% of the frame time waiting (for vsync)! So the idle thread has
plenty of time, and despite the lower priority, it gets a lot more
CPU, than my primary thread!


Saturday, July 30, 2005, 6:40:15 AM, you wrote:

> If the higher priority thread never ever waits, windows go out of their way
> and give some time to the lower priority thread in that case. Search
> "priority inversion" in MSDN. Perhaps sleeping a bit once in a while in the
> rendering thread to give a slice to the processing thread, might help you
> keep the situation under control.

> HTH,
> Alen

> ----- Original Message ----- 
> From: "Andras Balogh" <and...@gm...>
> To: <gam...@li...>
> Sent: Friday, July 29, 2005 21:22
> Subject: Re: [GD-General] responsive threading


>> I've tried setting the primary thread to time critical and the other
>> thread to idle (and I also disabled it's priority boost), and it does help
>> quite a bit, but I'm still seeing some small glitches. I'm not saying it's
>> bad, it's pretty good actually, and people are so used to bad framerates
>> nowadays that I'm sure they wouldn't complain a tiny bit. I do admit that
>> I'm a bit on the perfectionist side on this one :)
>>
>> I have tried setting the process priority class to real-time too, but in
>> that case my "idle" priority thread won't be really idle anymore, 
>> actually, the hiccups get much worse. I guess I could run in a separate
>> process even, but that's really overkill.
>>
>>
>> Andras
>>
>> Brian Hook wrote:
>>> At the risk of asking the obvious, have you also set your primary thread
>>> to run at the highest priority (REAL_TIME or whatever it's called on
>>> Windows)?
>>>
>>> I run audio mixing in a separate thread and by setting things 
>>> appropriately (and sleeping when necessary) I never get hiccups.
>>>
>>> Brian
>>>
>>> On Fri, 29 Jul 2005 14:35:21 -0600, Andras Balogh wrote:
>>>
>>>>I'm loading lots of "small" (avg. 1k x 1k) regions from lots of
>>>>huge (5GB+) wavelet compressed images. I'm using a 3rd party
>>>>library to do this (ECW), so I don't know the implementation
>>>>details, but I could look at the code, if I knew what to look for..
>>>>
>>>>
>>>>Andras
>>>>
>>>>
>>>>Mat Noguchi (BUNGIE) wrote:
>>>>
>>>>>What else are you doing on the IO thread to make it eat up so
>>>>>much CPU time?
>>>>>
>>>>>MSN
>>>>>-----Original Message-----
>>>>>From: gam...@li...
>>>>>[mailto:gam...@li...] On
>>>>>Behalf Of Andras Balogh Sent: Friday, July 29, 2005 10:44 AM To:
>>>>>gam...@li... Subject: [GD-General]
>>>>>responsive threading
>>>>>
>>>>>I'm developing a graphics application, that I want to be very
>>>>>responsive, very smooth, all the time! The rendering is pretty
>>>>>simple, the framerate is high and stable, so I just lock to
>>>>>vsync, and everything moves beautifully smooth! The CPU usage for
>>>>>this thread is always below 10%.
>>>>>
>>>>>However, I also have another thread, that has to do some heavy
>>>>>work. This thread does tons of IO operations, and also makes the
>>>>>CPU sweat!
>>>>>
>>>>>There is no synchronization between the two threads, so they
>>>>>should be able to run independently. Still, I'm experienceing
>>>>>sudden spikes in the
>>>>>
>>>>>framerate, when the second thread is really hammering. What could
>>>>>be the
>>>>>
>>>>>reason for this? Is it possible to do something about it, to
>>>>>regain the continuous framerate? I'm really not interested in the
>>>>>performance of the second thread, I just want my first thread to
>>>>>run as smooth as possible! I've tried lowering the priority for
>>>>>the second thread, and it
>>>>>
>>>>>does help, but there are still some spikes. Is it possible that
>>>>>it's Windows itself doing some operations at higher priority,
>>>>>like memory management, IO stuff, and the likes?
>>>>>
>>>>>Thanks,
>>>>>
>>>>>
>>>>>Andras

Re: [GD-General] responsive threading

[Alen L. <ale...@cr...>]

If the higher priority thread never ever waits, windows go out of their way 
and give some time to the lower priority thread in that case. Search 
"priority inversion" in MSDN. Perhaps sleeping a bit once in a while in the 
rendering thread to give a slice to the processing thread, might help you 
keep the situation under control.

HTH,
Alen

----- Original Message ----- 
From: "Andras Balogh" <and...@gm...>
To: <gam...@li...>
Sent: Friday, July 29, 2005 21:22
Subject: Re: [GD-General] responsive threading


> I've tried setting the primary thread to time critical and the other 
> thread to idle (and I also disabled it's priority boost), and it does help 
> quite a bit, but I'm still seeing some small glitches. I'm not saying it's 
> bad, it's pretty good actually, and people are so used to bad framerates 
> nowadays that I'm sure they wouldn't complain a tiny bit. I do admit that 
> I'm a bit on the perfectionist side on this one :)
>
> I have tried setting the process priority class to real-time too, but in 
> that case my "idle" priority thread won't be really idle anymore, 
> actually, the hiccups get much worse. I guess I could run in a separate 
> process even, but that's really overkill.
>
>
> Andras
>
> Brian Hook wrote:
>> At the risk of asking the obvious, have you also set your primary thread 
>> to run at the highest priority (REAL_TIME or whatever it's called on 
>> Windows)?
>>
>> I run audio mixing in a separate thread and by setting things 
>> appropriately (and sleeping when necessary) I never get hiccups.
>>
>> Brian
>>
>> On Fri, 29 Jul 2005 14:35:21 -0600, Andras Balogh wrote:
>>
>>>I'm loading lots of "small" (avg. 1k x 1k) regions from lots of
>>>huge (5GB+) wavelet compressed images. I'm using a 3rd party
>>>library to do this (ECW), so I don't know the implementation
>>>details, but I could look at the code, if I knew what to look for..
>>>
>>>
>>>Andras
>>>
>>>
>>>Mat Noguchi (BUNGIE) wrote:
>>>
>>>>What else are you doing on the IO thread to make it eat up so
>>>>much CPU time?
>>>>
>>>>MSN
>>>>-----Original Message-----
>>>>From: gam...@li...
>>>>[mailto:gam...@li...] On
>>>>Behalf Of Andras Balogh Sent: Friday, July 29, 2005 10:44 AM To:
>>>>gam...@li... Subject: [GD-General]
>>>>responsive threading
>>>>
>>>>I'm developing a graphics application, that I want to be very
>>>>responsive, very smooth, all the time! The rendering is pretty
>>>>simple, the framerate is high and stable, so I just lock to
>>>>vsync, and everything moves beautifully smooth! The CPU usage for
>>>>this thread is always below 10%.
>>>>
>>>>However, I also have another thread, that has to do some heavy
>>>>work. This thread does tons of IO operations, and also makes the
>>>>CPU sweat!
>>>>
>>>>There is no synchronization between the two threads, so they
>>>>should be able to run independently. Still, I'm experienceing
>>>>sudden spikes in the
>>>>
>>>>framerate, when the second thread is really hammering. What could
>>>>be the
>>>>
>>>>reason for this? Is it possible to do something about it, to
>>>>regain the continuous framerate? I'm really not interested in the
>>>>performance of the second thread, I just want my first thread to
>>>>run as smooth as possible! I've tried lowering the priority for
>>>>the second thread, and it
>>>>
>>>>does help, but there are still some spikes. Is it possible that
>>>>it's Windows itself doing some operations at higher priority,
>>>>like memory management, IO stuff, and the likes?
>>>>
>>>>Thanks,
>>>>
>>>>
>>>>Andras
>
>
> -------------------------------------------------------
> SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
> from IBM. Find simple to follow Roadmaps, straightforward articles,
> informative Webcasts and more! Get everything you need to get up to
> speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
> _______________________________________________
> Gamedevlists-general mailing list
> Gam...@li...
> https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_id=557
> 

Re: [GD-General] responsive threading

[Beau A. <alb...@cy...>]

Mat makes a good point about MT memory allocations (and CRT).  We have 
ran into issues that stemmed from MT heap allocations in "huge" MT 
apps.  Probably a good place to start.

-Beau

Andras Balogh wrote:

> Yeah, that's what I was thinking too, that this decompressing library 
> makes some Win32 API calls, that result in the Windows kernel doing 
> some system maintanence at a higher priority..
>
> Mat Noguchi (BUNGIE) wrote:
>
>> Unfortunately, there are many function calls that can cause stalls under
>> multithreading: mixing blocking and non-blocking file i/o, allocating
>> memory with malloc/free, CRT functions that manipulate global state,
>> etc.
>
>
>
> -------------------------------------------------------
> SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
> from IBM. Find simple to follow Roadmaps, straightforward articles,
> informative Webcasts and more! Get everything you need to get up to
> speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
> _______________________________________________
> Gamedevlists-general mailing list
> Gam...@li...
> https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_id=557
>

Re: [GD-General] responsive threading

[Andras B. <and...@gm...>]

Yeah, that's what I was thinking too, that this decompressing library 
makes some Win32 API calls, that result in the Windows kernel doing some 
system maintanence at a higher priority..

Mat Noguchi (BUNGIE) wrote:
> Unfortunately, there are many function calls that can cause stalls under
> multithreading: mixing blocking and non-blocking file i/o, allocating
> memory with malloc/free, CRT functions that manipulate global state,
> etc.

RE: [GD-General] responsive threading

[Mat N. \(BUNGIE\) <mat...@mi...>]

Unfortunately, there are many function calls that can cause stalls under
multithreading: mixing blocking and non-blocking file i/o, allocating
memory with malloc/free, CRT functions that manipulate global state,
etc.

MSN
-----Original Message-----
From: gam...@li...
[mailto:gam...@li...] On Behalf Of
Andras Balogh
Sent: Friday, July 29, 2005 3:07 PM
To: gam...@li...
Subject: Re: [GD-General] responsive threading

I understand that, and I am thankful for your suggestions, but the=20
problem is that I don't really know what's going on in this=20
decompression library myself.. I was just assuming that whatever it=20
does, there is a way to make it not disturb my main thread.


Andras


Mat Noguchi (BUNGIE) wrote:
> Without knowing exactly what you are doing on both threads, it is very
> hard to determine the source of framerate spikes that are caused by
> multithreading.
>=20
> MSN
> -----Original Message-----
> From: gam...@li...
> [mailto:gam...@li...] On Behalf Of
> Andras Balogh
> Sent: Friday, July 29, 2005 2:34 PM
> To: gam...@li...
> Subject: Re: [GD-General] responsive threading
>=20
> Even if this was the case, it would hurt overall performance, no?=20
> Overall performance is very good, I just don't like the occasional
> spikes...
>=20
>=20
> Mat Noguchi (BUNGIE) wrote:
>=20
>>If you are both reading and decompressing on the IO thread, you are
>>likely causing a lot of context switches if you have not modified the
>>priority on either thread.
>>
>>MSN
>=20


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. =
http://ads.osdn.com/?ad_id=3D7477&alloc_id=3D16492&op=3Dclick
_______________________________________________
Gamedevlists-general mailing list
Gam...@li...
https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_id=3D557

Re: [GD-General] responsive threading

[Andras B. <and...@gm...>]

I understand that, and I am thankful for your suggestions, but the 
problem is that I don't really know what's going on in this 
decompression library myself.. I was just assuming that whatever it 
does, there is a way to make it not disturb my main thread.


Andras


Mat Noguchi (BUNGIE) wrote:
> Without knowing exactly what you are doing on both threads, it is very
> hard to determine the source of framerate spikes that are caused by
> multithreading.
> 
> MSN
> -----Original Message-----
> From: gam...@li...
> [mailto:gam...@li...] On Behalf Of
> Andras Balogh
> Sent: Friday, July 29, 2005 2:34 PM
> To: gam...@li...
> Subject: Re: [GD-General] responsive threading
> 
> Even if this was the case, it would hurt overall performance, no? 
> Overall performance is very good, I just don't like the occasional
> spikes...
> 
> 
> Mat Noguchi (BUNGIE) wrote:
> 
>>If you are both reading and decompressing on the IO thread, you are
>>likely causing a lot of context switches if you have not modified the
>>priority on either thread.
>>
>>MSN
> 

RE: [GD-General] responsive threading

[Mat N. \(BUNGIE\) <mat...@mi...>]

Without knowing exactly what you are doing on both threads, it is very
hard to determine the source of framerate spikes that are caused by
multithreading.

MSN
-----Original Message-----
From: gam...@li...
[mailto:gam...@li...] On Behalf Of
Andras Balogh
Sent: Friday, July 29, 2005 2:34 PM
To: gam...@li...
Subject: Re: [GD-General] responsive threading

Even if this was the case, it would hurt overall performance, no?=20
Overall performance is very good, I just don't like the occasional
spikes...


Mat Noguchi (BUNGIE) wrote:
> If you are both reading and decompressing on the IO thread, you are
> likely causing a lot of context switches if you have not modified the
> priority on either thread.
>=20
> MSN


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. =
http://ads.osdn.com/?ad_id=3D7477&alloc_id=3D16492&op=3Dclick
_______________________________________________
Gamedevlists-general mailing list
Gam...@li...
https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_id=3D557