Thread: [GD-General] worker threads, level loading and progress bars
Brought to you by:
vexxed72
From: <cas...@ya...> - 2002-08-07 02:21:55
|
Hi, to display a progress bar during the level loads, I've always used somekind of callback mechanism, or a simple 'DoWork' function that performs a little part of the work in each call and returns a special value when the work has been finished. Those approaches worked ok, but the progress of the bar wasn't smooth and the resulting code was not very clean: int DoWork( int prev_step ) { if( prev_step==0 ) { "do some work" return ++prev_step; } else if( prev_step==1 ) { ... } ... else return finish_step; } I've been thinking about moving to a multithreaded approach. This worked nicely until I tried to load textures, that of course didn't work because I had to create a new rendering context for the new thread. My system and specially the render was not designed with multithreading in mind, so I've noticed that this approach would need many changes in my current design. (creating multiple rendering context, locking and unlocking the render, etc.). Loading and uploading textures is almost half of the job, so unless I change my architecture, the threaded approach doesn't seem very usefull, since I still have to use the previous dirty work partition. Instead of using multiple threads I could also try to use the current method, but cleanly. For example by using an abstract job class, and a job queue that manages and process the pending jobs. Decomposing all the work in simple jobs, childs of the base job class and created by the job queue like in a parametrized factory. Queueing the jobs at the beginning and processing them secuentially on each 'DoWork' call. But before doing so, I would like to know what other people are doing. Thanks in advance, Ignacio Castaño cas...@ya... _______________________________________________________________ Yahoo! Messenger Nueva versión: Webcam, voz, y mucho más ¡Gratis! Descárgalo ya desde http://messenger.yahoo.es |
From: <cas...@ya...> - 2002-08-07 02:38:10
|
Hmm... it seems to my now, that the problem with the multiple threads could be easily solved. The only OpenGL operation performed by the worker thread is to upload textures, so my texture manager could have a list of textures waiting to be uploaded, and I could process them on each game loop. Anyway, I still would like to hear how others have solve this problem. Ignacio Castaño cas...@ya... Ignacio Castaño wrote: > Hi, > to display a progress bar during the level loads, I've always used somekind > of callback mechanism, or a simple 'DoWork' function that performs a little > part of the work in each call and returns a special value when the work has > been finished. Those approaches worked ok, but the progress of the bar > wasn't smooth and the resulting code was not very clean: > ... _______________________________________________________________ Copa del Mundo de la FIFA 2002 El único lugar de Internet con vídeos de los 64 partidos. ¡Apúntante ya! en http://fifaworldcup.yahoo.com/fc/es/ |
From: Thatcher U. <tu...@tu...> - 2002-08-07 03:30:42
|
I took a stab at this (loading in a background thread) in Soul Ride. In Win32, it's pretty easy to set up a new OpenGL render context, and use wglShareLists() so that textures defined in one thread are usable in the other thread. In captivity it worked, pretty well in fact. But at the time (early/mid 2000, IIRC) it was highly vulnerable to bugs in driver thread support, so I gave up and fell back on the DoWork() pattern. Under X, there seems to be a way to share textures across threads, but I never got around to trying it. Nowadays I try to use SDL for all video setup & config, and it does not have a facility like wglShareLists :( The worker queue thing works as well; my Chunked LOD demo uses this pattern -- I load the image data in the loader thread, and bind it in the foreground thread. But it still means some extra, unnecessary work for the foreground thread; plus the coding work. -Thatcher On Aug 07, 2002 at 04:44 +0200, Ignacio Casta?o wrote: > Hmm... it seems to my now, that the problem with the multiple threads could > be easily solved. > > The only OpenGL operation performed by the worker thread is to upload > textures, so my texture manager could have a list of textures waiting to be > uploaded, and I could process them on each game loop. > > Anyway, I still would like to hear how others have solve this problem. > > > Ignacio Casta?o > cas...@ya... > > > Ignacio Casta?o wrote: > > Hi, > > to display a progress bar during the level loads, I've always used > somekind > > of callback mechanism, or a simple 'DoWork' function that performs a > little > > part of the work in each call and returns a special value when the work > has > > been finished. Those approaches worked ok, but the progress of the bar > > wasn't smooth and the resulting code was not very clean: > > ... > > > _______________________________________________________________ > Copa del Mundo de la FIFA 2002 > El ?nico lugar de Internet con v?deos de los 64 partidos. > ?Ap?ntante ya! en http://fifaworldcup.yahoo.com/fc/es/ > > > ------------------------------------------------------- > This sf.net email is sponsored by:ThinkGeek > Welcome to geek heaven. > http://thinkgeek.com/sf > _______________________________________________ > Gamedevlists-general mailing list > Gam...@li... > https://lists.sourceforge.net/lists/listinfo/gamedevlists-general > Archives: > http://sourceforge.net/mailarchive/forum.php?forum_id=557 -- Thatcher Ulrich http://tulrich.com |
From: Javier A. <ja...@py...> - 2002-08-07 07:27:05
|
My approach has always been to have a global subsystem with the following API: ProgressBar::Init() ProgressBar::StartSection(percent, numTicks) ProgressBar::Progress() which calls ProgressBar::Render() Then spread calls to Progress() around the calls that load stuff. This is probably the callback mechanism Ignacio mentioned, but I don't like the idea of passing the "ProgressCallback" object around these calls - it makes the API uglier without need. Proper adjustment of the "percent" and "numTicks" of each section for each level, can make it quite smooth. (Log times and number of actual calls, for each level load, then put those into the config file, log again to check and you're set). Advancing the progress bar based on some realtime clock will give you trouble. Render() decides if it's a good time to update the visual progress bar depending on how frequently do you . (Note: when optimizing load times on "Commandos: Beyond The Call Of Duty", this one showed up as taking 50% of the load time - it wasn't being smart about when to update and was causing 1000's of PageFlip()s) This system is terribly simple, doesn't require any fancy technologies, plus it doesn't bind knowledge of the internals of any subsystem to any other, so it's architecturaly as clean as you can get. Fancy solutions like multithreading may be more interesting but are so much less practical in all respects... If you want a console-style continuously animated progress bar then you need either more calls to Progress() or a secondary thread. The single-threaded approach may be slightly less efficient, but much easier and safer to implement. Thatcher Ulrich <tu...@tu...> wrote: > I took a stab at this (loading in a background thread) in Soul Ride. > In Win32, it's pretty easy to set up a new OpenGL render context, and > use wglShareLists() so that textures defined in one thread are usable > in the other thread. > > In captivity it worked, pretty well in fact. But at the time > (early/mid 2000, IIRC) it was highly vulnerable to bugs in driver > thread support, so I gave up and fell back on the DoWork() pattern. > > Under X, there seems to be a way to share textures across threads, but > I never got around to trying it. Nowadays I try to use SDL for all > video setup & config, and it does not have a facility like > wglShareLists :( > > The worker queue thing works as well; my Chunked LOD demo uses this > pattern -- I load the image data in the loader thread, and bind it in > the foreground thread. But it still means some extra, unnecessary > work for the foreground thread; plus the coding work. > > -Thatcher > > On Aug 07, 2002 at 04:44 +0200, Ignacio Casta?o wrote: >> Hmm... it seems to my now, that the problem with the multiple >> threads could be easily solved. >> >> The only OpenGL operation performed by the worker thread is to upload >> textures, so my texture manager could have a list of textures >> waiting to be uploaded, and I could process them on each game loop. >> >> Anyway, I still would like to hear how others have solve this >> problem. >> >> >> Ignacio Casta?o >> cas...@ya... >> >> >> Ignacio Casta?o wrote: >>> Hi, >>> to display a progress bar during the level loads, I've always used >>> somekind of callback mechanism, or a simple 'DoWork' function that >>> performs a little part of the work in each call and returns a >>> special value when the work has been finished. Those approaches >>> worked ok, but the progress of the bar wasn't smooth and the >>> resulting code was not very clean: ... >> >> >> _______________________________________________________________ >> Copa del Mundo de la FIFA 2002 >> El ?nico lugar de Internet con v?deos de los 64 partidos. >> ?Ap?ntante ya! en http://fifaworldcup.yahoo.com/fc/es/ >> >> >> ------------------------------------------------------- >> This sf.net email is sponsored by:ThinkGeek >> Welcome to geek heaven. >> http://thinkgeek.com/sf >> _______________________________________________ >> Gamedevlists-general mailing list >> Gam...@li... >> https://lists.sourceforge.net/lists/listinfo/gamedevlists-general >> Archives: >> http://sourceforge.net/mailarchive/forum.php?forum_id=557 > > ------------------------------------------------------- > This sf.net email is sponsored by:ThinkGeek > Welcome to geek heaven. > http://thinkgeek.com/sf > _______________________________________________ > Gamedevlists-general mailing list > Gam...@li... > https://lists.sourceforge.net/lists/listinfo/gamedevlists-general > Archives: > http://sourceforge.net/mailarchive/forum.php?forum_id=557 Javier Arevalo Pyro Studios |
From: <cas...@ya...> - 2002-08-07 15:57:39
|
Well, thanks for all the input. I'll finally stick with the threaded mechanism, with the loader thread loading the textures into memory, and the main thread uploading them to vram. The thing I like about this approach is that it can be extended with ease to load resources during the game without too much interruption. I still have a few calls to the progress bar in the worker thread, to update the progress position, but it's interpolated nicely in the main thread. My worker thread has a higher priority than the main thread, (above_normal in win2k and highest in win9x). So that it doesn't take only 50% of the loading time. This seems to work ok, but will probably also add some signals, to ensure that the worker thread doesn't take all the time. Ignacio Castaño cas...@ya... > Thatcher Ulrich <tu...@tu...> wrote: > > I took a stab at this (loading in a background thread) in Soul Ride. > > In Win32, it's pretty easy to set up a new OpenGL render context, and > > use wglShareLists() so that textures defined in one thread are usable > > in the other thread. > > > > In captivity it worked, pretty well in fact. But at the time > > (early/mid 2000, IIRC) it was highly vulnerable to bugs in driver > > thread support, so I gave up and fell back on the DoWork() pattern. > > > > Under X, there seems to be a way to share textures across threads, but > > I never got around to trying it. Nowadays I try to use SDL for all > > video setup & config, and it does not have a facility like > > wglShareLists :( > > > > The worker queue thing works as well; my Chunked LOD demo uses this > > pattern -- I load the image data in the loader thread, and bind it in > > the foreground thread. But it still means some extra, unnecessary > > work for the foreground thread; plus the coding work. > > > > -Thatcher > > > > On Aug 07, 2002 at 04:44 +0200, Ignacio Casta?o wrote: > >> Hmm... it seems to my now, that the problem with the multiple > >> threads could be easily solved. > >> > >> The only OpenGL operation performed by the worker thread is to upload > >> textures, so my texture manager could have a list of textures > >> waiting to be uploaded, and I could process them on each game loop. > >> > >> Anyway, I still would like to hear how others have solve this > >> problem. > >> > >> _______________________________________________________________ Yahoo! Messenger Nueva versión: Webcam, voz, y mucho más ¡Gratis! Descárgalo ya desde http://messenger.yahoo.es |
From: Corrinne Y. <cor...@el...> - 2002-08-20 02:06:03
|
I have a few questions for all of you. I would be grateful to hear from all your varied experience. Which physics structures do you always server broadcast and arbitrate? Which physics structures do you occasionally broadcast? Which physics structures do you client predict most of the time, and server arbitrate occasionally? Which physics structures do you only client predict and never worry about synchronizing? What about trying to do it in a peer to peer model instead of a full roundtrip of server client? Which physics structures do you think can get away with in peer to peer? Do you think an entire "simple death match" MP game can get away with it being peer to peer? What kind of things must you always arbitrate by a server? Can one "design" the game play out of any necessary server arbitration? It would save a lot of data structure bookkeeping if each client can become more powerful (and the server to arbitrate almost nothing, or just not exist at all). The bookkeeping isn't bad for simple player position, rotation, etc. But chains of complex physics reaction becomes quite a pain of the neck to be maintained and received from a "remote" arbitrator all the time. :( It easily "doubles" the code to type to have the server minds itself with so many complex collisions arbitrations. :( |
From: Cruise <cr...@ca...> - 2002-08-20 10:30:31
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: MD5 > I have a few questions for all of you. I would be grateful to hear from > all your varied experience. [snip] > Do you think an entire "simple death match" MP game can get away with it > being peer to peer? Depends on how many players your game will have. The current multiplayer trend, especially with the rise in team based multiplayer, is getting as many people into a match as possible. An upper limit of 64 players is not uncommon now. That's clearly unworkable in a full peer-to-peer system except on a LAN. > What kind of things must you always arbitrate by a server? Can one > "design" the game play out of any necessary server arbitration? > It would save a lot of data structure bookkeeping if each client can > become more powerful (and the server to arbitrate almost nothing, or > just not exist at all). Though that would be lovely from a bandwidth and latency point of view, it's just not practical in real terms. The more authority the client has, the more potential there is for someone to abuse it. The lesson from Half-Life is that if your game has any decent number of players, then some of those players will be cheating. And they'll be damn ingenious cheats too. The simple matter is that you can't trust client software anymore. You have to have an authoritative state-of-play machine. > The bookkeeping isn't bad for simple player position, rotation, etc. But > chains of complex physics reaction becomes quite a pain of the neck to > be maintained and received from a "remote" arbitrator all the time. :( > It easily "doubles" the code to type to have the server minds itself > with so many complex collisions arbitrations. :( Agreed. It is a shame. But the enviroment in which the code is deployed makes it necessary. [ cruise / www.casual-tempest.net / www.transference.org ] -----BEGIN PGP SIGNATURE----- Version: 2.6 iQCVAwUAPWIaMvdi0Z5STRufAQH9SwP/RjWxxk7NOpXRQtZITx1RyoK99QfRcDyt ucX4rZtbe4IFikLE4b1Tgla9hyWPq4xDEPaQlLDmhZxz2gBpjmFmBHKSovrHni3l xEYV7/G963QbKsmT+rT4hbZiuh4MeJB9/SiGC4VwzNYtjbSau17LbPIcGrzITW6m EHpVPDUtZKA= =FV3n -----END PGP SIGNATURE----- |
From: Corrinne Y. <cor...@el...> - 2002-08-20 23:54:20
|
> The bookkeeping isn't bad for simple player position, rotation, etc. But > chains of complex physics reaction becomes quite a pain of the neck to > be maintained and received from a "remote" arbitrator all the time. :( > It easily "doubles" the code to type to have the server minds itself > with so many complex collisions arbitrations. :( -- I wonder what most of you decide to do in your code regarding physics chain reactions of moving objects. -- Does server arbitrate all large to small physics, which rock or which object gets pushed or bounced or rag-dolled to which positions? -- Player positions rotation, even player opening doors, isn't such a big deal. Player motion that causes motion and position changes on a chain of other objects is a big deal. -- In a perfect world, every position of every object that moves must be arbitrated by server. Any object that has the potential of blocking or bouncing or twirling or pushing another object which eventually blocks a player must be centrally arbitrated. -- In the non-perfect world, the number of arbitrating entities becomes really large the more an application displays physics-ness in the world. Do all of you really arbitrate and (continually) broadcast all these moving dynamic objects big and small 's absolute positions and not client predict them at all? -- Or do you client predict some and "backtrack" when the server disagrees with the prediction? What are your backtrack methods if you do client predict. -- I gratefully await your experience in these areas. |
From: Cruise <cr...@ca...> - 2002-08-22 12:20:41
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: MD5 > -- I wonder what most of you decide to do in your code regarding physics > chain reactions of moving objects. > -- Does server arbitrate all large to small physics, which rock or which > object gets pushed or bounced or rag-dolled to which positions? If it could potentially affect another client, then yes. "Effects", anything that is mere eye-candy, can be generated entirely on the client. Explosions, bullet effects, decals, etc. > -- Player positions rotation, even player opening doors, isn't such a > big deal. Player motion that causes motion and position changes on a > chain of other objects is a big deal. > -- In a perfect world, every position of every object that moves must be > arbitrated by server. Any object that has the potential of blocking or > bouncing or twirling or pushing another object which eventually blocks a > player must be centrally arbitrated. Correct. > -- In the non-perfect world, the number of arbitrating entities becomes > really large the more an application displays physics-ness in the world. > Do all of you really arbitrate and (continually) broadcast all these > moving dynamic objects big and small 's absolute positions and not > client predict them at all? Fortunately, scale works for as well as against. You only need to tell each client about the things within the PVS. Naturally, if you're game world is a huge flat savannah, then the PVS will be quite large. But, hopefully, the game world will be correspondingly larger, and the physics "density" (the number of things that need to be tracked) less. Also, many things are deterministic (or have known random seeds), and don't need to be broadcast continually. Once the intial angle and velocity of a grenade is known, for example, the clients can /usually/ work out the path themselves. That doesn't always hold true, obviously, especially if you have a highly interactive enviroment. However, you can get away with a lower broadcast frequency. Player states are the most important. You can predict most everything else "good enough" as long as you know what the players are doing. > -- Or do you client predict some and "backtrack" when the server > disagrees with the prediction? What are your backtrack methods if you do > client predict. Yes. That way the player can play quite happily until the next broadcast. The client can work on it's own data until such time as the server's becomes available. In Half-Life, backtracking simply resets the positions to the server's version. Because there tends to be little in the way of moving objects, and fewer of any size or speed, this is an acceptable method. Every so often a player will "flicker", but people are used to that online. If speeds are high, or many noticable objects are being predicated, then you require something more subtle, such as bringing the positions into line over several frames. This can result in things rolling up hill, and other weirdness, so it isn't ideal either. There isn't really a good answer, unfortunately. There is another difficult choice, too. An uncorrected difference can mean enough motion to take someone out of cover on the client, while in fact they stopped. If the local player then shoots this projected version, what do you do? If you use the client's version, players complain about being shot "round corners". If you use the servers version, then players complain about seemingly "invulnerable" opponents. Again, Half-Life recently switched to the first method from the second, as in general, it makes life much easier for clients on slower connections. They don't have to worry about "leading" or second-guessing other players so much, as if they shoot the version on their machine, then it counts as a hit, even if the player wasn't in actuality "there". > -- I gratefully await your experience in these areas. [ cruise / www.casual-tempest.net / www.transference.org ] -----BEGIN PGP SIGNATURE----- Version: 2.6 iQCVAwUAPWTXG/di0Z5STRufAQEhMQP/ZFXsb04tFoA+eulkpMSICWvaF/uIwqIi rbbczKjsgi9kiTZQoGR9tW6HN9Ah1tU7hJKwJymOsx9noyHZ/nS0ELdstflFO/yr Q+8ITO+w5ykQSeCh76PH4Ia1+TdtKitID1FVjD6i0to9ZKKqqTTZXxhxwiX2NiEm elllfbtF3Ng= =xCgb -----END PGP SIGNATURE----- |
From: Aaron D. <ri...@in...> - 2002-08-24 02:29:42
|
> The simple matter is that you can't trust client software anymore. You > have to have an authoritative state-of-play machine. Urrr. Not necessarily. In a lock-step peer-to-peer setup (such as the original doom which just broadcast players controls to other players), cheating is still not possible unless All players cheat as the game would otherwise get out of sync on the cheaters PC. - Aaron |