Menu
▾
▴
quesa-develop
|
From: Roger H. <rog...@mi...> - 2007-02-25 11:59:13
|
On 24 Feb, 2007, at 03:11, James Walker wrote: > Roger Holmes wrote: > >> More optimisation. Especially I would like to reduce going all >> around the houses for a group to submit the groups within it. >> One recursive routine in the stack trace is unavoidable, but at >> the moment we have three routines repeatedly calling each >> other. One knows what the submission mode is e.g Picking >> but the other ones keep checking the mode, which must be >> slowing us down. There has got to be a better way. > > > I'm not sure which 3 routines you're talking about here. When you > submit a group for rendering, the call stack looks something like > > Q3Object_Submit > E3Object_Submit > E3View_SubmitRetained > e3view_submit_retained_render > e3group_display_submit_contents > e3group_submit_contents > Q3Object_Submit Yes, in the debugger that's right. If you compile a version for profiling (names on, optimisations on), then profile it with Shark, only three of these show up. I guess some get optimised out. As far as optimisation goes then it is just the three which matter, but of course it would be nicer when debugging to not have all of these calls showing up. > > I say "something like" because if you subclass the view object, as > Geom > Test does, you have the subclass submit-retained-render method in > there. > It's true that two of these, e3group_display_submit_contents and > e3group_submit_contents, check the view mode, but > E3View_GetViewMode is > just an accessor and should be quite fast. But there's the switch statement. There is the call to E3DisplayGroup::GetState, there is the checking of its success/failure, and it only ever returns success. There is loads of code which does nothing constructive towards getting the rendering/picking or whatever done, its just administrative red tape. > > One way I see to shorten the cycle is to have e3group_submit_contents > call E3View_SubmitRetained directly instead of calling > Q3Object_Submit. > That looks safe, so I'll go ahead and commit it. Yes that's a step in the right direction. > > Also, is anyone using kQ3CallbackElementTypeBeforeRender and > kQ3CallbackElementTypeAfterRender, which are checked every time > through > e3view_submit_retained_render? Not me either. > I put those in a few years back, but I don't use them any more, as I > decided that I can do more with less trouble by subclassing the > view object. Sounds good, I was thinking about having one set of routines for when there was one and another set for when there wasn't, but if we can get rid of them completely then thats even better. How about this: e3group_display_metahandler returns the same routine for the four instances case kQ3XMethodTypeObjectSubmitBounds: case kQ3XMethodTypeObjectSubmitPick: case kQ3XMethodTypeObjectSubmitRender: case kQ3XMethodTypeObjectSubmitWrite: These could be four separate routines, call them e3group_display_submit_contents_Bounds e3group_display_submit_contents_Pick e3group_display_submit_contents_Render e3group_display_submit_contents_Write These would know the view mode and so would not have to get it or switch on it. Roger. |
|
From: Roger H. <rog...@mi...> - 2007-02-26 19:41:54
|
> How about this: > > e3group_display_metahandler returns the same routine for the four > instances > case kQ3XMethodTypeObjectSubmitBounds: > case kQ3XMethodTypeObjectSubmitPick: > case kQ3XMethodTypeObjectSubmitRender: > case kQ3XMethodTypeObjectSubmitWrite: > > These could be four separate routines, call them > e3group_display_submit_contents_Bounds > e3group_display_submit_contents_Pick > e3group_display_submit_contents_Render > e3group_display_submit_contents_Write > These would know the view mode and so would > not have to get it or switch on it. Sorry about replying to my own e-mails, but in the same way as above, e3group_metahandler returns the same routine for three instances case kQ3XMethodTypeObjectSubmitBounds: case kQ3XMethodTypeObjectSubmitPick: case kQ3XMethodTypeObjectSubmitRender: These can also be usefully separated. I did not say in the original email, but the separated instances can then be optimised by removing the checks on the mode, and eliminating the stuff which is not required, which makes the routines smaller and hence more likely to remain in the cache as well as not executing so many instructions. One other thing I noticed, E3Bit_IsSet would probably be very slightly quicker coded as ((TQ3Boolean) (((_bf) & (_b)) != 0)) rather than ((TQ3Boolean) (((_bf) & (_b)) == (_b))) Alternatively, just use E3Bit_AnySet instead, and maybe rename E3Bit_IsSet to be E3Bit_AllSet as there nothing stopping you passing the first parameter as multiple bits. Roger. |
|
From: James W. <ja...@fr...> - 2007-02-27 03:08:51
|
I was looking at the view state stack, and wondered if we really need the attributeSet member of TQ3ViewStackItem. It causes a Q3Object_Duplicate call every time we push the stack, and it looks like it's only there to support Q3View_GetAttributeState and Q3View_GetAttributeSetState. Internally, we only call Q3View_GetAttributeSetState when writing view hints, and never call Q3View_GetAttributeState. So I would think that whenever Q3View_GetAttributeState or Q3View_GetAttributeSetState is called, we could just build an attribute set on the fly from the various attribute members of TQ3ViewStackItem. Anyone see a problem with that? -- James W. Walker, Innoventive Software LLC <http://www.frameforge3d.com/> |
|
From: Roger H. <rog...@mi...> - 2007-02-27 11:24:20
|
On 27 Feb, 2007, at 03:08, James Walker wrote: > I was looking at the view state stack, and wondered if we really need > the attributeSet member of TQ3ViewStackItem. It causes a > Q3Object_Duplicate call every time we push the stack, and it looks > like > it's only there to support Q3View_GetAttributeState and > Q3View_GetAttributeSetState. Internally, we only call > Q3View_GetAttributeSetState when writing view hints, and never call > Q3View_GetAttributeState. So I would think that whenever > Q3View_GetAttributeState or Q3View_GetAttributeSetState is called, we > could just build an attribute set on the fly from the various > attribute > members of TQ3ViewStackItem. Anyone see a problem with that? I am not sure how much generality we would lose by doing this. I did come across a program which included custom attributes in its data which I picked up on in the Microspot Renderer. I do use lots of custom attributes but our ones do not affect the rendering. I would rather use custom groups or custom geometries to do those sorts of things. I think we can check if there any other attributes which we don't understand. If there are some then we proceed as now, otherwise we do as you suggest. Hopefully the former case will be very rare. Maybe we could have a flag to say if there are currently any custom attributes there, and only do the full check when the attribute set gets changed. I agree we do not want to be calling Q3Object_Duplicate for every group in the data. Roger. |
|
From: James W. W. <os...@jw...> - 2007-02-27 17:34:38
|
On Feb 27, 2007, at 3:23 AM, Roger Holmes wrote: > On 27 Feb, 2007, at 03:08, James Walker wrote: > >> I was looking at the view state stack, and wondered if we really need >> the attributeSet member of TQ3ViewStackItem. It causes a >> Q3Object_Duplicate call every time we push the stack, and it looks >> like >> it's only there to support Q3View_GetAttributeState and >> Q3View_GetAttributeSetState. Internally, we only call >> Q3View_GetAttributeSetState when writing view hints, and never call >> Q3View_GetAttributeState. So I would think that whenever >> Q3View_GetAttributeState or Q3View_GetAttributeSetState is called, we >> could just build an attribute set on the fly from the various >> attribute >> members of TQ3ViewStackItem. Anyone see a problem with that? > > > I am not sure how much generality we would lose by doing this. I did > come across a program which included custom attributes in its data > which I > picked up on in the Microspot Renderer. I do use lots of custom > attributes but our ones do not affect the rendering. I would rather > use custom > groups or custom geometries to do those sorts of things. > > I think we can check if there any other attributes which we don't > understand. If there are some then we proceed as now, otherwise we > do as you > suggest. Hopefully the former case will be very rare. Maybe we > could have a > flag to ay if there are currently any custom attributes there, and > only do > the full check when the attribute set gets changed. > > I agree we do not want to be calling Q3Object_Duplicate for every > group > in the data. I'm not sure how any custom attributes would get in that attribute set. It looks like the only way an attribute is added to that set is by e3view_stack_update_attribute, which is called by e3view_stack_update only for specific known attribute types. |
|
From: Roger H. <rog...@mi...> - 2007-02-27 18:55:58
|
On 27 Feb, 2007, at 17:34, James W. Walker wrote: > > On Feb 27, 2007, at 3:23 AM, Roger Holmes wrote: > >> On 27 Feb, 2007, at 03:08, James Walker wrote: >> >>> I was looking at the view state stack, and wondered if we really >>> need >>> the attributeSet member of TQ3ViewStackItem. It causes a >>> Q3Object_Duplicate call every time we push the stack, and it looks >>> like >>> it's only there to support Q3View_GetAttributeState and >>> Q3View_GetAttributeSetState. Internally, we only call >>> Q3View_GetAttributeSetState when writing view hints, and never call >>> Q3View_GetAttributeState. So I would think that whenever >>> Q3View_GetAttributeState or Q3View_GetAttributeSetState is >>> called, we >>> could just build an attribute set on the fly from the various >>> attribute >>> members of TQ3ViewStackItem. Anyone see a problem with that? >> >> >> I am not sure how much generality we would lose by doing this. I did >> come across a program which included custom attributes in its data >> which I >> picked up on in the Microspot Renderer. I do use lots of custom >> attributes but our ones do not affect the rendering. I would rather >> use custom >> groups or custom geometries to do those sorts of things. >> >> I think we can check if there any other attributes which we don't >> understand. If there are some then we proceed as now, otherwise we >> do as you >> suggest. Hopefully the former case will be very rare. Maybe we >> could have a >> flag to ay if there are currently any custom attributes there, and >> only do >> the full check when the attribute set gets changed. >> >> I agree we do not want to be calling Q3Object_Duplicate for every >> group >> in the data. > > I'm not sure how any custom attributes would get in that attribute > set. It looks like the only way an attribute is added to that set is > by e3view_stack_update_attribute, which is called by > e3view_stack_update only for specific known attribute types. Well that's fine then. I thought there was a way of adding attributes to the view object itself and that was what we were talking about. So carry on. Roger. |
|
From: James W. <ja...@fr...> - 2007-02-27 20:07:02
|
Roger Holmes wrote: > On 27 Feb, 2007, at 17:34, James W. Walker wrote: >> I'm not sure how any custom attributes would get in that attribute >> set. It looks like the only way an attribute is added to that set is >> by e3view_stack_update_attribute, which is called by >> e3view_stack_update only for specific known attribute types. > > Well that's fine then. I thought there was a way of adding attributes > to the view object itself and that was what we were talking about. > > So carry on. A view does have a default attribute set (Q3View_SetDefaultAttributeSet) that gets submitted at the start of a submitting loop. Maybe that's what you were thinking of. -- James W. Walker, Innoventive Software LLC <http://www.frameforge3d.com/> |
|
From: Roger H. <rog...@mi...> - 2007-02-28 12:06:21
|
On 27 Feb, 2007, at 20:06, James Walker wrote: > A view does have a default attribute set > (Q3View_SetDefaultAttributeSet) > that gets submitted at the start of a submitting loop. Maybe that's > what you were thinking of. Yes that must be it. |
|
From: James W. <ja...@fr...> - 2007-02-27 20:48:22
|
James Walker wrote: > I was looking at the view state stack, and wondered if we really need > the attributeSet member of TQ3ViewStackItem. It causes a > Q3Object_Duplicate call every time we push the stack, and it looks like > it's only there to support Q3View_GetAttributeState and > Q3View_GetAttributeSetState. Internally, we only call > Q3View_GetAttributeSetState when writing view hints, and never call > Q3View_GetAttributeState. So I would think that whenever > Q3View_GetAttributeState or Q3View_GetAttributeSetState is called, we > could just build an attribute set on the fly from the various attribute > members of TQ3ViewStackItem. Anyone see a problem with that? OK, I've checked in a version of E3View.c in which attributeSet is gone, and I saw about a 3% increase in frame rate for the Multibox test in Geom Test. -- James W. Walker, Innoventive Software LLC <http://www.frameforge3d.com/> |
|
From: Roger H. <rog...@mi...> - 2007-02-28 12:07:22
|
On 27 Feb, 2007, at 20:48, James Walker wrote: > > OK, I've checked in a version of E3View.c in which attributeSet is > gone, > and I saw about a 3% increase in frame rate for the Multibox test in > Geom Test. That's great. Thank you. |
|
From: James W. <ja...@fr...> - 2007-03-12 19:54:59
|
Roger Holmes wrote (on 2/17/07): > I would like to go back to being able to store lights in display groups, > preferably so that when their group gets submitted, they get added > to the light group so that I would no longer have to do a pre-pass > to read all the lights and add them to the light group myself with > their correct transforms and then afterward remove them from the > light group. Lights are real things in the real world, we do not need > to do anything with them until EndRender time, so why do they > have to be known before StartRender time? While I agree that it would be nice to be able to have lights anywhere in the scene, I don't know how to do it without two passes. I don't know what you mean by "go back to being able to store lights in display groups". When was this possible? The QD3D manual says: "A display group is a group of objects that are drawable. Drawable objects include geometric objects, styles, transforms, attributes and attribute sets, and other display groups." No mention of lights. In answer to the question "why do they have to be known before StartRender time", my answer is: because any OpenGL-based renderer needs to have the lights set up before you render anything that should be lit by said lights. -- James W. Walker, Innoventive Software LLC <http://www.frameforge3d.com/> |
|
From: Roger H. <rog...@mi...> - 2007-03-13 00:42:24
|
On 12 Mar, 2007, at 19:54, James Walker wrote: > Roger Holmes wrote (on 2/17/07): > >> I would like to go back to being able to store lights in display >> groups, >> preferably so that when their group gets submitted, they get added >> to the light group so that I would no longer have to do a pre-pass >> to read all the lights and add them to the light group myself with >> their correct transforms and then afterward remove them from the >> light group. Lights are real things in the real world, we do not need >> to do anything with them until EndRender time, so why do they >> have to be known before StartRender time? > > While I agree that it would be nice to be able to have lights anywhere > in the scene, I don't know how to do it without two passes. > > I don't know what you mean by "go back to being able to store > lights in > display groups". When was this possible? The QD3D manual says: "A > display group is a group of objects that are drawable. Drawable > objects > include geometric objects, styles, transforms, attributes and > attribute > sets, and other display groups." No mention of lights. I think the manual was published after version 1.0 shipped. We had a series of much more detailed specs up to that time (I wish I still had them - some tidy person threw them away because 'we have the proper manual now'). I think it was about version 1.2 that Apple made display groups reject the adding of a light. Not that they ever had any effect, I always had to do a rendering pre-pass. Now I have to do pre-passes before rendering, before saving (to convert lights to custom attributes and custom light groups to display groups) and do a post pass after reading to convert them back again. I also do special processing during the finding of bounding boxes and bounding balls so that they include the lights, and have to have light widgets so that picking works. > > In answer to the question "why do they have to be known before > StartRender time", my answer is: because any OpenGL-based renderer > needs > to have the lights set up before you render anything that should be > lit > by said lights. Well if that is a restriction of OpenGL then fair enough. I just think that no processing of lights can be done until a (non shadow) renderer has the Z buffer set up with depths and references to the triangles, and whilst the renderer builds the list of triangles (and culls some) it could have set up the list of lights whilst it parsed the group structures. |
|
From: James W. <ja...@fr...> - 2007-03-13 01:04:44
|
Roger Holmes wrote: >> In answer to the question "why do they have to be known before >> StartRender time", my answer is: because any OpenGL-based renderer >> needs >> to have the lights set up before you render anything that should be >> lit >> by said lights. > > Well if that is a restriction of OpenGL then fair enough. I just > think that no processing of lights can be done until a (non shadow) > renderer has the Z buffer set up with depths and references to the > triangles, and whilst the renderer builds the list of triangles (and > culls some) it could have set up the list of lights whilst it parsed > the group structures. Either I'm misunderstanding you, or you have a mental model of rendering that doesn't correspond to the way any of our built-in renderers work. There isn't any separate phase of setting up the Z buffer with depths and references to triangles. There's a continuous process of breaking triangles into fragments, updating the Z and color buffers with new fragment data, and so on. Once a particular triangle has passed through the pipeline, there is no opportunity to change the way it has been lit. -- James W. Walker, Innoventive Software LLC <http://www.frameforge3d.com/> |
|
From: Roger H. <rog...@mi...> - 2007-03-13 14:03:33
|
On 13 Mar, 2007, at 01:04, James Walker wrote: > Roger Holmes wrote: > >>> In answer to the question "why do they have to be known before >>> StartRender time", my answer is: because any OpenGL-based renderer >>> needs >>> to have the lights set up before you render anything that should be >>> lit >>> by said lights. >> >> Well if that is a restriction of OpenGL then fair enough. I just >> think that no processing of lights can be done until a (non shadow) >> renderer has the Z buffer set up with depths and references to the >> triangles, and whilst the renderer builds the list of triangles (and >> culls some) it could have set up the list of lights whilst it parsed >> the group structures. > > Either I'm misunderstanding you, or you have a mental model of > rendering > that doesn't correspond to the way any of our built-in renderers work. > There isn't any separate phase of setting up the Z buffer with depths > and references to triangles. There's a continuous process of breaking > triangles into fragments, updating the Z and color buffers with new > fragment data, and so on. Once a particular triangle has passed > through > the pipeline, there is no opportunity to change the way it has been > lit. No. My mental model is that of the plug-in (shadow) renderer we ship for Quesa, which I wrote. To speed up rendering, I do not do the slow calculations for the texturing, shading, lighting (and shadowing) of a z buffer pixel until I know it is front most. Game programmers have control of their data and can cull lots of stuff which is behind other items, but for fully user defined data this is not easy, probably impossible without using more time than it saves. So for each front most pixel there are dozens behind, and with user defined ordering of the data that means I can save doing the calculations for many of them. Roger. |
|
From: James W. <ja...@fr...> - 2007-03-13 17:01:16
|
Roger Holmes wrote: > No. My mental model is that of the plug-in (shadow) renderer we ship > for Quesa, which I wrote. To speed up rendering, I do not do the slow > calculations for the texturing, shading, lighting (and shadowing) of > a z buffer pixel until I know it is front most. Game programmers have > control of their data and can cull lots of stuff which is behind > other items, but for fully user defined data this is not easy, > probably impossible without using more time than it saves. So for > each front most pixel there are dozens behind, and with user defined > ordering of the data that means I can save doing the calculations for > many of them. Oh, interesting. Does your renderer use the GPU or is it all on the CPU? By the way, I notice that Quesa's contributors page has broken links for MicroSpot. Do you have replacements for them? -- James W. Walker, Innoventive Software LLC <http://www.frameforge3d.com/> |
|
From: Roger H. <rog...@mi...> - 2007-03-13 18:40:30
|
On 13 Mar, 2007, at 17:00, James Walker wrote: > Roger Holmes wrote: > >> No. My mental model is that of the plug-in (shadow) renderer we ship >> for Quesa, which I wrote. To speed up rendering, I do not do the slow >> calculations for the texturing, shading, lighting (and shadowing) of >> a z buffer pixel until I know it is front most. Game programmers have >> control of their data and can cull lots of stuff which is behind >> other items, but for fully user defined data this is not easy, >> probably impossible without using more time than it saves. So for >> each front most pixel there are dozens behind, and with user defined >> ordering of the data that means I can save doing the calculations for >> many of them. > > Oh, interesting. Does your renderer use the GPU or is it all on > the CPU? No it only uses the CPU. Version 3.6 used only one CPU but 4.0 will use them all. It is not an interactive renderer but it supports shadows and the quality is configurable to balance the complexity of the drawing, the available processing power, the time the user is willing to wait and the quality required. Turn off the bells and whistles and it could be used as an interactive renderer. A user visited us on Friday and showed us her 860,000 triangle model, and even the interactive renderer was struggling with it, and she was willing to wait several minutes for a fully rendered image, and she was using 3.6 under Rosetta! > > By the way, I notice that Quesa's contributors page has broken > links for > MicroSpot. Do you have replacements for them? > Could it please be changes to reference our home page www.microspot.co.uk Robin Landsbert no longer works for Microspot. I am quite happy to credit him, but his e-mail address is no longer valid. Maybe my name should be first anyway as I was always more senior than Robin and in view of my big speed up exercise a while back. Although we have an internal development build called 3D World, the issued version still is for OS9 on PowerPC (which uses real QuickDraw 3D). Our shipping OS-X products which use Quesa are called Microspot Interiors and Microspot Modeller. We are about to ship a 4.0 version of Microspot Interiors for Intel and PowerPC and then we will do the extra plugins to make a 4.0 version of Modeller too. The program icons will be changing too. |
|
From: James W. W. <os...@jw...> - 2007-02-18 19:33:10
|
On Feb 18, 2007, at 6:35 AM, Roger Holmes wrote: >> Could you give me an example or two of these APIs? Do you mean >> OpenGL proper, or AGL? > > Will do this when I'm back in the office. What is AGL, please excuse > my ignorance. AGL is the Carbon-specific API for OpenGL, for things like creating a rendering context. These function names all start with "agl". One case you might be thinking of is the FSAA support for ATI cards, which calls aglSetInteger. I never found any documentation for that FSAA feature unless you count messages in mailing list archives. >>> Multi-processor support (I have already added this to the Microspot >>> renderer but maybe it could be used in other area too). >> >> Is this pretty much the same as multi-threading support? > > Apple/Next have messed about with terminology here. I am not talking > about cooperative multi-threading on a single processor - no point for > Quesa. All the current Macs have have at least dual core Intel > processors > and often four cores in two processors, and the last top of the > line G5s > have quad processors too. It is nice to see the Activity Monitor > showing > 400% processor utilisation during rendering and have it finish in > about > a third of the time it used to. One of the slowest parts of the > interactive > renderer is currently the reformatting of textures ready for OpenGL. > This could be shared between processors. ... by using (preemptive) threads, presumably. On the Mac, it might be possible to do the texture format conversion using the vImage framework, but it would be better to do it in a cross-platform way. The Boost threads library might help with that. >>> Fixing bugs in the transparent path - textures with alpha channels >>> cause strange effects where objects overlap - even if the entire >>> alpha channel is full of 0xFFs. Set the same texture to have no >>> alpha and the problem goes away. >> >> There's a bug about better transparency sorting, I wonder if that's >> what you're talking about. I keep hoping that one of these years the >> graphics cards will have a way to handle transparency without so much >> hassle. > > Sounds like we are sorting the triangles when we should be splitting > them where they intersect. If by "intersect" you mean the 2D projections onto the window rather than the 3D triangles, then yes. But that sounds like a programming nightmare to me. >>> Maybe finishing Apple's work with surface UV clipping e.g. a >>> hemisphere. >> >> I don't know what you are talking about here, can you expand on this? > > There are two types of UVs, surface UVs and shading UVs. There has > been > some confusion with there over the years. The original intention > apparently > was that shading UVs would be used for texture shaders and surface UVs > would be used to clip the object to minimum and maximum UVs. By > specifying > a maximum V of 0.5 on a sphere would give a hemisphere, by specifying > a maximum U of 0.25 and V of 0.5 would give an eighth of a sphere with > four flat quarters of a circle extending to the centre of the sphere. > > Similarly you could use it in on a cone to cut the point off etc. Oh, you're talking about u/v limits on geometries. I don't think that has anything to do with the surface uv and shading uv attributes. The Ellipsoid might be the only case where that hasn't been handled yet. |
|
From: James W. W. <os...@jw...> - 2007-02-18 21:37:01
|
On Feb 18, 2007, at 11:33 AM, James W. Walker wrote: > Oh, you're talking about u/v limits on geometries. I don't think > that has anything to do with the surface uv and shading uv > attributes. The Ellipsoid might be the only case where that hasn't > been handled yet. I'm wrong, it's not implemented for the Torus either. |
