quesa-develop

How I find memory leak?

[Daniele C. <dca...@in...>]

I am searching a memory leak in my application.In TQ3Status E3Exit(void) 
function I read:

Q3Memory_DumpRecording( "Quesa-leaks.txt", "Q3Exit" );

Can it useful for me? How I can activate? I don't find documentaion 
about it. Have you some suggest for me?

Re: How I find memory leak?

[James W. W. <os...@jw...>]

On Feb 2, 2007, at 5:25 AM, Daniele Cavallini wrote:

> I am searching a memory leak in my application.In TQ3Status E3Exit 
> (void)
> function I read:
>
> Q3Memory_DumpRecording( "Quesa-leaks.txt", "Q3Exit" );
>
> Can it useful for me? How I can activate? I don't find documentaion
> about it. Have you some suggest for me?

To use it, you need a copy of Quesa that has been compiled with the  
macro Q3_DEBUG set to 1.  It will produce a text file listing the  
types and numbers of leaked references.  In the Mac CFM case, it also  
shows backtraces.

Re: How I find memory leak?

[Daniele C. <dca...@in...>]

It is a very useful function.
I set Q3_DEBUG and  I set "Q3Memory_StartRecording()"  at start of 
application and I obtain a file like this:

/========== START DUMP Fri Feb 02 18:15:06 2007 Q3Exit ==========
InteractiveRenderer (033535EC)   2
Set (03353804)   1
Quesa:DepthBitsElement (033538DC)
Quesa:Slab (03353A64)
Quesa:Slab (03353ACC)
Quesa:Slab (03353B34)
Quesa:Slab (03353B9C)
View (03354964)
AttributeSet (03354A7C)   1
Win32DCDrawContext (03354B54)   1
ViewAngleCamera (03354C1C)   1
PhongIllumination (03354E0C)   1
LightGroup (03354E94)   1
AmbientLight (03354EFC)   1
PointLight (03354FBC)   1
FillStyle (0039FCD4)   1
InterpolationStyle (03354CC4)   1
BackfacingStyle (03354D2C)   1
OrderedDisplayGroup (033557C4)   1
DisplayGroup (03355E1C)   1
AttributeSet (03355EA4)   3
Myatt (03355F7C)
Polygon (03356084)   1
AttributeSet (0335625C)   1
TriMesh (033566AC)   1
TriMesh (03356EEC)   1
Set (03356C44)   1
Quesa:IR:OptTriMeshCache (03356D1C)
========== END DUMP ==========/

Are all memory leak?


James W. Walker ha scritto:
> On Feb 2, 2007, at 5:25 AM, Daniele Cavallini wrote:
>
>   
>> I am searching a memory leak in my application.In TQ3Status E3Exit 
>> (void)
>> function I read:
>>
>> Q3Memory_DumpRecording( "Quesa-leaks.txt", "Q3Exit" );
>>
>> Can it useful for me? How I can activate? I don't find documentaion
>> about it. Have you some suggest for me?
>>     
>
> To use it, you need a copy of Quesa that has been compiled with the  
> macro Q3_DEBUG set to 1.  It will produce a text file listing the  
> types and numbers of leaked references.  In the Mac CFM case, it also  
> shows backtraces.
>
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier.
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Quesa-develop mailing list
> Que...@li...
> https://lists.sourceforge.net/lists/listinfo/quesa-develop
>
>   

Re: How I find memory leak?

[James W. <ja...@fr...>]

Daniele Cavallini wrote:
> It is a very useful function.
> I set Q3_DEBUG and  I set "Q3Memory_StartRecording()"  at start of 
> application and I obtain a file like this:
> 
> /========== START DUMP Fri Feb 02 18:15:06 2007 Q3Exit ==========
> InteractiveRenderer (033535EC)   2
> Set (03353804)   1
> Quesa:DepthBitsElement (033538DC)
> Quesa:Slab (03353A64)
> Quesa:Slab (03353ACC)
> Quesa:Slab (03353B34)
> Quesa:Slab (03353B9C)
> View (03354964)
> AttributeSet (03354A7C)   1
> Win32DCDrawContext (03354B54)   1
> ViewAngleCamera (03354C1C)   1
> PhongIllumination (03354E0C)   1
> LightGroup (03354E94)   1
> AmbientLight (03354EFC)   1
> PointLight (03354FBC)   1
> FillStyle (0039FCD4)   1
> InterpolationStyle (03354CC4)   1
> BackfacingStyle (03354D2C)   1
> OrderedDisplayGroup (033557C4)   1
> DisplayGroup (03355E1C)   1
> AttributeSet (03355EA4)   3
> Myatt (03355F7C)
> Polygon (03356084)   1
> AttributeSet (0335625C)   1
> TriMesh (033566AC)   1
> TriMesh (03356EEC)   1
> Set (03356C44)   1
> Quesa:IR:OptTriMeshCache (03356D1C)
> ========== END DUMP ==========/
> 
> Are all memory leak?

The fact that no reference count is greater than 2 indicates that it is 
not a serious leak, you probably just neglected to dispose all objects 
before calling Q3_Exit.  Furthermore, they are not all independent.  For 
example, once you stop leaking renderers, the Slab and DepthBitsElement 
leaks will probably go away.
-- 
   James W. Walker, Innoventive Software LLC
   <http://www.frameforge3d.com/>

Re: Replicated textures

[Roger H. <rog...@mi...>]

On 13 Feb, 2007, at 16:01, James W. Walker wrote:

>
> On Feb 13, 2007, at 5:12 AM, Roger Holmes wrote:
>
>> I have been thinking about a problem we are having with multiple
>> objects having the same texture on them. They are gobbling up
>> all the logical address space and also causing thrashing on
>> Macs with less than 2GB of RAM.
>
> Been there...
>
>> At one time, when we read in objects, we would compare all
>> their textures with all existing textures and all their own other
>> textures. We would then make identical ones share the same
>> texture shader objects. This worked fine except we had to take
>> it out because users were complaining about the speed of
>> reading files.
>
> When I read an object, I compare (hashes of) its textures to each
> other and share if possible, but I don't compare to other objects.
> That's an interesting thought.
>
>> The comparisons had the obvious speed ups of comparing
>> the size etc first before comparing the actual pixels. What I
>> am now thinking about is storing a hash of the pixels so
>> that most of the time I will only have to compare the hash.
>> When two identical hashes are found, I do not mind doing
>> the comparison because the chances are very good that
>> the textures really are the same and I will be saving memory.
>
> You'd have to be careful to invalidate the cached hash if the texture
> or its storage is edited.

Can I presume the edit index of the storage object is kept up to date?

>
>> The question is where to store the hashes. I could put them
>> in a custom attribute. This would be fine for me, but would
>> we be missing an opportunity. If they were stored in the
>> texture shaders or even the storage objects, would this
>> be of any use to either the renderers or the saving code
>> or anywhere else?
>
> It's not immediately obvious to me how it would be of use.

Ok, but you have done the same as I'm going to so maybe it
would help the next person who wants to reinvent the wheel.

Maybe we could have a method on the storage object which gives a
hash code. If we stored a cached value and the edit index (which
gets set to an invalid value in the NEW method) in the storage
object's header, then when we are asked for the hash, we compare
the edit indexes. If they match we return the cached value, if not
we update the copy of the edit index, hash the data block, store it
in the cache and return its value.

The extra information in the header would not need to be saved
on disc.

Roger

Re: Replicated textures

[James W. <ja...@fr...>]

Roger Holmes wrote:

> Can I presume the edit index of the storage object is kept up to date?

Good question.  Q3Storage_SetData bumps the edit index, but there are 
probably a number of other ways to change a storage object.  For 
instance, I suppose you could use Q3MemoryStorage_GetBuffer to get a 
pointer to the bytes, and then change some bytes, and the storage object 
would have know way of knowing it was edited.


>>> The question is where to store the hashes. I could put them
>>> in a custom attribute. This would be fine for me, but would
>>> we be missing an opportunity. If they were stored in the
>>> texture shaders or even the storage objects, would this
>>> be of any use to either the renderers or the saving code
>>> or anywhere else?
>> It's not immediately obvious to me how it would be of use.
> 
> Ok, but you have done the same as I'm going to so maybe it
> would help the next person who wants to reinvent the wheel.
> 
> Maybe we could have a method on the storage object which gives a
> hash code. If we stored a cached value and the edit index (which
> gets set to an invalid value in the NEW method) in the storage
> object's header, then when we are asked for the hash, we compare
> the edit indexes. If they match we return the cached value, if not
> we update the copy of the edit index, hash the data block, store it
> in the cache and return its value.
> 
> The extra information in the header would not need to be saved
> on disc.

Of course, if it's a file storage, you'd have to read the file to 
compute the hash... though I don't know if anyone is likely to use a 
file storage for texture data.
-- 
   James W. Walker, Innoventive Software LLC
   <http://www.frameforge3d.com/>

Re: Replicated textures

[Roger H. <rog...@mi...>]

On 13 Feb, 2007, at 20:41, James Walker wrote:

> Roger Holmes wrote:
>
>> Can I presume the edit index of the storage object is kept up to  
>> date?
>
> Good question.  Q3Storage_SetData bumps the edit index, but there are
> probably a number of other ways to change a storage object.  For
> instance, I suppose you could use Q3MemoryStorage_GetBuffer to get a
> pointer to the bytes, and then change some bytes, and the storage  
> object
> would have know way of knowing it was edited.

In the documentation and header file comments we would have to note
something like : If you modify a memory storage object, you must call
<The routine which increments the edited count> before calling the
hash routine, otherwise you may get the old value of the hash.
If we write a texture equal function, we would have to have the
same note for it too as it would call the hash routine.


>
>>>> The question is where to store the hashes. I could put them
>>>> in a custom attribute. This would be fine for me, but would
>>>> we be missing an opportunity. If they were stored in the
>>>> texture shaders or even the storage objects, would this
>>>> be of any use to either the renderers or the saving code
>>>> or anywhere else?
>>> It's not immediately obvious to me how it would be of use.
>>
>> Ok, but you have done the same as I'm going to so maybe it
>> would help the next person who wants to reinvent the wheel.
>>
>> Maybe we could have a method on the storage object which gives a
>> hash code. If we stored a cached value and the edit index (which
>> gets set to an invalid value in the NEW method) in the storage
>> object's header, then when we are asked for the hash, we compare
>> the edit indexes. If they match we return the cached value, if not
>> we update the copy of the edit index, hash the data block, store it
>> in the cache and return its value.
>>
>> The extra information in the header would not need to be saved
>> on disc.
>
> Of course, if it's a file storage, you'd have to read the file to
> compute the hash... though I don't know if anyone is likely to use a
> file storage for texture data.


Yes unlikely. Maybe we only support it for RAM storage objects.
Do file storage objects have an edit count? On disc they have a
last modified date we could use instead. But why make work for
ourselves which we don't expect anybody will ever use.

I suppose we could keep the hash routine private and only
expose a texture compare in the API.


Roger.

Re: How I find memory leak?

[Daniele C. <dca...@in...>]

I found and resolved. Luckily there is the function 
"Q3Memory_StartRecording".
I think it is important a little of documentation just for to know it exist.
James Walker wrote:
> Daniele Cavallini wrote:
>   
>> It is a very useful function.
>> I set Q3_DEBUG and  I set "Q3Memory_StartRecording()"  at start of 
>> application and I obtain a file like this:
>>
>> /========== START DUMP Fri Feb 02 18:15:06 2007 Q3Exit ==========
>> InteractiveRenderer (033535EC)   2
>> Set (03353804)   1
>> Quesa:DepthBitsElement (033538DC)
>> Quesa:Slab (03353A64)
>> Quesa:Slab (03353ACC)
>> Quesa:Slab (03353B34)
>> Quesa:Slab (03353B9C)
>> View (03354964)
>> AttributeSet (03354A7C)   1
>> Win32DCDrawContext (03354B54)   1
>> ViewAngleCamera (03354C1C)   1
>> PhongIllumination (03354E0C)   1
>> LightGroup (03354E94)   1
>> AmbientLight (03354EFC)   1
>> PointLight (03354FBC)   1
>> FillStyle (0039FCD4)   1
>> InterpolationStyle (03354CC4)   1
>> BackfacingStyle (03354D2C)   1
>> OrderedDisplayGroup (033557C4)   1
>> DisplayGroup (03355E1C)   1
>> AttributeSet (03355EA4)   3
>> Myatt (03355F7C)
>> Polygon (03356084)   1
>> AttributeSet (0335625C)   1
>> TriMesh (033566AC)   1
>> TriMesh (03356EEC)   1
>> Set (03356C44)   1
>> Quesa:IR:OptTriMeshCache (03356D1C)
>> ========== END DUMP ==========/
>>
>> Are all memory leak?
>>     
>
> The fact that no reference count is greater than 2 indicates that it is 
> not a serious leak, you probably just neglected to dispose all objects 
> before calling Q3_Exit.  Furthermore, they are not all independent.  For 
> example, once you stop leaking renderers, the Slab and DepthBitsElement 
> leaks will probably go away.
>   

Replicated textures

[Roger H. <rog...@mi...>]

I have been thinking about a problem we are having with multiple
objects having the same texture on them. They are gobbling up
all the logical address space and also causing thrashing on
Macs with less than 2GB of RAM.

At one time, when we read in objects, we would compare all
their textures with all existing textures and all their own other
textures. We would then make identical ones share the same
texture shader objects. This worked fine except we had to take
it out because users were complaining about the speed of
reading files.

The comparisons had the obvious speed ups of comparing
the size etc first before comparing the actual pixels. What I
am now thinking about is storing a hash of the pixels so
that most of the time I will only have to compare the hash.
When two identical hashes are found, I do not mind doing
the comparison because the chances are very good that
the textures really are the same and I will be saving memory.

The question is where to store the hashes. I could put them
in a custom attribute. This would be fine for me, but would
we be missing an opportunity. If they were stored in the
texture shaders or even the storage objects, would this
be of any use to either the renderers or the saving code
or anywhere else?

Roger Holmes.

Re: Replicated textures

[James W. W. <os...@jw...>]

On Feb 13, 2007, at 5:12 AM, Roger Holmes wrote:

> I have been thinking about a problem we are having with multiple
> objects having the same texture on them. They are gobbling up
> all the logical address space and also causing thrashing on
> Macs with less than 2GB of RAM.

Been there...

> At one time, when we read in objects, we would compare all
> their textures with all existing textures and all their own other
> textures. We would then make identical ones share the same
> texture shader objects. This worked fine except we had to take
> it out because users were complaining about the speed of
> reading files.

When I read an object, I compare (hashes of) its textures to each  
other and share if possible, but I don't compare to other objects.   
That's an interesting thought.

> The comparisons had the obvious speed ups of comparing
> the size etc first before comparing the actual pixels. What I
> am now thinking about is storing a hash of the pixels so
> that most of the time I will only have to compare the hash.
> When two identical hashes are found, I do not mind doing
> the comparison because the chances are very good that
> the textures really are the same and I will be saving memory.

You'd have to be careful to invalidate the cached hash if the texture  
or its storage is edited.

> The question is where to store the hashes. I could put them
> in a custom attribute. This would be fine for me, but would
> we be missing an opportunity. If they were stored in the
> texture shaders or even the storage objects, would this
> be of any use to either the renderers or the saving code
> or anywhere else?

It's not immediately obvious to me how it would be of use.