gobo-eiffel-develop — Development discussions

Re: [gobo-eiffel-develop] Memory allocation overhead

[Emmanuel S. [ES] <ma...@ei...>]

> I just saw that in your last check-in you used MEMORY.free.
> This is a bad idea in my opinion. And it won't help anyway 
> when using gec+boehm (it's currently implemented as a no-op).

And this is actually very dangerous because even if you think there are no
references to the object, the generated C code might still have a reference
or two. So at the next GC cycle it could basically cause a problem (be a
segmentation violation or something else). I'm not sure why it was added in
MEMORY at the first place, but I would not use it.

Manu

Re: [gobo-eiffel-develop] Memory allocation overhead

[Eric B. <er...@go...>]

Eric Bezault wrote:
> Colin Paul Adams wrote:
>> Do you use free at all? Obviously that won't reduce the number of
>> allocations, but it will reduce the number of objects the GC has to
>> deal with. Does it give any advantage over assigning Void to the
>> reference?
> 
> I don't use free.

I just saw that in your last check-in you used MEMORY.free.
This is a bad idea in my opinion. And it won't help anyway
when using gec+boehm (it's currently implemented as a no-op).

I talked about reducing the amount of garbage generated,
not about manually managing the memory.

-- 
Eric Bezault
mailto:er...@go...
http://www.gobosoft.com

Re: [gobo-eiffel-develop] Memory allocation overhead

[Eric B. <er...@go...>]

Colin Paul Adams wrote:
> But I am now wondering about code inlining. It appears that gec does
> not inline all three routines new_child_tree_iterator, release_iterator
> and new_descendant_tree_iterator in the code below. (Note that the bulk of
> the class is commented out. The test program then runs in 21 minutes
> and 8 seconds, as opposed to 20 minutes and 28 seconds (+- 3 seconds)
> with the creations inlined, and no call to release_iterator (there is
> an extra assignment to Void of the iterator passed to release_iterator
> following the call to release_iterator, but I hardly think that can
> account for the difference). With all the commented-out code restored,
> (and assignemt of 4 default values to attributes in the creation
> procedures, which costs about 6 seconds) the time rises to 21 minutes
> and 52 seconds. 

I would surprise me that the level of inlining currently implemented
in gec has anything to do with the time actually spent in the GC that
was shown in your profiling results.

-- 
Eric Bezault
mailto:er...@go...
http://www.gobosoft.com

Re: [gobo-eiffel-develop] Memory allocation overhead

[Eric B. <er...@go...>]

Colin Paul Adams wrote:
>>>>>> "Eric" == Eric Bezault <er...@go...> writes:
> 
>     Eric> technique that I use is to try to reuse objects, rather than
>     Eric> giving them back to the GC and creating new ones right
>     Eric> after. In gec, I have AST visitor classes to implement the
>     Eric> different Eiffel "Degree" compilation passes. They all try
>     Eric> to keep the intermediary objects that they need to process a
>     Eric> given imput class. They reuse these intermediary objects
>     Eric> when processing the next input class, and so forth.
> 
> Can you point me to a specific example?

Instead of having iterator objects, I use the visitor pattern.
See the descendants of ET_AST_PROCESSOR. And for a given task
I use the same visitor object on all classes, instead of having
a different object each time. This object can then keep some
context that will be reinitialized each time (without necessarily
having to create new objects). For example in
ET_FEATURE_ADAPTATION_RESOLVER, this visitor uses hash-tables
in order to make sense out of the inheritance clause feature
adaptation of the class being processed. This visitor object
is reused for all input classes, without having to create a new
set of hash-tables each time.

> I am currently trying this with the iterators used to evaluate XPath
> sequences. I am retaining copies on a once DS_ARRAYED_STACK. So far,
> this is actually increasing the runtime (but it appears sensitive to
> the size of the stack, so I am currently trying with just a DS_CELL to
> see if this is better).

I didn't mean to implement yourself a memory management system
by hand. It's not obvious that the time spent doing this manual
memory management, with a pool of objects, will be more efficient
than the GC itself.

-- 
Eric Bezault
mailto:er...@go...
http://www.gobosoft.com

Re: [gobo-eiffel-develop] Multiple root elements in system.xace

[Eric B. <er...@go...>]

Berend de Boer wrote:
> Hello Guys,
> 
> It would be convenient for me if system.xace would allow multiple root
> elements. 
> 
> The use case is this: print the exception message via a rescue clause on
> the application's root class when compiled in boost mode; but still give
> the full stack trace when compiled in debug mode.
> 
> However, SmartEiffel doesn't give a stack trace when you put in a rescue
> clause, so you can only have a rescue clause on the creation routine
> that is used in boost mode, and not in the creation one for debug mode.
> 
> So ideally system.xace would allow me to see:
> 
>   <root class="BP_EMAIL_TASK" creation="make_debug" if="${DEBUG}"/>
>   <root class="BP_EMAIL_TASK" creation="make_rescue_debug" unless="${DEBUG}"/>
> 
> Is that easy to add?

I just tried it and it seems to already work the way
you suggested above.

-- 
Eric Bezault
mailto:er...@go...
http://www.gobosoft.com

Re: [gobo-eiffel-develop] Memory allocation overhead

[Colin P. A. <co...@co...>]

>>>>> "Colin" == Colin Paul Adams <co...@co...> writes:

    Colin> I am currently trying this with the iterators used to
    Colin> evaluate XPath sequences. I am retaining copies on a once
    Colin> DS_ARRAYED_STACK. So far, this is actually increasing the
    Colin> runtime (but it appears sensitive to the size of the stack,
    Colin> so I am currently trying with just a DS_CELL to see if this
    Colin> is better).

In fact the DS_CELL proved to be slower than a small stack.

But I am now wondering about code inlining. It appears that gec does
not inline all three routines new_child_tree_iterator, release_iterator
and new_descendant_tree_iterator in the code below. (Note that the bulk of
the class is commented out. The test program then runs in 21 minutes
and 8 seconds, as opposed to 20 minutes and 28 seconds (+- 3 seconds)
with the creations inlined, and no call to release_iterator (there is
an extra assignment to Void of the iterator passed to release_iterator
following the call to release_iterator, but I hardly think that can
account for the difference). With all the commented-out code restored,
(and assignemt of 4 default values to attributes in the creation
procedures, which costs about 6 seconds) the time rises to 21 minutes
and 52 seconds. 

I can't account for any of this.


class XM_XPATH_ITERATOR_POOL

feature -- Creation

	new_child_tree_iterator (a_starting_node: XM_XPATH_TREE_NODE; a_node_test: XM_XPATH_NODE_TEST): XM_XPATH_TREE_CHILD_ENUMERATION is
			-- New or reused child iterator for `a_starting_node'
		require
			starting_node_not_void: a_starting_node /= Void
			node_test_not_void: a_node_test /= Void
		do
--			if child_tree_iterators.is_empty then
				create Result.make (a_starting_node, a_node_test)
--			else
--				Result := child_tree_iterators.item
--				Result.make (a_starting_node, a_node_test)
--				child_tree_iterators.remove
--			end
		ensure
			new_child_tree_iterator_not_void: Result /= Void
		end

	new_descendant_tree_iterator (a_starting_node: XM_XPATH_TREE_NODE; a_node_test: XM_XPATH_NODE_TEST;
		a_include_self: BOOLEAN): XM_XPATH_TREE_DESCENDANT_ENUMERATION is
			-- New or reused descendant iterator for `a_starting_node'
		require
			starting_node_not_void: a_starting_node /= Void
			node_test_not_void: a_node_test /= Void
		do
--			if descendant_tree_iterators.is_empty then
				create Result.make (a_starting_node, a_node_test, a_include_self)
--			else
--				Result := descendant_tree_iterators.item
--				Result.make (a_starting_node, a_node_test, a_include_self)
--				descendant_tree_iterators.remove
--			end
		ensure
			new_descendant_tree_iterator_not_void: Result /= Void
		end
			
feature -- Removal

	release_iterator (a_iterator: XM_XPATH_SEQUENCE_ITERATOR [XM_XPATH_ITEM]) is
			-- Return `a_iterator' to free memory.
		require
			a_iterator_not_void: a_iterator /= Void
		do
--			if a_iterator.is_tree_child_enumeration then
--				if child_tree_iterators.count < Maximum_queue_length then
--					child_tree_iterators.put (a_iterator.as_tree_child_enumeration)
--				end
--			elseif a_iterator.is_tree_descendant_enumeration then
--				if descendant_tree_iterators.count < Maximum_queue_length then
--					descendant_tree_iterators.put (a_iterator.as_tree_descendant_enumeration)
--				end
--			end
		end

feature {NONE} -- Implementation

	Maximum_queue_length: INTEGER is 5
			-- Limit on queue size for any single iterator type

--	child_tree_iterators: DS_ARRAYED_STACK [XM_XPATH_TREE_CHILD_ENUMERATION]
--			-- Spare iterators over the child axis of XM_XPATH_TREE_NODEs
--		once
--			create Result.make (Maximum_queue_length)
--		ensure
--			child_tree_iterators_not_void: Result /= Void
--		end

--	descendant_tree_iterators: DS_ARRAYED_STACK [XM_XPATH_TREE_DESCENDANT_ENUMERATION]
--			-- Spare iterators over the descendant axis of XM_XPATH_TREE_NODEs
--		once
--			create Result.make (Maximum_queue_length)
--		ensure
--			descendant_tree_iterators_not_void: Result /= Void
--		end

end

-- 
Colin Adams
Preston Lancashire

Re: [gobo-eiffel-develop] Memory allocation overhead

[Colin P. A. <co...@co...>]

>>>>> "Eric" == Eric Bezault <er...@go...> writes:

    Eric> technique that I use is to try to reuse objects, rather than
    Eric> giving them back to the GC and creating new ones right
    Eric> after. In gec, I have AST visitor classes to implement the
    Eric> different Eiffel "Degree" compilation passes. They all try
    Eric> to keep the intermediary objects that they need to process a
    Eric> given imput class. They reuse these intermediary objects
    Eric> when processing the next input class, and so forth.

Can you point me to a specific example?

I am currently trying this with the iterators used to evaluate XPath
sequences. I am retaining copies on a once DS_ARRAYED_STACK. So far,
this is actually increasing the runtime (but it appears sensitive to
the size of the stack, so I am currently trying with just a DS_CELL to
see if this is better).
-- 
Colin Adams
Preston Lancashire

Re: [gobo-eiffel-develop] Memory allocation overhead

[Eric B. <er...@go...>]

Colin Paul Adams wrote:
>>>>>> "Eric" == Eric Bezault <er...@go...> writes:
> 
>     Eric> That does not work in Eiffel. Each time you will access the
>     Eric> expanded C, you will not get a reference to it but a copy of
>     Eric> it.
> 
> That's what I was afraid of.
> 
> Perhaps we should have a syntax for this - reference assignment (note
> that I don't know if I can benefit from this or not - but even if I
> can't there are surely applications that can).

In Eiffel, expanded does not mean that its memory is a subpart
of the memory of another object (this is the compiler which
does -- or possibly does not -- optimize it in such a way).
What Eiffel means by expanded is that an expanded object cannot
be shared by two different objects, and as a consequence the
language talks about expanded in terms of copy semantics.
Allowing reference assignment as you suggest would just open
a can of worms in the language definition.

-- 
Eric Bezault
mailto:er...@go...
http://www.gobosoft.com

Re: [gobo-eiffel-develop] Memory allocation overhead

[Colin P. A. <co...@co...>]

>>>>> "Eric" == Eric Bezault <er...@go...> writes:

    Eric> That does not work in Eiffel. Each time you will access the
    Eric> expanded C, you will not get a reference to it but a copy of
    Eric> it.

That's what I was afraid of.

Perhaps we should have a syntax for this - reference assignment (note
that I don't know if I can benefit from this or not - but even if I
can't there are surely applications that can).
-- 
Colin Adams
Preston Lancashire

Re: [gobo-eiffel-develop] Memory allocation overhead

[Lothar S. <ll...@we...>]

Hello Colin,

Saturday, March 29, 2008, 2:07:09 AM, you wrote:

>>>>>> "Lothar" == Lothar Scholz <ll...@we...> writes:

CPA>     Lothar> Exactly for this reason some java compilers have a special
CPA>     Lothar> optimization inside the code generator when it detects
CPA>     Lothar> sequences of strings concatenations. It's just a to
CPA>     Lothar> important operation.

CPA> so what do they do about it?

The compiler catches this and generates code for a a string buffer object
that gets filled with the concatenated parts. Very simple to implement.

-- 
Best regards,
 Lothar                            mailto:ll...@we...

Re: [gobo-eiffel-develop] Memory allocation overhead

[Berend de B. <be...@po...>]

>>>>> "Colin" == Colin Paul Adams <co...@co...> writes:

>>>>> "Eric" == Eric Bezault <er...@go...> writes:
    Colin> Is there a more efficient way of doing this sort of thing?

Perhaps we should have:

  STRING_.concat(<<once "1", once "2", once "3", once "4">>)

Strings are indeed somewhat trouble in Eiffel. Although the default case
is safe, immutable strings and syntactic sugar for efficient string
concatenations would have been nice.

-- 
Cheers,

Berend de Boer

[gobo-eiffel-develop] Multiple root elements in system.xace

[Berend de B. <be...@po...>]

Hello Guys,

It would be convenient for me if system.xace would allow multiple root
elements. 

The use case is this: print the exception message via a rescue clause on
the application's root class when compiled in boost mode; but still give
the full stack trace when compiled in debug mode.

However, SmartEiffel doesn't give a stack trace when you put in a rescue
clause, so you can only have a rescue clause on the creation routine
that is used in boost mode, and not in the creation one for debug mode.

So ideally system.xace would allow me to see:

  <root class="BP_EMAIL_TASK" creation="make_debug" if="${DEBUG}"/>
  <root class="BP_EMAIL_TASK" creation="make_rescue_debug" unless="${DEBUG}"/>

Is that easy to add?

-- 
Cheers,

Berend de Boer

Re: [gobo-eiffel-develop] Memory allocation overhead

[Lothar S. <ll...@we...>]

Hello Colin,

Friday, March 28, 2008, 7:50:06 PM, you wrote:

>>>>>> "Eric" == Eric Bezault <er...@go...> writes:

CPA>     Eric> Otherwise I try to be very careful with
CPA>     Eric> strings. String concatenations and substring operations
CPA>     Eric> create a lot of intermediary objects. Be careful as well
CPA>     Eric> when strings get resized (even implicitly by some
CPA>     Eric> operations).

CPA> Is there a more efficient way of doing this sort of thing?

CPA>    a_preceding_path := parent.path
CPA>     if STRING_.same_string (a_preceding_path, "/") then
CPA>         Result := STRING_.concat (a_preceding_path, node_name)
CPA>     else
CPA>         Result := STRING_.concat (a_preceding_path, "/")
CPA>         Result := STRING_.appended_string (Result, node_name)
CPA>         Result := STRING_.appended_string (Result, "[")
CPA>         Result := STRING_.appended_string (Result, simple_number)
CPA>         Result := STRING_.appended_string (Result, "]")
CPA>     end

Exactly for this reason some java compilers have a special optimization
inside the code generator when it detects sequences of strings
concatenations. It's just a to important operation.


-- 
Best regards,
 Lothar                            mailto:ll...@we...

Re: [gobo-eiffel-develop] Memory allocation overhead

[Colin P. A. <co...@co...>]

>>>>> "Lothar" == Lothar Scholz <ll...@we...> writes:

    Lothar> Exactly for this reason some java compilers have a special
    Lothar> optimization inside the code generator when it detects
    Lothar> sequences of strings concatenations. It's just a to
    Lothar> important operation.

so what do they do about it?
-- 
Colin Adams
Preston Lancashire

Re: [gobo-eiffel-develop] Memory allocation overhead

[Colin P. A. <co...@co...>]

>>>>> "Eric" == Eric Bezault <er...@go...> writes:

    Eric> Otherwise I try to be very careful with
    Eric> strings. String concatenations and substring operations
    Eric> create a lot of intermediary objects. Be careful as well
    Eric> when strings get resized (even implicitly by some
    Eric> operations).

Is there a more efficient way of doing this sort of thing?

   a_preceding_path := parent.path
    if STRING_.same_string (a_preceding_path, "/") then
	Result := STRING_.concat (a_preceding_path, node_name)
    else
	Result := STRING_.concat (a_preceding_path, "/")
	Result := STRING_.appended_string (Result, node_name)
	Result := STRING_.appended_string (Result, "[")
	Result := STRING_.appended_string (Result, simple_number)
	Result := STRING_.appended_string (Result, "]")
    end
-- 
Colin Adams
Preston Lancashire

Re: [gobo-eiffel-develop] Memory allocation overhead

[Colin P. A. <co...@co...>]

>>>>> "Eric" == Eric Bezault <er...@go...> writes:

    >> Another possibility is to avoid OO techniques. For instance, I
    >> know from last weekend's profiling that there is a VERY large
    >> number of XM_XPATH_UNTYPED_ATOMIC_VALUE objects created by
    >> conversion from XM_XPATH_STRING_VALUE. Untyped-atomic is
    >> XPath's coercible data type. Although untyped-atomic does not
    >> inherit from xs:string in the XPath type hierarchy, I have
    >> implemented XM_XPATH_UNTYPED_ATOMIC_VALUE as inheriting from
    >> XM_XPATH_STRING_VALUE for convenience, as they are nearly
    >> identical excpet for the coercing behaviour. So a clear saving
    >> could be made by merging the two classes with a BOOLEAN to
    >> indicate which type is actualy meant. Then the coercion to
    >> xs:string is simply implemented by flipping the BOOLEAN. I
    >> suspect this is going to be a big saving, but it is very
    >> anti-OO.

    Eric> This might be the kind of things I could use
    Eric> indeed. 

It helped, although not as dramatically as eliminating my ARRAY
[BOOLEAN]s. Eliminating these 4 arrays takes the time down from 71 minutes
to 30 minutes. This change to the untyped atomic values brings it
further down to 22 minutes.

    Eric> Another
    Eric> technique that I use is to try to reuse objects, rather than
    Eric> giving them back to the GC and creating new ones right
    Eric> after. In gec, I have AST visitor classes to implement the
    Eric> different Eiffel "Degree" compilation passes. They all try
    Eric> to keep the intermediary objects that they need to process a
    Eric> given imput class. They reuse these intermediary objects
    Eric> when processing the next input class, and so forth.

Presumably you can only do this because you know when you have
finished with an object, and don't have to rely on the garbage
collector working out when it can free memory.

I don't know how many such cases I am likely to find. If I do, then
perhaps there is the possibility of doing memory pooling - allocating
a single chunk of memory for a large number of identical objects.

I'm not sure how to go about that in Eiffel. I would guess that if I
have a reference class C, with default_create as a creation procedure
(is that necessary?) then if I create an ARRAY [expanded C] the memory
will all be claimed in one call to malloc. Then if I can get a
reference to one of these objects (how do I do that in Eiffel?), I can
call the real creation procedure (it will need to be exported to the
calling class) to initialize the referenced expanded object. I then
just have to keep track of free slots for reusing the memory.
-- 
Colin Adams
Preston Lancashire

Re: [gobo-eiffel-develop] Calculating Checksum in Eiffel

[Andreas L. <ale...@ra...>]

On Wed, 2008-03-26 at 18:22 +0000, Howard Thomson wrote:
> Hi Andreas,
> 
> Well I would wrap C routines for either MD5 or SHA1, which I have been 
> intending to complete Real-Soon-Now for sometime.
> 
> Only four C calls are needed:
> 
> 	sha1.initialize
> 	sha1.process (address: POINTER; count: INTEGER)
> 	sha1.finalize
> 	signature := sha1.result_as_string
> 
> or something like that ...
> 
> I will send you a couple of Eiffel files that will need adapting; they were 
> last used with SmartEiffel and the result_as_string routine will need to be 
> adapted to Gobo's string and character routines.
> 
> See md5deep.sourceforge.net
> 
> If you get it working, I would be obliged if you would return the working 
> classes ...

Hi Howard,

many thanks for your code. Sorry for not getting back earlier. Stefan
Mori was really excited by the idea of implementing the Whirlpool
algorithm and has already finished it by now. If you are interested you
can grab it from
http://svn.origo.ethz.ch/wsvn/eiffelstudio/branches/eth/cdd/Eiffel_61/Src/Eiffel/cdd/support/#_branches_eth_cdd_Eiffel_61_Src_Eiffel_cdd_support_

Andreas

Re: [gobo-eiffel-develop] Calculating Checksum in Eiffel

[Howard T. <how...@di...>]

Hi Andreas,

Well I would wrap C routines for either MD5 or SHA1, which I have been 
intending to complete Real-Soon-Now for sometime.

Only four C calls are needed:

	sha1.initialize
	sha1.process (address: POINTER; count: INTEGER)
	sha1.finalize
	signature := sha1.result_as_string

or something like that ...

I will send you a couple of Eiffel files that will need adapting; they were 
last used with SmartEiffel and the result_as_string routine will need to be 
adapted to Gobo's string and character routines.

See md5deep.sourceforge.net

If you get it working, I would be obliged if you would return the working 
classes ...

Howard


#############################################################
indexing

	description:	"Hashing Algorithms"

deferred class FILE_HASH

feature

	init is
			-- Initialise prior to processing a byte sequence
		deferred
		end

	process(area: POINTER; count: INTEGER) is
			-- Process a contiguous sequence of bytes
		deferred
		end

	finalise is
			-- Post processing of remaining steps
		deferred
		end

end
#############################################################
indexing

	description:	"SHA1 Secure Hash Algorithm"

class HASH_STREAM_SHA1

inherit

	FILE_HASH	-- Note name to be changed

create

	make

feature -- Attributes

	sha1_context: POINTER

	result_buffer: ARRAY [ INTEGER_8 ]

feature -- Routines

	make is
		local
			b: ARRAY [ INTEGER_8 ]
		do
			create b.make (1, sha1_context_size_temp)
			sha1_context := b.to_external
			create result_buffer.make(1, 20)
		end

	init is
		do
			sha1_init (sha1_context)
		end

	process (data: POINTER; count: INTEGER) is
		do
			sha1_process (sha1_context, data, count)
		end

	finalise is
		do
			sha1_finalise (result_buffer.to_external, sha1_context)
		end

	result_as_string: STRING is
		local
			i: INTEGER
		do
			create Result.make (40)
			from
				i := 1
			until
				i > 20
			loop
			--	Result.append ((result_buffer @ i).to_hex_string)
				(result_buffer @ i).to_hexadecimal_in(Result)
				i := i + 1
			end
		end

feature {NONE} -- TEMP

	sha1_context_size_temp: INTEGER is
		do
			Result := (5 + 2) * 8
			Result := Result + 64
		end

feature {NONE} -- External routines, SHA1 implementation

	sha1_context_size: INTEGER is
			-- Size of the C struct containing the SHA1 Context
		external "C use %"md5deep.h%""
		alias "sizeof(SHA1_CTX)"
		end

	sha1_init (p: POINTER) is
			-- Call initialisation routine for the Context
		external "C"
		alias "SHA1Init"
		end

	sha1_process (p_ctx: POINTER; p_data: POINTER; n: INTEGER) is
		external "C"
		alias "SHA1Update"
		end

	sha1_finalise (p_result: POINTER; p_ctx: POINTER) is
		external "C"
		alias "SHA1Final"
		end

invariant

	sha1_context /= default_pointer

end
################################################################

On Friday 14 Mar 2008, Andreas Leitner wrote:
> Hi all,
>
> we need to calculate the checksum of a stream of characters. Basically
> we would like to a quick way to check if two streams of characters are
> he same. False positives are not a big deal, since we can always fall
> back and compare the streams character for character (as long as it does
> not happen too often).
>
> Does anybody know if such an algorithm is already implemented somewhere
> in Eiffel that we could use?
>
> I only found Franck's library
> (http://eiffelzone.com/esd/crc/index.html), but it is written against
> TowerEiffel using BIT, both of which are no longer supported.
>
> many thanks in advance,
> Andreas
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> gobo-eiffel-develop mailing list
> gob...@li...
> https://lists.sourceforge.net/lists/listinfo/gobo-eiffel-develop



-- 
Howard Thomson

--
"Only two things are infinite, the universe and human stupidity,
and I'm not sure about the former." -- Albert Einstein 

Re: [gobo-eiffel-develop] Does gec support CECIL yet?

[Eric B. <er...@go...>]

Colin Adams wrote:
> This means using CECIL. Does gec support that yet? 

Not yet.

-- 
Eric Bezault
mailto:er...@go...
http://www.gobosoft.com

[gobo-eiffel-develop] Does gec support CECIL yet?

[Colin A. <col...@go...>]

I am thinking of embedding Gestalt into Firefox 3.0 later in the year
(I have finally managed to get information on how to approach the
problem). This means using CECIL. Does gec support that yet? If not I
can use ISE (probably the performance difference will not be marked in
situations where a browser is being used, but I am anxious that it is
as fast as possible, as it will be a bit of a showcase for Eiffel).

Ideally, what I want to do is to be able to build a shared library, so
Gestalt can be called by other languages (php, python, ruby etc.), and
use the same library from the Firefox interface.

Re: [gobo-eiffel-develop] Memory allocation overhead

[Eric B. <er...@go...>]

Daniel Tuser wrote:
> This kind of information is very interesting in my opinion. But I never 
> read something comparable. I am not sure if you have a long term plan to 
> use http://gobo-eiffel.wiki.sourceforge.net/, but I would be willing to 
> publish it there. Is that ok?

I'm not a big fan of Wikis, so I have no long term plan to use
the Gobo project Wiki. I'm fine if Gobo users want to use this
space to publish Gobo related information. So feel free to use
it.

-- 
Eric Bezault
mailto:er...@go...
http://www.gobosoft.com

Re: [gobo-eiffel-develop] Memory allocation overhead

[Daniel T. <dan...@gm...>]

Eric Bezault wrote:
> Colin Paul Adams wrote:
>   
>> But I'm interested in what approaches/techniques you use to avoid
>> extra allocations.
>>
>> Do you use free at all? Obviously that won't reduce the number of
>> allocations, but it will reduce the number of objects the GC has to
>> deal with. Does it give any advantage over assigning Void to the
>> reference?
>>     
>
> I don't use free.
>
>   
>> Expanded attributes? This implies having default_create as a creation
>> procedure, whereas the Gobo standard is to have an empty make.
>>     
>
> I don't use expanded.
>
>   
>> Another possibility is to avoid OO techniques. For instance, I know
>> from last weekend's profiling that there is a VERY large number of
>> XM_XPATH_UNTYPED_ATOMIC_VALUE objects created by conversion from
>> XM_XPATH_STRING_VALUE. Untyped-atomic is XPath's coercible data
>> type. Although untyped-atomic does not inherit from xs:string in the
>> XPath type hierarchy, I have implemented XM_XPATH_UNTYPED_ATOMIC_VALUE
>> as inheriting from XM_XPATH_STRING_VALUE for convenience, as they are
>> nearly identical excpet for the coercing behaviour. So a clear saving
>> could be made by merging the two classes with a BOOLEAN to indicate
>> which type is actualy meant. Then the coercion to xs:string is simply
>> implemented by flipping the BOOLEAN. I suspect this is going to be a
>> big saving, but it is very anti-OO.
>>     
>
> This might be the kind of things I could use indeed. Otherwise I try
> to be very careful with strings. String concatenations and substring
> operations create a lot of intermediary objects. Be careful as well
> when strings get resized (even implicitly by some operations). Likewise
> with DS_ARRAYED_... classes: try to create them with a capacity which
> is not too big but not too small to avoid too many resizings. I know,
> it's often not easy to choose the best capacity at creation time!
> I also try to share objects as much as possible. Of course, when
> shared, we have to be very careful that these objects don't have
> their state modified. When that happens, then we need to clone it
> beforehand. I use a lot of shared objects for the tokens when generating
> the ASTs in gec. Another technique that I use is to try to reuse
> objects, rather than giving them back to the GC and creating new
> ones right after. In gec, I have AST visitor classes to implement
> the different Eiffel "Degree" compilation passes. They all try to
> keep the intermediary objects that they need to process a given
> imput class. They reuse these intermediary objects when processing
> the next input class, and so forth. I probably use other techniques,
> but it's already a good start. One thing to remember is that when an
> implementation technique is not "very" OO, try at least to make it
> look as if it was OO in the class interface
This kind of information is very interesting in my opinion. But I never 
read something comparable. I am not sure if you have a long term plan to 
use http://gobo-eiffel.wiki.sourceforge.net/, but I would be willing to 
publish it there. Is that ok?

Re: [gobo-eiffel-develop] Memory allocation overhead

[Eric B. <er...@go...>]

Colin Paul Adams wrote:
> But I'm interested in what approaches/techniques you use to avoid
> extra allocations.
> 
> Do you use free at all? Obviously that won't reduce the number of
> allocations, but it will reduce the number of objects the GC has to
> deal with. Does it give any advantage over assigning Void to the
> reference?

I don't use free.

> Expanded attributes? This implies having default_create as a creation
> procedure, whereas the Gobo standard is to have an empty make.

I don't use expanded.

> Another possibility is to avoid OO techniques. For instance, I know
> from last weekend's profiling that there is a VERY large number of
> XM_XPATH_UNTYPED_ATOMIC_VALUE objects created by conversion from
> XM_XPATH_STRING_VALUE. Untyped-atomic is XPath's coercible data
> type. Although untyped-atomic does not inherit from xs:string in the
> XPath type hierarchy, I have implemented XM_XPATH_UNTYPED_ATOMIC_VALUE
> as inheriting from XM_XPATH_STRING_VALUE for convenience, as they are
> nearly identical excpet for the coercing behaviour. So a clear saving
> could be made by merging the two classes with a BOOLEAN to indicate
> which type is actualy meant. Then the coercion to xs:string is simply
> implemented by flipping the BOOLEAN. I suspect this is going to be a
> big saving, but it is very anti-OO.

This might be the kind of things I could use indeed. Otherwise I try
to be very careful with strings. String concatenations and substring
operations create a lot of intermediary objects. Be careful as well
when strings get resized (even implicitly by some operations). Likewise
with DS_ARRAYED_... classes: try to create them with a capacity which
is not too big but not too small to avoid too many resizings. I know,
it's often not easy to choose the best capacity at creation time!
I also try to share objects as much as possible. Of course, when
shared, we have to be very careful that these objects don't have
their state modified. When that happens, then we need to clone it
beforehand. I use a lot of shared objects for the tokens when generating
the ASTs in gec. Another technique that I use is to try to reuse
objects, rather than giving them back to the GC and creating new
ones right after. In gec, I have AST visitor classes to implement
the different Eiffel "Degree" compilation passes. They all try to
keep the intermediary objects that they need to process a given
imput class. They reuse these intermediary objects when processing
the next input class, and so forth. I probably use other techniques,
but it's already a good start. One thing to remember is that when an
implementation technique is not "very" OO, try at least to make it
look as if it was OO in the class interface.

-- 
Eric Bezault
mailto:er...@go...
http://www.gobosoft.com

Re: [gobo-eiffel-develop] Memory allocation overhead

[Colin P. A. <co...@co...>]

>>>>> "Colin" == Colin Paul Adams <co...@co...> writes:

>>>>> "Eric" == Eric Bezault <er...@go...> writes:
    Eric> You can try reducing the amount of garbage generated

    Colin> So as an example, in XM_XPATH_STATIC_PROPERTY, I have four
    Colin> ARRAY [BOOLEAN] which are included in every
    Colin> expression. Presumably it would be much cheaper to remove
    Colin> the booleans from the arrays (I can't think why I put them
    Colin> in ARRAYs in the first place).

This seems very productive. I changed the first two arrays to be just
a series of BOOLEANs, and the runtime came down from 71 minutes to 50
minutes (my target is sub-one-minute, so there's a way to go, but it's
a good start).

So I'll do the other two ARRAYs next.

Another possibility is actually  to merge all the BOOLEANs as a
bit-field. This will save memory but not the number of objects, so I'm
not really interested in doing this. It might hurt performance.

But I'm interested in what approaches/techniques you use to avoid
extra allocations.

Do you use free at all? Obviously that won't reduce the number of
allocations, but it will reduce the number of objects the GC has to
deal with. Does it give any advantage over assigning Void to the
reference?

Expanded attributes? This implies having default_create as a creation
procedure, whereas the Gobo standard is to have an empty make.

Another possibility is to avoid OO techniques. For instance, I know
from last weekend's profiling that there is a VERY large number of
XM_XPATH_UNTYPED_ATOMIC_VALUE objects created by conversion from
XM_XPATH_STRING_VALUE. Untyped-atomic is XPath's coercible data
type. Although untyped-atomic does not inherit from xs:string in the
XPath type hierarchy, I have implemented XM_XPATH_UNTYPED_ATOMIC_VALUE
as inheriting from XM_XPATH_STRING_VALUE for convenience, as they are
nearly identical excpet for the coercing behaviour. So a clear saving
could be made by merging the two classes with a BOOLEAN to indicate
which type is actualy meant. Then the coercion to xs:string is simply
implemented by flipping the BOOLEAN. I suspect this is going to be a
big saving, but it is very anti-OO.
-- 
Colin Adams
Preston Lancashire

Re: [gobo-eiffel-develop] Memory allocation overhead

[Colin P. A. <co...@co...>]

>>>>> "Eric" == Eric Bezault <er...@go...> writes:

    Eric> You can try reducing the amount of garbage generated

So as an example, in XM_XPATH_STATIC_PROPERTY, I have four ARRAY
[BOOLEAN] which are included in every expression. Presumably it would
be much cheaper to remove the booleans from the arrays (I can't think
why I put them in ARRAYs in the first place).
-- 
Colin Adams
Preston Lancashire