gobo-eiffel-develop — Development discussions

Re: [gobo-eiffel-develop] Gobo Eiffel parser: indexing and assertion expressions

[Martin P. <mar...@un...>]

On Sonntag, 11. Januar 2009, Eric Bezault wrote:
> When parsing, you need to use an AST factory of type
> ET_DECORATED_AST_FACTORY instead of ET_AST_FACTORY.

That works great! Thank you very much, also for the "note" 
implementation!

Kind regards,
Martin

Re: [gobo-eiffel-develop] "note" keyword in Gobo Eiffel parser

[Eric B. <er...@go...>]

Martin Piskernig wrote:
> Since EiffelStudio 6.4 development releases now support the ECMA "note" 
> keyword additional to "indexing" and they already began to convert 
> their libraries to use it, I ask myself how I can activate support of 
> the keyword in the Gobo Eiffel parser. I did not find a switch like it 
> exists for the e.g. "attribute" keyword. Also from a first look in the 
> lexer/parser sources I don't see any reference to the keyword.
> 
> Could you please add support for it or show me how to handle it?

This is now implemented in the SVN repository.

> The "attribute" keyword is now also supported and setting 
> UT_SHARED_ISE_VERSIONS.ise_latest should enable it (or possibly add 
> ise_6_4_latest or something like that).

Done.

-- 
Eric Bezault
mailto:er...@go...
http://www.gobosoft.com

Re: [gobo-eiffel-develop] Gobo Eiffel parser: indexing and assertion expressions

[Eric B. <er...@go...>]

Martin Piskernig wrote:
> I have another question (I just started to use the Gobo Eiffel parser 
> some days ago, please be patient with me...).
> 
> Why is ET_CLASS.first_indexing and ET_FEATURE.first_indexing always 
> Void? Did I forget to do some parsing? The classes and features 
> themselves all show and can be processed nicely, only they seem to have 
> absolutely no indexing... 

When parsing, you need to use an AST factory of type
ET_DECORATED_AST_FACTORY instead of ET_AST_FACTORY.

> The same is true for assertion expressions, be it preconditions, 
> postconditions or invariants: ET_ASSERTION.expression is always Void 
> (on the other hand, i can use ET_TAGGED_ASSERTION.tag without 
> problems). 

Because of ambiguities in the syntax, it is possible that
an assertion of the type:

     tag: expression

ends up in the AST as two assertions nodes: one the tag
and one for the expression.

> btw: Is there an easy way to convert an expression to a string 
> representation? I would need to output assertions...

If you used ET_DECORATED_AST_FACTORY with `keep_all_breaks'
set to True, then you can use ET_AST_PRINTER to print the node
as it was in the text file. Otherwise, you can also use
ET_AST_PRETTY_PRINTER, or inherit from one of those to
customize your own printer.

-- 
Eric Bezault
mailto:er...@go...
http://www.gobosoft.com

[gobo-eiffel-develop] Gobo Eiffel parser: indexing and assertion expressions

[Martin P. <mar...@un...>]

Hi!

I have another question (I just started to use the Gobo Eiffel parser 
some days ago, please be patient with me...).

Why is ET_CLASS.first_indexing and ET_FEATURE.first_indexing always 
Void? Did I forget to do some parsing? The classes and features 
themselves all show and can be processed nicely, only they seem to have 
absolutely no indexing... 

The same is true for assertion expressions, be it preconditions, 
postconditions or invariants: ET_ASSERTION.expression is always Void 
(on the other hand, i can use ET_TAGGED_ASSERTION.tag without 
problems). 

btw: Is there an easy way to convert an expression to a string 
representation? I would need to output assertions...

Please help! Did I miss to call some additional parsing routine?

Kind regards,
Martin

[gobo-eiffel-develop] "note" keyword in Gobo Eiffel parser

[Martin P. <mar...@un...>]

Hi!

Since EiffelStudio 6.4 development releases now support the ECMA "note" 
keyword additional to "indexing" and they already began to convert 
their libraries to use it, I ask myself how I can activate support of 
the keyword in the Gobo Eiffel parser. I did not find a switch like it 
exists for the e.g. "attribute" keyword. Also from a first look in the 
lexer/parser sources I don't see any reference to the keyword.

Could you please add support for it or show me how to handle it?

The "attribute" keyword is now also supported and setting 
UT_SHARED_ISE_VERSIONS.ise_latest should enable it (or possibly add 
ise_6_4_latest or something like that).

Kind regards,
Martin

[gobo-eiffel-develop] New GC for Gobo -- First successful compile with new GC enabled

[Howard T. <how...@di...>]

Hi all,

Somewhat to my surprise, gec with the new GC as the allocator and
with collection functioning has completed a successful compilation of
itself, although the same code when invoked as my IDE did crash.

So it is getting close ...

Merry Christmas all,

Howard Thomson

--
"Only two things are infinite, the universe and human stupidity,
and I'm not sure about the former." -- Albert Einstein 

Re: [gobo-eiffel-develop] New GC for Gobo -- Progress report

[Howard T. <how...@di...>]

Hi Paul,

As regards shared-libraries, for incremental compilation during development I am
envisaging that a generated executable will be split between a compile-every-time
base program, a shared-library containing routines generated 'last time' [where a
previous compilation has taken place], and one or more 'update' shared-libraries,
with some dynamic selection at run-time of most-recently generated routines from
the shared-library set referenced by the base program.

The compiler will use dynamic library loading and call-backs to establish the
available code signatures that are available, where a signature includes any and
all types, arguments, results, options etc that affected code generation, and
generate a new base program and shared-library updates.

I am also working on a set of LLVM style classes to generate LLVM assembler
code and [eventually] machine code directly, at least for Linux/ELF/X86_64.

My most immediate priority is to achieve GC and threading functionality so that I
can use gec and Eiffel to write long-running multi-threaded programs, of which I
have several future projects in the early planning stages.

I will have a look at the SCons tools ...

Regards,

Howard

On Monday 15 Dec 2008, you wrote:
> Hi,
> 
> On Mon, Dec 15, 2008 at 8:26 PM, Howard Thomson
> <how...@di...> wrote:
> > Does anyone have any priority suggestions for what I should work on after getting the
> > GC working, other than multi-threading which I am working on, and incremental
> > compilation on which I have some thoughts and [Linux only] some work in progress ?
> 
> Make sure one can compile shared libraries (.so) on Linux! Pretty
> please. :-) I would vote for this far ahead of multi-threading
> support.
> 
> A SCons tool for the Gobo Eiffel Compiler would also be very nice. Cf:
> http://www.scons.org/wiki/EiffelStudioTool
> 
> Finally I wish either I could get time to do, or someone else could
> do, a clean simple Eiffel wrapping of the Tokyo Cabinet C API. See:
> http://tokyocabinet.sourceforge.net/index.html
> 
> But that's just me wishing!
> 
> > Merry Christmas to all !
> 
> And to you too!
> 
> /Paul
> 



-- 
Howard Thomson

--
"Only two things are infinite, the universe and human stupidity,
and I'm not sure about the former." -- Albert Einstein 

Re: [gobo-eiffel-develop] New GC for Gobo -- Progress report

[Paul C. <pa...@se...>]

Hi,

On Mon, Dec 15, 2008 at 8:26 PM, Howard Thomson
<how...@di...> wrote:
> Does anyone have any priority suggestions for what I should work on after getting the
> GC working, other than multi-threading which I am working on, and incremental
> compilation on which I have some thoughts and [Linux only] some work in progress ?

Make sure one can compile shared libraries (.so) on Linux! Pretty
please. :-) I would vote for this far ahead of multi-threading
support.

A SCons tool for the Gobo Eiffel Compiler would also be very nice. Cf:
http://www.scons.org/wiki/EiffelStudioTool

Finally I wish either I could get time to do, or someone else could
do, a clean simple Eiffel wrapping of the Tokyo Cabinet C API. See:
http://tokyocabinet.sourceforge.net/index.html

But that's just me wishing!

> Merry Christmas to all !

And to you too!

/Paul

-- 
Paul Cohen
www.seibostudios.se
mobile: +46 730 787 035
e-mail: pau...@se...

[gobo-eiffel-develop] New GC for Gobo -- Progress report

[Howard T. <how...@di...>]

Hi all,

I had hoped to be able to announce that my new GC for the Gobo Eiffel Compiler
was working for this Christmas, but it was not to be ... [unless a miracle intervenes !]

It is, however, getting very close, managing some 73 GC collection cycles, where a
cycle is delimited by 1000 4kb page allocations from raw memory, when used as the
allocator for 'gec'.

The remaining problem, assuming of course that only one remains(!), is related to
Manifest Arrays during yy_build_tables, and is in any case most likely caused by
incomplete marking of the reachable set of objects before freeing unmarked objects.

Does anyone have any priority suggestions for what I should work on after getting the
GC working, other than multi-threading which I am working on, and incremental
compilation on which I have some thoughts and [Linux only] some work in progress ?

Merry Christmas to all !

Regards,

Howard Thomson

--
"Only two things are infinite, the universe and human stupidity,
and I'm not sure about the former." -- Albert Einstein 

Re: [gobo-eiffel-develop] Two wishes for Christmas

[Berend de B. <be...@po...>]

>>>>> "Eric" == Eric Bezault <er...@go...> writes:

    Eric> Bernd Schoeller wrote:
    >> I think GOBO always tried to be better than EiffelBase, fixing
    >> its problems. All it does is to disarm a well-known trap when
    >> coding. I cannot imagine that people are relying on the fact that
    >> 'last_string' is the actual read buffer.

    Eric> My experience working on large Eiffel projects involving many
    Eric> developers is that people are often using things that we would
    Eric> have not imaging they would rely on. So I would not be
    Eric> surprised that somewhere some people rely on the fact that
    Eric> `last_string' is a buffer (if only for performance reason).
    Eric> So what I'll try to do in the first place is to try to
    Eric> identify all usage of `last_string' in the code I have access
    Eric> to in order to see the real impact of such a change.

The performance impact would be very ugly for some of my cases. I very
often rely on the fact that strings are not cloned.

And if this is changed, eposix has to change as well, and probably many
other libraries.

I'm afraid the situation will only further trip up students as they will
be seeing a lot of last_string_buffer in actual code because it has to
perform and use GBs of memory, and mimic that.

I think education is the goal here: don't clone your strings. The
problem is people coming from Pascal/Java backgrounds where strings have
different behaviour. No one would expect last_some_class to be a twin.

Learn to live with it.

-- 
Cheers,

Berend de Boer

Re: [gobo-eiffel-develop] Two wishes for Christmas

[Eric B. <er...@go...>]

Bernd Schoeller wrote:
> I think GOBO always tried to be better than EiffelBase, fixing its
> problems. All it does is to disarm a well-known trap when coding. I cannot
> imagine that people are relying on the fact that 'last_string' is the
> actual read buffer.

My experience working on large Eiffel projects involving many
developers is that people are often using things that we
would have not imaging they would rely on. So I would not be
surprised that somewhere some people rely on the fact that
`last_string' is a buffer (if only for performance reason).
So what I'll try to do in the first place is to try to identify
all usage of `last_string' in the code I have access to in order
to see the real impact of such a change.


> (For my understanding, a DS_ARRAYED_LIST is not an ARRAY, there is no need
> for consistency. I am probably not seeing the problem.)

A stack is not a list. Nevertheless stacks have `put', `item', 'remove'
etc. (instead of `push', `top', `pop'). This is for consistency of
naming across the whole library, even when there is no apparent
relation between classes. Likewise for list (be it arrayed or not)
and array and any other container.

Honestly, I don't see why it is so much of a problem to use `force'
instead of `put' when you want the `force' semantics. It's not as if
the features did not exist. Here it looks more like you don't like
the name `force' for this functionality but would prefer `put'. But
as I said, I tried to keep some naming consistency through out the
library. So your request sounds as if you wanted to be able to use
`push' on stacks. It breaks naming consistency.

-- 
Eric Bezault
mailto:er...@go...
http://www.gobosoft.com

Re: [gobo-eiffel-develop] Two wishes for Christmas

[Bernd S. <ber...@co...>]

Hi Eric,

Thanks for your answer.

On Fri, 05 Dec 2008 19:49:04 +0100, Eric Bezault <er...@go...>
wrote:
>> a) I wish that 'last_string' (of "input streams") is renamed to
>> 'read_string_buffer' (or 'last_string_buffer') and a new 'last_string'
>> returns 'read_string_buffer.twin'. I have seen too many poor souls trip
>> over this one.
>
> It has been implemented like that in Gobo to be compatible
> with EiffelBase. I agree with you that it is not intuitive.
> I'll change it when/if it is changed in EiffelBase.

I think GOBO always tried to be better than EiffelBase, fixing its
problems. All it does is to disarm a well-known trap when coding. I cannot
imagine that people are relying on the fact that 'last_string' is the
actual read buffer.

>> (now the big wish)
>>
>> b) I wish that the 'put_*' implementations of dynamic data structure
>> really start behaving dynamically, even if they are implemented using
>> static containers. They should behave the same way that 'force_*' does.
>> The current behavior of 'put_*' is renamed ot 'brittle_put_*' or
> similar.
>>
>> Both points are common pitfalls. The thing that everybody is always
> using
>> 'force_*' instead of 'put_*', just to prevent problems, is ugly and
> feels
>> strange. It also makes changing back and forth between ISE and GOBO very
>> difficult.
>
> I do use `put_*'. For what it's worth, I do believe that EiffelBase's
> usage of `put' and `force' is inconsistent. In some classes it means
> something, in other classes it means something else.

That is my main reason to use GOBO (though it has many other benefits):
that GOBO has cleaned up the meaning of features and made them consistent.
This is great and clearly has been a huge effort. I love GOBO for that.

> In Gobo I tried
> to make it more consistent through out the whole library. In particular
> it is consistent with ARRAY. I'll be happy to rename `put' to something
> else (if only you provide more attractive names than the one you
> suggested), and `force' to `put' if this is done that way as well in
> a consistent way in EiffelBase (including in ARRAY). In the meantime,
> I much prefer to keep it like it is in Gobo.

(For my understanding, a DS_ARRAYED_LIST is not an ARRAY, there is no need
for consistency. I am probably not seeing the problem.)

I accept your final word here. You have walked the way, I did not.

> Bernd, you should stop believing in Santa Claus.

I never believed in Santa Claus (the tradition in our family is different).

I probably should have opened this discussion in a different way, or just
not opened it at all. Sorry about that. I will keep my mouth shut and just
'force' my way through GOBO.

Bernd

Re: [gobo-eiffel-develop] Two wishes for Christmas

[Eric B. <er...@go...>]

Bernd Schoeller wrote:
> Dear List,
> 
> Christmas time is coming up and I think it is the right time for my  
> personal wish list.
> 
> 
> 
> I wish for two changes in GOBO. Both points have bothered not only me, but  
> many GOBO users I know. I have trained many students in the last few years  
> on how to use GOBO, and nearly everbody found the current situation  
> unusual.
> 
> (first the small one)
> 
> a) I wish that 'last_string' (of "input streams") is renamed to  
> 'read_string_buffer' (or 'last_string_buffer') and a new 'last_string'  
> returns 'read_string_buffer.twin'. I have seen too many poor souls trip  
> over this one.

It has been implemented like that in Gobo to be compatible
with EiffelBase. I agree with you that it is not intuitive.
I'll change it when/if it is changed in EiffelBase.

> (now the big wish)
> 
> b) I wish that the 'put_*' implementations of dynamic data structure  
> really start behaving dynamically, even if they are implemented using  
> static containers. They should behave the same way that 'force_*' does.  
> The current behavior of 'put_*' is renamed ot 'brittle_put_*' or similar.
> 
> Both points are common pitfalls. The thing that everybody is always using  
> 'force_*' instead of 'put_*', just to prevent problems, is ugly and feels  
> strange. It also makes changing back and forth between ISE and GOBO very  
> difficult.

I do use `put_*'. For what it's worth, I do believe that EiffelBase's
usage of `put' and `force' is inconsistent. In some classes it means
something, in other classes it means something else. In Gobo I tried
to make it more consistent through out the whole library. In particular
it is consistent with ARRAY. I'll be happy to rename `put' to something
else (if only you provide more attractive names than the one you
suggested), and `force' to `put' if this is done that way as well in
a consistent way in EiffelBase (including in ARRAY). In the meantime,
I much prefer to keep it like it is in Gobo.

Bernd, you should stop believing in Santa Claus.

-- 
Eric Bezault
mailto:er...@go...
http://www.gobosoft.com

[gobo-eiffel-develop] Two wishes for Christmas

[Bernd S. <ber...@co...>]

Dear List,

Christmas time is coming up and I think it is the right time for my  
personal wish list.



I wish for two changes in GOBO. Both points have bothered not only me, but  
many GOBO users I know. I have trained many students in the last few years  
on how to use GOBO, and nearly everbody found the current situation  
unusual.

(first the small one)

a) I wish that 'last_string' (of "input streams") is renamed to  
'read_string_buffer' (or 'last_string_buffer') and a new 'last_string'  
returns 'read_string_buffer.twin'. I have seen too many poor souls trip  
over this one.

(now the big wish)

b) I wish that the 'put_*' implementations of dynamic data structure  
really start behaving dynamically, even if they are implemented using  
static containers. They should behave the same way that 'force_*' does.  
The current behavior of 'put_*' is renamed ot 'brittle_put_*' or similar.

Both points are common pitfalls. The thing that everybody is always using  
'force_*' instead of 'put_*', just to prevent problems, is ugly and feels  
strange. It also makes changing back and forth between ISE and GOBO very  
difficult.

None of the proposed changes should break existing code, as they always  
replace a feature with a more permissive version. It might make some code  
slower. But the alternatives are still there. With the name 'brittle_put',  
it seems obvious that it has a stronger precondition than the regular  
'put'. Other possible prefix suggestions for these features are 'frail_'  
or just 'low_' (short and in the idea of 'low-level').



A good and not too stressful Advent season to everybody,

   Bernd

-- 
Bernd Schoeller, PhD, CTO
Comerge AG, Technoparkstrasse 1, CH-8055 Zurich, www.comerge.net

Re: [gobo-eiffel-develop] ecb instead of ec

[Bernd S. <be...@fa...>]

On Wed, 26 Nov 2008 13:44:35 +0100, Eric Bezault <er...@go...>  
wrote:

> Bernd Schoeller wrote:
>> Is there any reason why 'geant' uses ISE's 'ec' and not 'ecb' for   
>> compilation? 'ecb' seems is much lighter and significantly faster to   
>> compile code.
>
> The reason is that geant was implemented well before ecb was created.
> This can be changed of course. But to have faster compilation I suggest
> using gec ;-)

I would love(!) to switch to GEC at any time, IF I the final details of  
GEC could be implemented (e.g. contract checking, multi-threading,  
internal). It is a pity that we have a very fast compiler that is 90%  
ready for production.

(This is just to motivate you! Keep up the good work :-) )

Bernd

Re: [gobo-eiffel-develop] ecb instead of ec

[Eric B. <er...@go...>]

Bernd Schoeller wrote:
> Is there any reason why 'geant' uses ISE's 'ec' and not 'ecb' for  
> compilation? 'ecb' seems is much lighter and significantly faster to  
> compile code.

The reason is that geant was implemented well before ecb was created.
This can be changed of course. But to have faster compilation I suggest
using gec ;-)

-- 
Eric Bezault
mailto:er...@go...
http://www.gobosoft.com

[gobo-eiffel-develop] ecb instead of ec

[Bernd S. <be...@fa...>]

Hi,

Is there any reason why 'geant' uses ISE's 'ec' and not 'ecb' for  
compilation? 'ecb' seems is much lighter and significantly faster to  
compile code.

Bernd

[gobo-eiffel-develop] New release: Gobo 3.9

[Eric B. <er...@go...>]

Hello,

A new release of Gobo is available from the usual location:

       https://sourceforge.net/projects/gobo-eiffel/

This version should work with the forthcoming release of
ISE's EiffelStudio 6.3.

In order to avoid binary file incompatibilities between
operating systems, the Gobo package does not contain executables
anymore. Instead of that, the directory $GOBO/bin contains the
C files of the Gobo Eiffel compiler which will be used by a script
to populate this directory with the binary files for your platform.
When installing the Gobo package, you will now have to run either:

       %GOBO%\install.bat msc

under Windows, or:

       $GOBO/install.sh gcc

under Unix/Linux in order to generate the binary files
of the Gobo tools in the directory $GOBO/bin.

For more details about this new release of Gobo, please
read the Readme file and the online documentation:

       http://gobo-eiffel.sf.net/gobo/Readme.txt
       http://gobo-eiffel.sf.net/gobo/index.html

--
Eric Bezault
mailto:er...@go...
http://www.gobosoft.com

[gobo-eiffel-develop] ST_STRING_ROUTINES

[Colin P. A. <co...@co...>]

indexing

	description:

		"Support routines used by ST_STRING and ST_STRING_BUILDER"

	library: "Gobo Eiffel String Library"
	copyright: "Copyright (c) 2008, Colin Adams and others"
	license: "MIT License"
	date: "$Date:  $"
	revision: "$Revision:  $"

class ST_STRING_ROUTINES

inherit
	
	ST_IMPORTED_UNICODE_CHARACTER_CLASS_ROUTINES

feature -- Access

	string_sums (a_strings: ARRAY [READABLE_STRING_GENERAL]): INTEGER is
			-- Total number of characters in `a_strings'
		require
			a_strings_not_void: a_strings /= Void
			all_strings_not_void: not a_strings.has (Void)
		local
			i, l_count: INTEGER
		do
			from
				i := 1
			until
				i > a_strings.count
			loop
				l_count := l_count + a_strings.item (i).count
				i := i + 1
			end
			Result := l_count
		ensure
			string_sums_non_negative: Result >= 0
		end
	
feature -- Status report
	
	is_all_whitespace (chars: STRING): BOOLEAN is
			-- Does `chars' consist only of XML white-space characters?
		local
			counter: INTEGER
		do
			from
				counter := 1
				Result := True
			variant
				chars.count + 1 - counter
			until
				Result = False or else counter > chars.count
			loop
				if not is_xml_space (chars.item_code (counter)) then
					Result := False
				end
				counter := counter + 1
			end
		end

	is_alphanumeric (a_character_code: INTEGER): BOOLEAN is
			-- Does `a_character_code' represent an alphanumeric character?
		require
			positive_character_code: a_character_code > 0
		do
			Result := unicode_character_class.unicode.valid_code (a_character_code)
				and then (unicode_character_class.is_letter (a_character_code)
							 or else unicode_character_class.is_number (a_character_code))
		end

	is_xml_space (a_code: INTEGER): BOOLEAN is
			-- Does `a_code' represent an XML space?
		do
			Result := a_code = 32 or a_code = 9 or a_code = 10 or a_code = 13
		end
	
end

-- 
Colin Adams
Preston Lancashire

[gobo-eiffel-develop] ST_STRING_BUILDER

[Colin P. A. <co...@co...>]

indexing

	description:

		"[
        Objects for constructing XPath UTF-32 strings.
        Usage pattern is:
        1) Create builder.
        2) Call appending features.
        3) Call `set_built' followed by `new_string'.
        4) Call `reuse' and loop to (2).
       ]"

	library: "Gobo Eiffel String Library"
	copyright: "Copyright (c) 2008, Colin Adams and others"
	license: "MIT License"
	date: "$Date:  $"
	revision: "$Revision:  $"

class ST_STRING_BUILDER

inherit
	
	ANY

	ST_STRING_ROUTINES
		export {NONE} all end

	ST_SHARED_STRINGS
		export {NONE} all end

create

	make

feature {NONE} -- Initialization

	make (a_capacity: INTEGER) is
			-- Initialize for a string of expected size `a_capacity'.
		require
			a_capacity_non_negative: a_capacity >= 0
		do
			create array.make (1, a_capacity)
			count := 0
			is_building := True
		ensure
			correct_capacity: array.count = a_capacity
			empty: count = 0
			ready_to_build: is_building
		end

feature -- Initialization

	reuse (a_capacity: INTEGER) is
			-- Prepare to reuse `Current' for a string of expected size `a_capacity'.
		require
			a_capacity_non_negative: a_capacity >= 0
			not_ready: not is_building
		do
			array.conservative_resize (1, a_capacity)
			count := 0
			is_building := True
		ensure
			correct_capacity: array.capacity >= a_capacity
			empty: count = 0
			ready_to_build: is_building
		end

feature -- Access

	new_string: ST_STRING is
			-- Newly constructed string
		require
			built: not is_building
		do
			if count = 0 then
				Result := shared_empty_string
			else
				create Result.make_from_codepoints (array.subarray (1, count.as_integer_32))
			end
		ensure
			new_string_not_void: Result /= Void
		end
	
	index_of_code (a_code: NATURAL_32): INTEGER is
			-- Index of first built character matching `a_code';
			-- Result = 0 = no match
		require
			ready_to_build: is_building
		local
			l_finished: BOOLEAN
		do
			from
				Result := 1 
			until
				l_finished
			loop
				if Result > count then
					Result := 0
					l_finished := True
				elseif array.item (Result) = a_code then
					l_finished := True
				else
					Result := Result + 1
				end
			end
		ensure
			index_of_code_in_range: Result >= 0 and Result <= count
		end
	
	code (a_index: INTEGER): NATURAL_32 is
			-- code at `a_index'
		require
			ready_to_build: is_building
			good_index: a_index > 0 and a_index <= count
		do
			Result := array.item (a_index)
		end

feature -- Measurement

	count: INTEGER
			-- Number of characters added to `Current' since last call to `reuse'
	
	is_empty: BOOLEAN is
			-- Have any characters been added?
		do
			Result := count = 0
		ensure
			definition: Result = (count = 0)
		end
			
feature -- Status report

	is_building: BOOLEAN
			-- Is `Current' ready for receiving characters?

feature -- Status setting

	set_built is
			-- Set `is_building' to `False'.
		do
			is_building := False
		ensure
			not_ready: not is_building
		end

feature -- Basic operations

	append_character (a_character: CHARACTER_8) is
			-- Add `a_character' at end of `Current'.
		require
			ready_to_build: is_building
		do
			array.conservative_resize (1, 1 + count)
			array.put (a_character.code.as_natural_32, count + 1)
			count := count + 1
		ensure
			correct_count: count = old count + 1
			correct_character_appended: array.item (count) = a_character.code.as_natural_32
		end
	
	append_code (a_character: NATURAL_32) is
			-- Add `a_character' at end of `Current'.
		require
			ready_to_build: is_building
		do
			array.conservative_resize (1, 1 + count)
			array.put (a_character, count + 1)
			count := count + 1
		ensure
			correct_count: count = old count + 1
			correct_character_appended: array.item (count) = a_character
		end
	
	append_code_multiple (a_character: NATURAL_32; a_count: INTEGER) is
			-- Add `a_character' at end of `Current' `a_count' times.
		require
			ready_to_build: is_building
			strictly_positive_count: a_count > 0
		local
			i: INTEGER
		do
			array.conservative_resize (1, a_count + count)
			from
				i := 1
			until
				i > a_count
			loop
				array.put (a_character, count + 1)
				count := count + 1
				i := i + 1
			end
		ensure
			correct_count: count = old count + a_count
			correct_character_appended: array.subarray (old count + 1, count).for_all (agent matches_code (?, a_character))
		end

	replace_code (a_character: NATURAL_32; a_index: INTEGER) is
			-- Replace `a_character' at `a_index'
		require
			ready_to_build: is_building
			a_index_valid: a_index > 0 and a_index <= count
		do
			array.put (a_character, a_index)
		ensure
			count_unchanged: count = old count
			correct_character_replaced: array.item (a_index) = a_character
		end

	append_string (a_string: READABLE_STRING_GENERAL) is
			-- Add all characters from `a_string' at end of `Current'.
		require
			a_string_not_void: a_string /= Void
			ready_to_build: is_building
		local
			i, l_count, l_new_count: INTEGER
		do
			l_count := a_string.count
			if l_count > 0 then
				l_new_count := count + l_count
				array.conservative_resize (1, l_new_count)
				from
					i := 1
				until
					i > l_count
				loop
					array.put (a_string.code (i), count + i)
					i := i + 1
				end
				count := l_new_count
			end
		ensure
			correct_count: count = old count + a_string.count
			correct_characters_appended: not a_string.is_empty implies characters_match (a_string, old count + 1, count)
		end

	append_strings (a_strings: ARRAY [READABLE_STRING_GENERAL]) is
			-- Add all characters from all of `a_strings' at end of `Current'.
		require
			a_strings_not_void: a_strings /= Void
			all_strings_not_void: not a_strings.has (Void)
			ready_to_build: is_building
		local
			j, k: INTEGER
			i, l_count, l_new_count: INTEGER
			l_string: READABLE_STRING_GENERAL
		do
			l_count := string_sums (a_strings)
			l_new_count := count + l_count
			if l_new_count > 0 then
				array.conservative_resize (1, l_new_count)
				from
					j := 1
					i := 1
				until
					j > a_strings.count
				loop
					from
						l_string := a_strings.item (j)
						k := 1
					until
						k > l_string.count
					loop
						array.put (l_string.code (k), count + i)
						i := i + 1
						k := k + 1
					end
					j := j + 1
				end
				check
					correct_i: i = l_new_count + 1 - count
					-- `i' is incremented on every inner loop
				end
			end
			count := l_new_count
		ensure
			correct_count: count = old count + string_sums (a_strings)
			correct_characters_appended: True -- TODO
		end
	
feature -- Removal

	remove_tail (a_n: INTEGER) is
			-- Remove final `a_n' characters from `Current'.
			-- If `a_n' > `count' remove all characters.
		require
			a_n_non_negative: a_n >= 0
		do
			if a_n > count then
				count := 0
			else
				count := count - a_n
			end
		ensure
			zero_characters_if_a_n_larger_than_count: a_n > old count implies count = 0
			correct_count: a_n <= old count implies count = old count - a_n
		end

feature {NONE} -- Implementation	

	array: ARRAY [NATURAL_32]
			-- Character buffer

	characters_match (a_string: READABLE_STRING_GENERAL; a_start, a_end: INTEGER): BOOLEAN is
			-- Does character sequence of `a_string' match `array.subarray (a_start, a_end)'?
		require
			a_string_not_void: a_string /= Void
			a_start_strictly_positive: a_start > 0
			a_end_greater_than_a_start: a_end > a_start
			a_end_small_enough: a_end <= count
			correct_string_count: a_string.count = (a_end - a_start + 1)
		local
			i, j: INTEGER
		do
			from
				Result := True
				i := a_start
				j := 1
			until
				not Result or i > a_end
			loop
				Result := a_string.code (j) = array.item (i)
				i := i + 1
				j := j + 1
			end
		end
	
	matches_code (a_code, a_other: NATURAL_32): BOOLEAN is
			-- Does `a_code' equal `a_other'?
		do
			Result := a_code = a_other
		ensure
			definition: Result = (a_code = a_other)
		end

invariant

	array_not_void: array /= Void
	count_non_negative: count >= 0

end

-- 
Colin Adams
Preston Lancashire

[gobo-eiffel-develop] ST_STRING

[Colin P. A. <co...@co...>]

indexing

	description:

		"[
	     Fast UTF-32 read-only strings.
        The emphasis is on cheap substring operations.
        Therefore `area' does not include an extra item for NULL termination (for C-strings)
        nor an initial extra item to avoid the cost of index translation.
      ]"

	library: "Gobo Eiffel String Library"
	copyright: "Copyright (c) 2008, Colin Adams and others"
	license: "MIT License"
	date: "$Date:  $"
	revision: "$Revision:  $"

class ST_STRING

inherit

	IMMUTABLE_STRING_GENERAL
		redefine
			is_equal
		end

	HASHABLE
		redefine
			is_equal
		end

	TO_SPECIAL [NATURAL_32]
		rename
			item as special_item,
			infix "@" as special_item_alias
		export
			{NONE} all
			{ST_STRING} area
		redefine
			is_equal, valid_index
		end

	ASCII
		export
			{NONE} all
		redefine
			is_equal
		end
	
	UC_IMPORTED_UNICODE_ROUTINES
		export
			{NONE} all
		redefine
			is_equal
		end
	
	ST_STRING_ROUTINES
		export
			{NONE} all
		redefine
			is_equal
		end

	ST_SHARED_STRINGS
		export
			{NONE} all
		redefine
			is_equal
		end

	ST_IMPORTED_UNICODE_FULL_CASE_MAPPING
		export
			{NONE} all
		redefine
			is_equal
		end
	
	ST_IMPORTED_UNICODE_NORMALIZATION_ROUTINES
		export
			{NONE} all
		redefine
			is_equal
		end

create

	make, make_from_string, make_shared, make_from_codepoints,
	make_from_codes_list, make_empty, make_filled
	
convert

	to_string: {STRING_32}, to_utf8_string: {UC_UTF8_STRING, STRING_8}

feature {NONE} -- Initialization

	make (a_size: INTEGER) is
			-- Initialize to `a_size' NULLs.
		require
			a_size_non_negative: a_size >= 0
		do
			make_area (a_size)
			if a_size > 0 then
				lower := 1
			end
		ensure
			correct_count: count = a_size
			lower_is_zero_for_empty_string: a_size = 0 implies lower = 0
			lower_is_one: a_size > 0 implies lower = 1
		end

	make_empty is
			-- Initialize as empty string.
		do
			make (0)
		ensure
			empty: is_empty
		end
	
	make_filled (a_code: NATURAL_32; a_size: INTEGER) is
			-- Initialize with `a_size' characters all of codepoint `a_code'.
		require
			a_code_valid: valid_code (a_code)
			a_size_non_negative: a_size >= 0
		local
			i: INTEGER
		do
			make_area (a_size)
			if a_size > 0 then
				lower := 1
			end
			from
				i := 1
				upper := a_size
			until
				i > upper
			loop
				area.put (a_code, i - 1)
				i := i + 1
			end
		ensure
			correct_count: count = a_size
			all_characters_correct: -- not yet supported by ISE: for_all (agent a_code.is_equal)
		end

	make_from_string (a_string: READABLE_STRING_GENERAL) is
			-- Initialize to same characters as `a_string'.
		require
			a_string_not_void: a_string /= Void
		local
			i: INTEGER
		do
			make_area (a_string.count)
			if a_string.count > 0 then
				lower := 1
			end
			from
				i := 1
				upper := a_string.count
			until
				i > upper
			loop
				area.put (a_string.code (i), i - 1)
				i := i + 1
			end
		ensure
			correct_count: count = a_string.count
			correct_character_sequence: for_all_with_index (agent same_item_as_string (a_string, ?, ?))
		end

	make_from_codepoints (a_array: ARRAY [NATURAL_32]) is
			-- Initialize `Current' by copying codepoints from `a_array'.
		require
			a_array_not_void: a_array /= Void
			a_array_all_valid: a_array.for_all (agent valid_code) 
		local
			i: INTEGER
		do
			make_area (a_array.count)
			if a_array.count > 0 then
				lower := 1
			end
			from
				i := 1
				upper := a_array.count
			until
				i > upper
			loop
				area.put (a_array.item (i), i - 1)
				i := i + 1
			end
		ensure
			correct_count: count = a_array.count
			correct_character_sequence: for_all_with_index (agent same_item_as_array (a_array, ?, ?))
		end

	make_from_codes_list (a_array: DS_ARRAYED_LIST [INTEGER_32]) is
			-- Initialize `Current' by copying codepoints from `a_array'.
		require
			a_array_not_void: a_array /= Void
		local
			i: INTEGER
		do
			make_area (a_array.count)
			if a_array.count > 0 then
				lower := 1
			end
			from
				i := 1
				upper := a_array.count
			until
				i > upper
			loop
				area.put (a_array.item (i).as_natural_32, i - 1)
				i := i + 1
			end
		ensure
			correct_count: count = a_array.count
		end

	make_shared (a_area: like area; a_start, a_end: INTEGER) is
			--	Initializing sharing `a_area' from `a_start' to `a_end' inclusive.
		require
			a_area_not_void: a_area /= Void
			a_start_large_enough: a_start > 0
			a_end_large_enough: a_end > 0
			a_end_not_smaller_than_a_start: a_end >= a_start
		do
			area := a_area
			lower := a_start
			upper := a_end
		ensure
			area_set: area = a_area
			lower_set: lower = a_start
			upper_set: upper = a_end
		end

feature -- Access

	lower: INTEGER
			-- Index of first character

	upper: INTEGER
			-- Index of last character

	code (a_index: INTEGER): NATURAL_32 is
			-- Unicode codepoint at `a_index'
		do
			Result := area.item (a_index + lower - 2)
		ensure then
			valid_code: valid_code (Result)
		end

	item, infix "@" (a_index: INTEGER): CHARACTER_32 is
			-- Unicode character at `a_index'
		require
			a_index_valid: valid_index (a_index)
		do
			Result := code (a_index).to_character_32
		ensure
			valid_character: unicode.valid_non_surrogate_code (Result.code)
		end

	hash_code: INTEGER is
			-- Hash code value
		local
			i: INTEGER
			l_area: like area
		do
			Result := internal_hash_code
			if Result = 0 then
				if not is_empty then
					-- The magic number `8388593' below is the greatest prime lower than
					-- 2^23 so that this magic number shifted to the left does not exceed 2^31.
					from
						i := lower - 1
						l_area := area
					until
						i = upper
					loop
						Result := ((Result \\ 8388593) |<< 8) + l_area.item (i).to_integer_32
						i := i + 1
					end
					internal_hash_code := Result
				end
			end
		end
	
	maximal_split (a_separators: READABLE_STRING_GENERAL): DS_LIST [ST_STRING] is
			-- `Current' tokenized by `a_separators';
			-- Each character in `Current' matching any in `a_separators' is discarded,
			--  and a (possibly empty) token is extracted.
			-- E.g. "//my/shares/top\directory/in/path/".maximal_split ("/\") yields:
			-- ("","","my","shares","top","directory","in","path","").
		require
			a_separators_not_void: a_separators /= Void
		do
			Result := internal_split (a_separators, True)
		ensure
			maximal_split_not_void: Result /= Void
			no_void_item: not Result.has (Void)
		end

	minimal_split (a_separators: READABLE_STRING_GENERAL): DS_LIST [ST_STRING] is
			-- `Current' tokenized by `a_separators';
			-- Each character in `Current' matching any in `a_separators' is discarded,
			--  and a token is extracted. Consecutive separator characters do not yield
			--  additional tokens.
			-- E.g. "//my/shares/top\directory/in/path/".maximal_split ("/\") yields:
			-- ("","my","shares","top","directory","in","path","").
		require
			a_separators_not_void: a_separators /= Void
		do
			Result := internal_split (a_separators, False)
		ensure
			minimal_split_not_void: Result /= Void
			no_void_item: not Result.has (Void)
		end
	
	trimmed: ST_STRING is
			-- Version of `Current' without leading or trailing XML white space characters (SPACE, TAB, LF and CR)
		do
			Result := stripped_of (" %T%N%R", True, True)
		ensure
			trimmed_not_void: Result /= Void
			not_longer: Result.count <= count
			no_leading_white_space: not Result.is_empty implies not (" %T%N%R").has_code (Result.code (1))
			no_trailing_forbidden_character: not Result.is_empty implies not (" %T%N%R").has_code (Result.code (Result.count))
		end
	
	stripped_of (a_forbidden: READABLE_STRING_GENERAL; a_leading_stripped, a_trailing_stripped: BOOLEAN): ST_STRING is
			-- Version of `Current' without any leading/trailing characters from `a_forbidden';
		require
			a_forbidden_not_void: a_forbidden /= Void
		local
			l_leading_index, l_trailing_index: INTEGER
		do
			l_leading_index := 1
			l_trailing_index := count
			if a_leading_stripped then
				from
				until
					not valid_index (l_leading_index) or else not a_forbidden.has_code (code (l_leading_index))
				loop
					l_leading_index := l_leading_index + 1
				end
			end
			if a_trailing_stripped then
				from
				until
					(not valid_index (l_trailing_index) or l_trailing_index = l_leading_index) or else (not a_forbidden.has_code (code (l_trailing_index)))
				loop
					l_trailing_index := l_trailing_index - 1
				end				
			end
			Result := substring (l_leading_index, l_trailing_index)
		ensure
			stripped_of_not_void: Result /= Void
			not_longer: Result.count <= count
			no_leading_forbidden_character: (a_leading_stripped and not Result.is_empty) implies not a_forbidden.has_code (Result.code (1))
			no_trailing_forbidden_character: (a_trailing_stripped and not Result.is_empty) implies not a_forbidden.has_code (Result.code (Result.count))
		end
	
	substring_index (a_other: ST_STRING; a_start_index: INTEGER): INTEGER is
			-- Index of first occurrence of `a_other' at or after `a_start_index';
			-- 0 if none
		require
			a_other_not_void: a_other /= Void
			valid_a_start_index: a_start_index >= 1 and a_start_index <= count + 1
		do
			Result := string_searcher.substring_index (Current, a_other, a_start_index, count)
		ensure
			valid_result: Result = 0 or else
				(a_start_index <= Result and Result <= count - a_other.count + 1)
			zero_if_absent: (Result = 0) =
				not substring (a_start_index, count).has_substring (a_other)
			at_this_index: Result >= a_start_index implies
				a_other.same_string (substring (Result, Result + a_other.count - 1))
			none_before: Result > a_start_index implies
				not substring (a_start_index, Result + a_other.count - 2).has_substring (a_other)
		end

feature -- Measurement

	count: INTEGER
			-- Number of characters in `Current'
		do
			if upper = 0 then
				Result := 0
			else
				Result := upper - lower + 1
			end
		end

	capacity: INTEGER is
			-- Number of characters allocated in Current
		do
			Result := count
		end
	
	occurrences (c: NATURAL_32): INTEGER is
			-- Number of times `c' appears in the string
		local
			i, l_count: INTEGER
			a: SPECIAL [NATURAL_32]
		do
			from
				l_count := count
				a := area
			until
				i = l_count
			loop
				if a.item (i) = c then
					Result := Result + 1
				end
				i := i + 1
			end
		ensure then
			zero_if_empty: count = 0 implies Result = 0
			recurse_if_not_found_at_first_position:
				(count > 0 and then code (1) /= c) implies
					Result = substring (2, count).occurrences (c)
			recurse_if_found_at_first_position:
				(count > 0 and then code (1) = c) implies
					Result = 1 + substring (2, count).occurrences (c)
		end

feature -- Comparison
	
	infix "<" (a_other: like Current): BOOLEAN is
			-- Is current object less than `other'?
		local
			l_count, l_other_count: INTEGER
		do
			if a_other /= Current then
				l_count := count
				l_other_count := a_other.count
				if l_count < l_other_count then
					Result := (leading_string_comparison (a_other, l_count) >= 0)
				else
					Result := (leading_string_comparison (a_other, l_other_count) > 0)
				end
			end
		end

	is_equal (a_other: like Current): BOOLEAN is
			-- Is `a_other' attached to an object of the same type
			-- as current object and identical to it?
		do
			Result := same_string (a_other)
		end

	same_case_insensitive (a_other: READABLE_STRING_GENERAL): BOOLEAN is
			-- Does `Current' represent the same character sequence (ignoring_case) as `a_other'?
			-- Full case folding is not yet implemented (TODO).
			-- Needs a more efficient implementation (TODO).
			-- Normalization is not considered. To check that two strings are
			--  equal under a given normalization form, compare normalized
			--  copies of the two strings.
		require
			a_other_not_void: a_other /= Void
		local
			l_string: ST_STRING
		do
			if a_other = Current then
				Result := True
			elseif a_other.count = count then
				l_string ?= a_other
				if l_string = Void then
					create l_string.make_from_string (a_other)
				end
				Result := to_lower.same_string (l_string.to_lower)
			end
		end
			
feature -- Status report

	valid_index (i: INTEGER): BOOLEAN is
			-- Is `i' within the bounds of the string?
		do
			Result := i > 0 and i <= count			
		end

	valid_code (a_code: like code): BOOLEAN is
			-- Is `a_code' a valid Unicode codepoint?
		do
			Result := unicode.valid_non_surrogate_code (a_code.as_integer_32)
		end
	
	is_natural, is_natural_32: BOOLEAN is
			-- Does `Current' represent a 32-bit unsigned integer?
		do
			Result := is_valid_integer_or_natural ({NUMERIC_INFORMATION}.type_natural_32)
		end
	
	is_integer, is_integer_32: BOOLEAN is
			-- Does `Current' represent a 32-bit signed integer?
		do
			Result := is_valid_integer_or_natural ({NUMERIC_INFORMATION}.type_integer_32)
		end
	
	is_integer_64: BOOLEAN is
			-- Does `Current' represent a 64-bit signed_integer?
		do
			Result := is_valid_integer_or_natural ({NUMERIC_INFORMATION}.type_integer_64)
		end

	is_double: BOOLEAN is
			-- Does `Current' represent a DOUBLE?
		local
			l_convertor: like ctor_convertor
		do
			if is_valid_as_string_8 then
				l_convertor := ctor_convertor
				l_convertor.parse_string_with_type (Current, {NUMERIC_INFORMATION}.type_double)
				Result := l_convertor.is_integral_double
			end
		ensure
			syntax_and_range:
				-- 'Result' is True if and only if the following two
				-- conditions are satisfied:
				--
				-- 1. In the following BNF grammar, the value of
				--	'Current' can be produced by "Real_literal":
				--
				-- Real_literal	= Mantissa [Exponent_part]
				-- Exponent_part = "E" Exponent
				--				 | "e" Exponent
				-- Exponent		= Integer_literal
				-- Mantissa		= Decimal_literal
				-- Decimal_literal = Integer_literal ["." [Integer]] | "." Integer
				-- Integer_literal = [Sign] Integer
				-- Sign			= "+" | "-"
				-- Integer		= Digit | Digit Integer
				-- Digit		= "0"|"1"|"2"|"3"|"4"|"5"|"6"|"7"|"8"|"9"
				--
				-- 2. The numerical value represented by 'Current'
				--	is within the range that can be represented
				--	by an instance of type DOUBLE.
		end
	
	is_real: BOOLEAN is
			-- Does `Current' represent a REAL?
		do
			Result := is_double
		ensure
			syntax_and_range:
				-- 'Result' is True if and only if the following two
				-- conditions are satisfied:
				--
				-- 1. In the following BNF grammar, the value of
				--	'Current' can be produced by "Real_literal":
				--
				-- Real_literal	= Mantissa [Exponent_part]
				-- Exponent_part = "E" Exponent
				--				 | "e" Exponent
				-- Exponent		= Integer_literal
				-- Mantissa		= Decimal_literal
				-- Decimal_literal = Integer_literal ["." [Integer]] | "." Integer
				-- Integer_literal = [Sign] Integer
				-- Sign			= "+" | "-"
				-- Integer		= Digit | Digit Integer
				-- Digit		= "0"|"1"|"2"|"3"|"4"|"5"|"6"|"7"|"8"|"9"
				--
				-- 2. The numerical value represented by 'Current'
				--	is within the range that can be represented
				--	by an instance of type REAL.
		end

	is_empty: BOOLEAN is
			-- Is structure empty?
		do
			Result := (count = 0)
		end

	is_string_8: BOOLEAN is
			-- Is `Current' a STRING_8?
		do
		end

	is_string_32: BOOLEAN is
			-- Is `Current' a STRING_32?
		do
		end

	is_valid_as_string_8: BOOLEAN is
			-- Is `Current' convertible to STRING_8 without information loss?
		do
			Result := for_all (agent is_character_8)
		end
	
	is_hexadecimal: BOOLEAN is
			-- Is `Current' made up of characters 0-9 or A-F or a-f?
		local
			i, l_count: INTEGER
			l_code: NATURAL_32
		do
			l_count := count
			if l_count = 0 then
				Result := False
			else
				Result := True
				from i := 1 until i > l_count loop
					l_code := code (i)
					if (l_code < Zero.as_natural_32 or l_code > Nine.as_natural_32) and (l_code < Lower_a.as_natural_32 or l_code > Lower_f.as_natural_32) and
						(l_code < Upper_a.as_natural_32 or l_code > Upper_f.as_natural_32)
					 then
						Result := False
						i := l_count + 1 -- Jump out of the loop.
					else
						i := i + 1
					end
				end
			end
		end

	is_base64: BOOLEAN is
			-- Is `Current' made up of characters +, /, =, XML whitespace, 0-9 or A-Z or a-z?
		local
			i, l_count: INTEGER
			l_code: NATURAL_32
		do
			l_count := count
			if l_count = 0 then
				Result := False
			else
				Result := True
				from i := 1 until i > l_count loop
					l_code := code (i)
					if (l_code < Zero.as_natural_32 or l_code > Nine.as_natural_32) and (l_code < Lower_a.as_natural_32 or l_code > Lower_z.as_natural_32) and
						(l_code < Upper_a.as_natural_32 or l_code > Upper_z.as_natural_32) and
						l_code /= Plus.as_natural_32 and l_code /= Slash.as_natural_32 and
						l_code /= Equal_sign.as_natural_32 and l_code /= Blank.as_natural_32 and
						l_code /= Tabulation.as_natural_32 and l_code /= Carriage_return.as_natural_32 and l_code /= Line_feed.as_natural_32
					then
						Result := False
						i := l_count + 1 -- Jump out of the loop.
					else
						i := i + 1
					end
				end
			end
		end
	
	has_substring (a_other: ST_STRING): BOOLEAN is
			-- Does `Current' contain `a_other'?
		require
			a_other_not_void: a_other /= Void
		do
			if a_other = Current then
				Result := True
			elseif a_other.count <= count then
				Result := substring_index (a_other, 1) > 0
			end
		ensure
			false_if_too_small: count < a_other.count implies not Result
			true_if_initial: (count >= a_other.count and then
				a_other.same_string (substring (1, a_other.count))) implies Result
			recurse: (count >= a_other.count and then
				not a_other.same_string (substring (1, a_other.count))) implies
					(Result = substring (2, count).has_substring (a_other))
		end
	
	is_nfd: BOOLEAN is
			-- Is `Current' in NFD normal form?
		do
			Result := normalization.is_nfd (Current)
		end

	is_nfkd: BOOLEAN is
			-- Is `Current' in NFKD normal form?
		do
			Result := normalization.is_nfkd (Current)
		end

	is_nfc: BOOLEAN is
			-- Is `Current' in NFC normal form?
		do
			Result := normalization.is_nfc (Current)
		end

	is_nfkc: BOOLEAN is
			-- Is `Current' in NFKC normal form?
		do
			Result := normalization.is_nfkc (Current)
		end

feature -- New strings

	concatenated_before (a_other: READABLE_STRING_GENERAL): ST_STRING is
			-- New string formed by concatenating `Current' before `a_other'
		require
			a_other_not_void: a_other /= Void
		local
			l_builder: ST_STRING_BUILDER
		do
			create l_builder.make (count + a_other.count)
			l_builder.append_strings (<<Current, a_other>>)
			l_builder.set_built
			Result := l_builder.new_string
		ensure
			concatenated_with_not_void: Result /= Void
			correct_count: Result.count = count + a_other.count
			correct_prefix: Result.substring (1, count).same_string (Current)
			correct_suffix: Result.substring (1 + count, Result.count).same_string (a_other)
		end
	
	concatenated_after (a_other: READABLE_STRING_GENERAL): ST_STRING is
			-- New string formed by concatenating `Current' after `a_other'
		require
			a_other_not_void: a_other /= Void
		local
			l_builder: ST_STRING_BUILDER
		do
			create l_builder.make (count + a_other.count)
			l_builder.append_strings (<<a_other, Current>>)
			l_builder.set_built
			Result := l_builder.new_string
		ensure
			concatenated_with_not_void: Result /= Void
			correct_count: Result.count = count + a_other.count
			correct_suffix: Result.substring (1 + a_other.count, Result.count).same_string (Current)
			correct_prefix: Result.substring (1, a_other.count).same_string (a_other)
		end
	
	concatenated_before_strings (a_strings: ARRAY [READABLE_STRING_GENERAL]): ST_STRING is
			-- New string formed by concatenating `Current' before all of `a_strings'.
		require
			a_strings_not_void: a_strings /= Void
			all_strings_not_void: not a_strings.has (Void)
		local
			l_builder: ST_STRING_BUILDER
		do
			create l_builder.make (count + string_sums (a_strings))
			l_builder.append_string (Current)
			l_builder.append_strings (a_strings)
			l_builder.set_built
			Result := l_builder.new_string
		ensure
			concatenated_before_strings_not_void: Result /= Void
			correct_count: Result.count = count + string_sums (a_strings)
			correct_prefix: Result.substring (1, count).same_string (Current)
			correct_suffix: True -- TODO
		end
	
	concatenated_before_code (a_code: NATURAL_32): ST_STRING is
			-- New string formed by concatenating `Current' before `a_code'
		local
			l_builder: ST_STRING_BUILDER
		do
			create l_builder.make (count + 1)
			l_builder.append_string (Current)
			l_builder.append_code (a_code)
			l_builder.set_built
			Result := l_builder.new_string
		ensure
			concatenated_with_not_void: Result /= Void
			correct_count: Result.count = count + 1
			correct_prefix: Result.substring (1, count).same_string (Current)
			correct_suffix: Result.code (Result.count) = a_code
		end

feature -- Iteration

	for_all (a_test: PREDICATE [ANY, TUPLE [NATURAL_32]]): BOOLEAN is
			-- Is `a_test' true for all codepoints?
		require
			a_test_not_void: a_test /= Void
		local
			l_tuple: TUPLE [NATURAL_32]
			i: INTEGER
		do
			Result := True
			if count > 0 then
				from
					create l_tuple
					i := lower
				until
					i > upper or not Result
				loop
					l_tuple.put (area.item (i - 1), 1)
					Result := a_test.item (l_tuple)
					i := i + 1
				end
			end
		end

	for_all_with_index (a_test: PREDICATE [ANY, TUPLE [NATURAL_32, INTEGER]]): BOOLEAN is
			-- Is `a_test' true for all codepoints?
		require
			a_test_not_void: a_test /= Void
		local
			l_tuple: TUPLE [NATURAL_32, INTEGER]
			i: INTEGER
		do
			Result := True
			if count > 0 then
				from
					create l_tuple
					i := lower
				until
					i > upper or not Result
				loop
					l_tuple.put (area.item (i - 1), 1)
					l_tuple.put (i - lower + 1, 2)
					Result := a_test.item (l_tuple)
					i := i + 1
				end
			end
		end

	do_all, do_forward (a_action: PROCEDURE [ANY, TUPLE [NATURAL_32]]) is
			-- Apply `a_action' to all codepoints.
			-- `a_action' must be precondition-free.
		require
			a_action_not_void: a_action /= Void
		local
			l_tuple: TUPLE [NATURAL_32]
			i: INTEGER
		do
			if count > 0 then
				from
					create l_tuple
					i := lower
				until
					i > upper
				loop
					l_tuple.put (area.item (i - 1), 1)
					a_action.call (l_tuple)
					i := i + 1
				end
			end
		end

	do_all_with_index, do_forward_with_index (a_action: PROCEDURE [ANY, TUPLE [NATURAL_32, INTEGER]]) is
			-- Apply `a_action' to all codepoints, passing index relative to `lower'.
			-- `a_action' must be precondition-free.
		require
			a_action_not_void: a_action /= Void		
		local
			l_tuple: TUPLE [NATURAL_32, INTEGER]
			i: INTEGER
		do
			if count > 0 then
				from
					create l_tuple
					i := lower
				until
					i > upper
				loop
					l_tuple.put (area.item (i - 1), 1)
					l_tuple.put (i - lower + 1, 2)
					a_action.call (l_tuple)
					i := i + 1
				end
			end
		end

feature -- Conversion

	to_string: STRING_32 is
			-- `Current' converted to STRING_32
		do
			create Result.make_filled (('%/0/').to_character_32, count)
			do_all_with_index (agent Result.put_code)
		ensure
			to_string_not_void: Result /= Void
			correct_count: count = Result.count
			correct_character_sequence: for_all_with_index (agent same_item_as_string (Result, ?, ?))
		end
	
	to_latin_1_string: STRING_8 is
			-- `Current' converted to ISO-8859-1
		require
			latin_1_string: is_valid_as_string_8
		do
			create Result.make_filled ('%/0/', count)
			do_all_with_index (agent Result.put_code)
		end

	to_utf8_string: UC_UTF8_STRING is
			-- `Current' converted to UTF-8
		do
			create Result.make_from_string_general (Current)
		ensure
			to_utf_8_not_void: Result /= Void
			correct_count: count = Result.count
			correct_character_sequence: for_all_with_index (agent same_item_as_string (Result, ?, ?))
		end
	
	to_natural, to_natural_32: NATURAL_32 is
			-- 32-bit natural value
		require
			is_natural: is_natural_32
		local
			l_convertor: like ctoi_convertor
		do
			l_convertor := ctoi_convertor
			l_convertor.parse_string_with_type (Current, {NUMERIC_INFORMATION}.type_no_limitation)
			Result := l_convertor.parsed_natural_32
		end
	
	to_integer, to_integer_32: INTEGER_32 is
			-- 32-bit signed integer value
		require
			is_integer: is_integer_32
		local
			l_convertor: like ctoi_convertor
		do
			l_convertor := ctoi_convertor
			l_convertor.parse_string_with_type (Current, {NUMERIC_INFORMATION}.type_no_limitation)
			Result := l_convertor.parsed_integer
		end
	
	to_integer_64: INTEGER_64 is
			-- 64-bit integer value
		require
			is_integer_64: is_integer_64
		local
			l_convertor: like ctoi_convertor
		do
			l_convertor := ctoi_convertor
			l_convertor.parse_string_with_type (Current, {NUMERIC_INFORMATION}.type_no_limitation)
			Result := l_convertor.parsed_integer_64
		end
	
	to_double: DOUBLE is
			-- "Double" value;
			-- for example, when applied to "123.0", will yield 123.0 (double)
		require
			represents_a_double: is_double
		local
			l_convertor: like ctor_convertor
		do
			l_convertor := ctor_convertor
			l_convertor.parse_string_with_type (Current, {NUMERIC_INFORMATION}.type_no_limitation)
			Result := l_convertor.parsed_double
		end
	
	to_real: REAL is
			-- Real value;
			-- for example, when applied to "123.0", will yield 123.0
		require
			represents_a_real: is_real
		do
			Result := to_double
		end
	
	to_lower: ST_STRING is
			-- New lower-case version of `Current'
		do
			Result := case_mapping.lower_string (Current)
		ensure
			to_lower_not_void: Result /= Void
		end

	to_upper: ST_STRING is
			-- New upper-case version of `Current'
		do
			Result := case_mapping.upper_string (Current)
		ensure
			to_upper_not_void: Result /= Void
		end
	
	as_nfd: ST_STRING is
			-- Canonical decomposition of `Current';
		do
			Result := normalization.as_nfd_st_string (Current)
		ensure
			as_nfd_not_void: Result /= Void
			is_nfd: Result.is_nfd
		end

	to_nfd: ST_STRING is
			-- Canonical decomposition of `Current';
		do
			Result := normalization.to_nfd_st_string (Current)
		ensure
			as_nfd_not_void: Result /= Void
			is_nfd: Result.is_nfd
			new_object: Result /= Current
		end
	
	as_nfkd: ST_STRING is
			-- Compatibility decomposition of `Current';
		do
			Result := normalization.as_nfkd_st_string (Current)
		ensure
			as_nfkd_not_void: Result /= Void
			is_nfkd: Result.is_nfkd
		end

	to_nfkd: ST_STRING is
			-- Compatibility decomposition of `Current';
		do
			Result := normalization.to_nfkd_st_string (Current)
		ensure
			as_nfkd_not_void: Result /= Void
			is_nfkd: Result.is_nfkd
			new_object: Result /= Current
		end		

	to_nfc: ST_STRING is
			-- Canonical decomposition then canonical composition of `Current';
		do
			Result := normalization.to_nfc_st_string (Current)
		ensure
			as_nfc_not_void: Result /= Void
			is_nfc: Result.is_nfc
			new_object: Result /= Current
		end

	to_nfkc: ST_STRING is
			-- Compatibility decomposition then canonical composition of `Current';
		do
			Result := normalization.to_nfkc_st_string (Current)
		ensure
			as_nfkc_not_void: Result /= Void
			is_nfkc: Result.is_nfkc
			new_object: Result /= Current
		end		
	
feature -- Duplication

	substring (a_start, a_end: INTEGER): ST_STRING is
			-- Shared-memory substring containing all characters at indices between `a_start' and `a_end';
			-- Memory is not shared if `Result' is empty string.
		do
			if a_end < a_start then
				Result := shared_empty_string
			else
				create Result.make_shared (area, a_start + lower - 1, a_end + lower - 1)
			end
		ensure then
			empty_string: a_end < a_start implies Result.is_empty
			lower: not Result.is_empty implies Result.lower = lower + a_start - 1
			upper: not Result.is_empty implies Result.upper = a_end +lower - 1
			same_area: not Result.is_empty implies Result.area = area
			substring_not_void: Result /= Void
		end

feature {NONE} -- Contract support

	same_item_as_string (a_string: READABLE_STRING_GENERAL; a_code: NATURAL_32; a_index: INTEGER): BOOLEAN is
			-- Does `a_code' equal `a_string.code (a_index)'?
		require
			a_string_not_void: a_string /= Void
			a_index_valid: valid_index (a_index)
		do
			Result := a_code = a_string.code (a_index)
		ensure
			definition: Result = (a_code = a_string.code (a_index))
		end

	same_item_as_array (a_array: ARRAY [NATURAL_32]; a_code: NATURAL_32; a_index: INTEGER): BOOLEAN is
			-- Does `a_code' equal `a_array.item (a_index)'?
		require
			a_array_not_void: a_array /= Void
			a_index_valid: valid_index (a_index)
		do
			Result := a_code = a_array.item (a_index)
		ensure
			definition: Result = (a_code = a_array.item (a_index))
		end

feature {NONE} -- Implementation

	leading_string_comparison (a_other: READABLE_STRING_GENERAL; a_count: INTEGER): INTEGER is
			-- Comparison of first `a_count' characters of `Current' with `a_other'
			-- 0 if equal, < 0 if `Current' < `a_other',
			-- Otherwise > 0
		require
			a_other_not_void: a_other /= Void
			valid_count: a_count <= count and a_count <= a_other.count
		local
			i: INTEGER
			l_code, l_other_code: NATURAL_32
		do
			from
				i := 1
			until
				i > a_count
			loop
				l_code := code (i)
				l_other_code := a_other.code (i)
				if l_code /= l_other_code then
					Result := (l_code.to_integer_32 - l_other_code.to_integer_32)
					i := a_count -- Jump out of loop
				end
				i := i + 1
			end
		end
	
	internal_split (a_separators: READABLE_STRING_GENERAL; a_maximal: BOOLEAN): DS_LINKED_LIST [ST_STRING] is
		require
			a_separators_not_void: a_separators /= Void
		local
			i, l_count, l_last: INTEGER
			l_last_separator: INTEGER
			l_code: NATURAL_32 
			l_item: ST_STRING			
		do
			create Result.make
			l_count := count
			if l_count > 0 then
				from
					i := 1
					l_last_separator := 0
					l_last := 1
				invariant
					last_separator: l_last_separator < i
					last_split: l_last <= i
				until
					i > l_count
				loop
					l_code := code (i)
					if a_separators.has_code (l_code) then
						if l_last_separator = i - 1 then
							if a_maximal then
								Result.force_last (shared_empty_string)
							end
						else
							if l_item = Void then
								l_item := substring (l_last, i - 1)
							else
								l_item := l_item.concatenated_before (substring (l_last, i - 1))
							end
							Result.force_last (l_item)
						end
						l_item := Void
						l_last_separator := i
						l_last := i + 1
					end
					i := i + 1
				end
				if l_last_separator = 0 then
					if l_item = Void then
						l_item := Current
					else
						l_item := l_item.concatenated_before (substring (l_last, l_count))
					end
					Result.force_last (l_item)
				elseif l_last_separator < l_count then
					if l_item = Void then
						l_item := substring (l_last, l_count)
					else
						l_item := l_item.concatenated_before (substring (l_last, l_count))
					end
					Result.force_last (l_item)
				elseif a_maximal then
					Result.force_last (shared_empty_string)
				end				
			end
		ensure
			iternal_split_not_void: Result /= Void
			no_void_item: not Result.has (Void)
		end
	
	is_valid_integer_or_natural (type: INTEGER): BOOLEAN is
			-- Is `Current' a valid number according to given `type'?
		local
			l_convertor: like ctoi_convertor
		do
			if is_valid_as_string_8 then
				l_convertor := ctoi_convertor
				l_convertor.reset (type)
				l_convertor.parse_string_with_type (Current, type)
				Result := l_convertor.is_integral_integer
			end
		end

feature {NONE} -- Agents

	is_same_code (a_other: READABLE_STRING_GENERAL; a_code: NATURAL_32; a_index: INTEGER): BOOLEAN is
			-- Does `a_other.code (a_index)' = `a_code'?
		require
			a_other_not_void: a_other /= Void
			a_index_valid: valid_index (a_index)
		do
			Result := a_other.code (a_index) = a_code
		ensure
			definition: Result = (a_other.code (a_index) = a_code)
		end

	is_character_8 (a_code: NATURAL_32): BOOLEAN is
			-- Is `a_code' in range for for CHARACTER_8?
		do
			Result := a_code < 256
		ensure
			definition: Result = (a_code < 256)
		end

invariant

	area_not_void: area /= Void
	all_code_points_valid: for_all (agent valid_code)
	count_is_not_greater_than_area_count: count <= area.count
	lower_large_enough: lower >= 0
	lower_small_enough: lower <= area.count + 1
	upper_large_enough: upper >= 0
	upper_not_larger_than_lower: upper >= lower

end

-- 
Colin Adams
Preston Lancashire

Re: [gobo-eiffel-develop] C inline externals with GEC

[Eric B. <er...@go...>]

Colin Paul Adams wrote:
> I've just managed to call a function written in Haskell from an Eiffel
> program (by using C external inline) with ISE 6.3.
> 
> When I compile instead with gec, and run the program, I get a
> segmentation fault.
> 
> Here is the backtrace from gdb:
> 
> (gdb) backtrace
> #0  0x0000000000452fc0 in evacuate ()
> #1  0x000000000043b7f8 in scavenge_static ()
> #2  0x0000000000439ea4 in GarbageCollect ()
> #3  0x00000000004351df in scheduleDoGC ()
> #4  0x00000000004353da in exitScheduler ()
> #5  0x00000000004345a2 in hs_exit_ ()
> #6  0x000000000041900c in T21f3 (C=0x1971f80, a1=7) at fib1.c:1200
> #7  0x00000000004191a8 in T21c10 () at fib1.c:1000
> #8  0x0000000000419356 in main (argc=<value optimized out>, argv=<value optimized out>)
>     at fib1.c:11616
> 
> hs_exit(0 is the function used to shutdown the Haskell runtime. So the
> actual call to the Haskell function has worked. (I implemented
> hs_exit() a a separate external - if I comment out the call to this
> external, then the gec-compiled program works fine.
> 
> Looking at the backtrace, I notice the call to GarbageCollect (). I'm
> assuming this is the Haskell garbage collector and not the boehm GC
> that the gec-compiled program is using. Especially as I have done
> nothing specifically to ask for GC from gec. I did have BOEHM_GC set,
> but I unset it, did a geant clobber, followed by a geant compile_ge,
> and the problem still persists.
> 
> Hm. I just tried the program again with a bigger argument (it's just
> generating Fibonacci numbers) with the call to hs_exit commented
> out. Now it crashes again, so presumably in the first case there was
> no actual call to the Haskell garbage collector until the runtime
> system was explicitly shutdown, whereas with the larger argument
> (calculating the fibonacci number for 11199 does a fair bit of work -
> using MA_DECIMAL instead of Haskell, the gec-compiled program takes
> over 10 ms.) it is presumably having to do some garbage collecting
> during the calculation.
> 
> I don't know if you can make any useful comments on this. Thinking
> aloud, it might be something along the lines of gec is using ordinary
> malloc(), in the absence of BoehmGC (is it?), whereas ISE might not be
> using malloc at all (?? - this sounds most unlikely).
> 
> Any thoughts? 

I don't know how the Haskell GC works, so I cannot really help
you here.

Yes, gec uses ordinary malloc.

-- 
Eric Bezault
mailto:er...@go...
http://www.gobosoft.com

Re: [gobo-eiffel-develop] C inline externals with GEC

[Colin P. A. <co...@co...>]

I tried compiling in the boehm GC and the results are the same.
-- 
Colin Adams
Preston Lancashire

[gobo-eiffel-develop] C inline externals with GEC

[Colin P. A. <co...@co...>]

I've just managed to call a function written in Haskell from an Eiffel
program (by using C external inline) with ISE 6.3.

When I compile instead with gec, and run the program, I get a
segmentation fault.

Here is the backtrace from gdb:

(gdb) backtrace
#0  0x0000000000452fc0 in evacuate ()
#1  0x000000000043b7f8 in scavenge_static ()
#2  0x0000000000439ea4 in GarbageCollect ()
#3  0x00000000004351df in scheduleDoGC ()
#4  0x00000000004353da in exitScheduler ()
#5  0x00000000004345a2 in hs_exit_ ()
#6  0x000000000041900c in T21f3 (C=0x1971f80, a1=7) at fib1.c:1200
#7  0x00000000004191a8 in T21c10 () at fib1.c:1000
#8  0x0000000000419356 in main (argc=<value optimized out>, argv=<value optimized out>)
    at fib1.c:11616

hs_exit(0 is the function used to shutdown the Haskell runtime. So the
actual call to the Haskell function has worked. (I implemented
hs_exit() a a separate external - if I comment out the call to this
external, then the gec-compiled program works fine.

Looking at the backtrace, I notice the call to GarbageCollect (). I'm
assuming this is the Haskell garbage collector and not the boehm GC
that the gec-compiled program is using. Especially as I have done
nothing specifically to ask for GC from gec. I did have BOEHM_GC set,
but I unset it, did a geant clobber, followed by a geant compile_ge,
and the problem still persists.

Hm. I just tried the program again with a bigger argument (it's just
generating Fibonacci numbers) with the call to hs_exit commented
out. Now it crashes again, so presumably in the first case there was
no actual call to the Haskell garbage collector until the runtime
system was explicitly shutdown, whereas with the larger argument
(calculating the fibonacci number for 11199 does a fair bit of work -
using MA_DECIMAL instead of Haskell, the gec-compiled program takes
over 10 ms.) it is presumably having to do some garbage collecting
during the calculation.

I don't know if you can make any useful comments on this. Thinking
aloud, it might be something along the lines of gec is using ordinary
malloc(), in the absence of BoehmGC (is it?), whereas ISE might not be
using malloc at all (?? - this sounds most unlikely).

Any thoughts? 
-- 
Colin Adams
Preston Lancashire

Re: [gobo-eiffel-develop] XML/XSLT processing times

[Franck A. <fr...@ne...>]

> 2. Linux cp command - 25 milliseconds.

this is possibly unfair (e.g. if done all in kernel space and returning
before it's really finished).

> 3. Gobo Eiffel XML parser using the example/xml/tree/formatter program
> (modified to operate in unicode string mode, as there were unicode
> character references in the XML file, and commenting out the DTD in
> the file, as the program will not process an external DTD) - 15
> seconds.

it would be interesting to check with a similar example with no unicode,
to see if we're hit by the unicode processing (which wouldn't surprise
me).

> 4. Gobo Eiffel XML parser using the example/xml/event/print program
> (same modifications as above). Output redirected to a file. - 26 seconds.
> 
> The last one was particularly intriguing, as I noticed half the time
> was kernel time. I redirected it to /dev/null instead and it came down
> to 10 seconds. I'm not sure what is going on at all here. Any thoughts?

hm interesting. a guess is that we may be feeding the output 1 character
at a time (you could check with strace) and the other side doesn't like
being fed that way (but it's not in the system call overhead as such, as
/dev/null has the same overhead). if it's the case, it's a bit
surprising (I'd have expected libc and/or the kernel to do buffering).

> ST_STRING might be useful anyway. Shall I post the classes here for
> review?

I'd say yes.