gobo-eiffel-develop — Development discussions

Re: [gobo-eiffel-develop] GEC introspection & debugger

[Wolfgang J. <wj...@so...>]

Hi Eric,

Eric Bezault wrote:
> Hi Wolfgang,
>
> Wolfgang Jansen wrote:
>>
>>
>> The following naming convention has been applied:
>>
>> 1) Introspection classes have prefix "IN_" and are located
>> in cluster $GOBO/library/intro.
>
> As an Eiffelist, I try to avoid abbreviation. So I would prefer
> introspection rather than just intro.
>
>> 2) Persistence classes have prefix "PC_" and are located
>> in cluster $GOBO/library/persist
>
> Same remark, persistence instead of persist.
>
>> 3) Debugger classes have prefix "DG_" and are located
>> in cluster $GOBO/library/tools/eiffel/debugger
>>
>> 4) Special versions of introspection classes used
>> during compilation have prefix "ET_IN_" and are located
>> in cluster $GOBO/library/tools/eiffel/generation/intro
>
> introspection rather than intro.
Done.
>
>>
>> Some problems and open questions remain:
>>
>> 2) The compiler generates test for Void targets like
>>      return ((T1)(((T0*)(GE_void(a1)))->id==66));
>>      l6 = ((GE_void(l5), (T0*)0));
>> In most cases this is welcome but is counter-productive
>> in case of introspection, ... which generate references
>> from C pointers: the compiler is not aware of the objects
>> and often generates code where the references are
>> erroneously treated as `Void'.
>> For example, when restoring an object from file in independent
>> store mode its contents (i.e. whether attributes are void or not)
>> is beyond the scope of the actual system's compilation.
>> This means that objects and their attributes (recursively to any
>> depth) obtained from C pointers must be treated as potentially
>> void (i.e. generate Void-tests like the 1st example above)
>> but must also be treated as potentially not-void
>> (i.e. *not* code like the 2nd example).
>
> I don't think that the problem is with void checking. The
> problem is with the dynamic type set builder. It is not aware
> that entities can have some given types at runtime because
> they will be attached to objects retrieved from storable
> files. Here you see the problem with the void checking. But
> the problem will be similar with dynamic binding. The
> compiler might think that an entity will have only to
> possible types at runtime and will generate the code for
> the dynamic binding accordingly. But if, after retrieving
> objects from storable files that entity can have a third
> type at runtime, then the dynamic binding will be buggy.
Of course, the reason is the typesets, but the effect is the false void 
check.
Anyway, we need the possibility that the following code

   read_xyz(s: STORABLE): XYZ
      do
         Result ?= s.retrieve_by_name("file_name")
      end

does not return Void (as is currently the case) if the object in question
is actually of type XYZ. Currently, the only way is misleading
the compiler like:

   read_xyz(s: STORABLE): XYZ
      local
          xyz: XYZ
      do
         if xyz /= Void then
            -- The following code is never run but ensures
            -- that XYZ belongs to the typeset of `s'.
            create xyz
            s := xyz
         end
         Result ?= s.retrieve_by_name("file_name")
      end

(Well, class STORABLE is not yet implemented.
Imagine that it had been implemented by forwarding its tasks
to the classes handling the persistence closure.
Then the result would be Void in the first example).
>
>> 3) To display more informative debuggee status the compiler
>> should collect more information. For example, some compiler
>> related classes have queries like `keyword: ET_KEYWORD'
>> which potentially provide a keyword's position in class text.
>> Unfortunately, the position is not always set.
>
> In order for the position to be always available, you have
> to use ET_DECORATED_AST_FACTORY instead of ET_AST_FACTORY.
>
There seems to be a misunderstanding: I don't use both.
I extract the information needed from the result of normal compilation
(i.e. from `ET_C_GENERATOR.current_dynamic_system')
after parsing and analysis but before code generation.
At this point the ET_KEYWORDs are ready with or without position.
So, I am interested in filling the ET_KEYWORDs during
normal compilation.

>> So, if you are interested, how should the new stuff be integrated
>> into the GOBO project?
>> Simply by updating via SVN? By a new SNV path (probably better
>> since the new stuff still in experimental state)? Another way?
>
> Another way would be to move the Gobo repository to git.
> Jocelyn Fiat already uses git to work and maintain a void-safe
> version of Gobo. It would be easier I think if the master
> repository would be under git. I guess having to go back and
> forth between SVN and git is not very practical.
>
> If we make the decision to move to git, then what I'm not sure
> yet is whether we should host the git repository at SourceForge,
> or use github.com. What I like with github (I'm not sure whether
> this is available in other git hosts like SourceForge) is that
> from the Gobo master git repository we could see who is working
> on his own version of Gobo in github. My feeling is that it makes
> things easier to follow when we want to merge/integrate
> development into the main/master Gobo repository.
I am open for many solutions.

WJ

-- 
Dr. Wolfgang Jansen
University of Potsdam, Germany
Institute of Computer Science
Tel: +49 331 / 977 3047
mailto: wj...@so...

Re: [gobo-eiffel-develop] GEC introspection & debugger

[Eric B. <er...@go...>]

Hi Wolfgang,

Wolfgang Jansen wrote:
> Hi,
> 
> long time ago I offered to provide code for introspection and
> persistence closure to the GOBO library as well as a debugger
> for then GEC. Work took more time than expected but now an
> experimental version is available.
> 
> 
> The following naming convention has been applied:
> 
> 1) Introspection classes have prefix "IN_" and are located
> in cluster $GOBO/library/intro.

As an Eiffelist, I try to avoid abbreviation. So I would prefer
introspection rather than just intro.

> 2) Persistence classes have prefix "PC_" and are located
> in cluster $GOBO/library/persist

Same remark, persistence instead of persist.

> 3) Debugger classes have prefix "DG_" and are located
> in cluster $GOBO/library/tools/eiffel/debugger
> 
> 4) Special versions of introspection classes used
> during compilation have prefix "ET_IN_" and are located
> in cluster $GOBO/library/tools/eiffel/generation/intro

introspection rather than intro.

> 5) The generated C code contains additional
> #defines, typedefs and global variables.
> They all have the prefix "GE_Z" or "GE_z".
> Of course, names and clusters may be changed.
> 
> 
> Changes of existing classes and C code:
> 
> The additional C code necessary for introspection
> and debugging (the persistence closure is a pure library
> and does not need additional C code) is generated
> in descendant classes of ET_C_GENERATOR which
> belong to cluster $GOBO/library/tools/eiffel/generation.
> There are just two changes in existing code:
> 
> 1) Class GEC, lines 199... and 234...:
> some local variables have been added and
>      create l_generator.make (l_system)
> is replaced by code that creates an object of the appropriate
> descendant class.
> 
> 2) Class ET_C_GENERATION:
> - No-operation routine "print_extension" has been added
> and is called near the end of routine "generate_c_code" (line 719).
> - Code flushing has been moved from line 13162 to line 894.
> - Line 1298 has been removed.
> 
> 3) Class ET_DYNAMIC_SYSTEM:
> - Second half of routine `compile_system' has been
> separated as routine `compile_more' (called from
> `compile_system' and new code).
> - Attributes `added_type' and `added_procedure'
> have been added (set by `compile_more').
> 
> 4) Classes ET_AGENT and ET_DYNAMIC_FEATURE
> have been made HASHABLE.
> 
> 5) $GOBO/gec/runtime/c/ge_exception.[hc]:
> A `typedef' has been added as well as code
> to jump to the debugger in case of an exception
> before unwinding the stack.
> 
> Other properties:
> 
> 1) The implementation of the new classes follows the command/query
> separation principle.
> 
> 2) Accessing of the additional generated C code is done
> by `external "C inline"' functions.
> The C code has almost everywhere one effective line only.
> But there may be more physical lines like:
> #ifdef SOME_CONSTANT
>  good_C_code
> #else
>  dummy_C_code
> #endif
> 
> 3) The implementation does not (not yet?) follow
> all GOBO's style guidelines.
> 
> 
> Some problems and open questions remain:
> 
> 1) I detected a few errors in the compiler or the
> generated C code. I will report them in a separate mail.
> 
> 2) The compiler generates test for Void targets like
>      return ((T1)(((T0*)(GE_void(a1)))->id==66));
>      l6 = ((GE_void(l5), (T0*)0));
> In most cases this is welcome but is counter-productive
> in case of introspection, ... which generate references
> from C pointers: the compiler is not aware of the objects
> and often generates code where the references are
> erroneously treated as `Void'.
> For example, when restoring an object from file in independent
> store mode its contents (i.e. whether attributes are void or not)
> is beyond the scope of the actual system's compilation.
> This means that objects and their attributes (recursively to any
> depth) obtained from C pointers must be treated as potentially
> void (i.e. generate Void-tests like the 1st example above)
> but must also be treated as potentially not-void
> (i.e. *not* code like the 2nd example).

I don't think that the problem is with void checking. The
problem is with the dynamic type set builder. It is not aware
that entities can have some given types at runtime because
they will be attached to objects retrieved from storable
files. Here you see the problem with the void checking. But
the problem will be similar with dynamic binding. The
compiler might think that an entity will have only to
possible types at runtime and will generate the code for
the dynamic binding accordingly. But if, after retrieving
objects from storable files that entity can have a third
type at runtime, then the dynamic binding will be buggy.

> 3) To display more informative debuggee status the compiler
> should collect more information. For example, some compiler
> related classes have queries like `keyword: ET_KEYWORD'
> which potentially provide a keyword's position in class text.
> Unfortunately, the position is not always set.

In order for the position to be always available, you have
to use ET_DECORATED_AST_FACTORY instead of ET_AST_FACTORY.

> 4) There is one inconvenience when installing the new stuff.
> Previously, installation of GEC consists of two compilations:
> first by use of ISE, SE or an old GEC, second, by the GEC
> just obtained. The inconvenience is that one more iteration
> is necessary: compile GEC once more by itself.
> Forthcoming installations do not need the extra compilation
> as long as the structure of store files is not changed.
> 
> 5) How can a user force the compiler to generate code for debugging?
> Currently I apply the SE style: if option "trace(yes)" is set in
> the ACE file then debugging code is generated (but no tracing code).
> Another solution may be in order.
> 
> 6) The most severe problem is how the debugger can be integrated
> into a system. The problem is as follows. The debugger is written
> in Eiffel (classes DG_DEBUGGER etc.) but the classes of the
> debuggee are not aware of that class and DG_DEBUGGER would not
> be compiled. The current solution is that after compiling the
> debuggee a seemingly second system of root class DG_DEBUGGER
> is compiled (therefore, the routine `compile_more')
> and all is put into *one* C code.
> The solution is rather tricky. How can this be managed better?
> 
> 
> So, if you are interested, how should the new stuff be integrated
> into the GOBO project?
> Simply by updating via SVN? By a new SNV path (probably better
> since the new stuff still in experimental state)? Another way?

Another way would be to move the Gobo repository to git.
Jocelyn Fiat already uses git to work and maintain a void-safe
version of Gobo. It would be easier I think if the master
repository would be under git. I guess having to go back and
forth between SVN and git is not very practical.

If we make the decision to move to git, then what I'm not sure
yet is whether we should host the git repository at SourceForge,
or use github.com. What I like with github (I'm not sure whether
this is available in other git hosts like SourceForge) is that
from the Gobo master git repository we could see who is working
on his own version of Gobo in github. My feeling is that it makes
things easier to follow when we want to merge/integrate
development into the main/master Gobo repository.


> Best regards
> WJ
> 
> PS:
> - The attachment contains the user's guide of the debugger.

Thanks. I will read it.

> - GOBO version used: GOBO39, SVN 6656

-- 
Eric Bezault
mailto:er...@go...
http://www.gobosoft.com

[gobo-eiffel-develop] GEC introspection & debugger

[Wolfgang J. <wj...@so...>]

Hi,

long time ago I offered to provide code for introspection and
persistence closure to the GOBO library as well as a debugger
for then GEC. Work took more time than expected but now an
experimental version is available.


The following naming convention has been applied:

1) Introspection classes have prefix "IN_" and are located
in cluster $GOBO/library/intro.

2) Persistence classes have prefix "PC_" and are located
in cluster $GOBO/library/persist

3) Debugger classes have prefix "DG_" and are located
in cluster $GOBO/library/tools/eiffel/debugger

4) Special versions of introspection classes used
during compilation have prefix "ET_IN_" and are located
in cluster $GOBO/library/tools/eiffel/generation/intro

5) The generated C code contains additional
#defines, typedefs and global variables.
They all have the prefix "GE_Z" or "GE_z".
Of course, names and clusters may be changed.


Changes of existing classes and C code:

The additional C code necessary for introspection
and debugging (the persistence closure is a pure library
and does not need additional C code) is generated
in descendant classes of ET_C_GENERATOR which
belong to cluster $GOBO/library/tools/eiffel/generation.
There are just two changes in existing code:

1) Class GEC, lines 199... and 234...:
some local variables have been added and
      create l_generator.make (l_system)
is replaced by code that creates an object of the appropriate
descendant class.

2) Class ET_C_GENERATION:
- No-operation routine "print_extension" has been added
and is called near the end of routine "generate_c_code" (line 719).
- Code flushing has been moved from line 13162 to line 894.
- Line 1298 has been removed.

3) Class ET_DYNAMIC_SYSTEM:
- Second half of routine `compile_system' has been
separated as routine `compile_more' (called from
`compile_system' and new code).
- Attributes `added_type' and `added_procedure'
have been added (set by `compile_more').

4) Classes ET_AGENT and ET_DYNAMIC_FEATURE
have been made HASHABLE.

5) $GOBO/gec/runtime/c/ge_exception.[hc]:
A `typedef' has been added as well as code
to jump to the debugger in case of an exception
before unwinding the stack.

Other properties:

1) The implementation of the new classes follows the command/query
separation principle.

2) Accessing of the additional generated C code is done
by `external "C inline"' functions.
The C code has almost everywhere one effective line only.
But there may be more physical lines like:
#ifdef SOME_CONSTANT
  good_C_code
#else
  dummy_C_code
#endif

3) The implementation does not (not yet?) follow
all GOBO's style guidelines.


Some problems and open questions remain:

1) I detected a few errors in the compiler or the
generated C code. I will report them in a separate mail.

2) The compiler generates test for Void targets like
      return ((T1)(((T0*)(GE_void(a1)))->id==66));
      l6 = ((GE_void(l5), (T0*)0));
In most cases this is welcome but is counter-productive
in case of introspection, ... which generate references
from C pointers: the compiler is not aware of the objects
and often generates code where the references are
erroneously treated as `Void'.
For example, when restoring an object from file in independent
store mode its contents (i.e. whether attributes are void or not)
is beyond the scope of the actual system's compilation.
This means that objects and their attributes (recursively to any
depth) obtained from C pointers must be treated as potentially
void (i.e. generate Void-tests like the 1st example above)
but must also be treated as potentially not-void
(i.e. *not* code like the 2nd example).

3) To display more informative debuggee status the compiler
should collect more information. For example, some compiler
related classes have queries like `keyword: ET_KEYWORD'
which potentially provide a keyword's position in class text.
Unfortunately, the position is not always set.

4) There is one inconvenience when installing the new stuff.
Previously, installation of GEC consists of two compilations:
first by use of ISE, SE or an old GEC, second, by the GEC
just obtained. The inconvenience is that one more iteration
is necessary: compile GEC once more by itself.
Forthcoming installations do not need the extra compilation
as long as the structure of store files is not changed.

5) How can a user force the compiler to generate code for debugging?
Currently I apply the SE style: if option "trace(yes)" is set in
the ACE file then debugging code is generated (but no tracing code).
Another solution may be in order.

6) The most severe problem is how the debugger can be integrated
into a system. The problem is as follows. The debugger is written
in Eiffel (classes DG_DEBUGGER etc.) but the classes of the
debuggee are not aware of that class and DG_DEBUGGER would not
be compiled. The current solution is that after compiling the
debuggee a seemingly second system of root class DG_DEBUGGER
is compiled (therefore, the routine `compile_more')
and all is put into *one* C code.
The solution is rather tricky. How can this be managed better?


So, if you are interested, how should the new stuff be integrated
into the GOBO project?
Simply by updating via SVN? By a new SNV path (probably better
since the new stuff still in experimental state)? Another way?

Best regards
WJ

PS:
- The attachment contains the user's guide of the debugger.
- GOBO version used: GOBO39, SVN 6656

-- 
Dr. Wolfgang Jansen
University of Potsdam, Germany
mailto: wj...@so...

Re: [gobo-eiffel-develop] Bug in DS_LEFT_LEANING_RED_BLACK_TREE_SET.walk_left

[Daniel T. <dan...@gm...>]

Hi Eric
I will correct that as soon as possible.

Eric Bezault wrote:
> Hi Daniel,
>
> When I run the test cases with all assertions enabled, I get this
> loop invariant violation.
>
>
>
> C:\gobo\test\structure>geant test_debug_ise
>
> Testing xstructure...
> Preparing Test Cases
> Compiling Test Cases
> Running Test Cases
>
> Test Summary for xstructure
>
> # Passed:     199 tests
> # Failed:     0 test
> # ABORTED:    1 test
> # Total:      200 tests (1674 assertions)
>
> Test Results:
> ABORT:  [DS_TEST_LEFT_LEANING_RED_BLACK_TREE_SET.test_set_operations] 
> Eiffel exeption
> ------------------------------------------------------------------------------- 
>
> Class / Object      Routine                Nature of exception  Effect
> ------------------------------------------------------------------------------- 
>
> DS_LEFT_LEANING_RED_BLACK_TREE_SET
>                     walk_left @6           result_is_node_2:
> <0000000002236D5C>  (From DS_LEFT_LEANING_RED_BLACK_TREE_CONTAINER)
>                                            Loop invariant violated.   
> Fail
> ------------------------------------------------------------------------------- 
>
> DS_LEFT_LEANING_RED_BLACK_TREE_SET
>                     walk_left @6
> <0000000002236D5C>  (From DS_LEFT_LEANING_RED_BLACK_TREE_CONTAINER)
>                                            Routine failure.   Fail
> ------------------------------------------------------------------------------- 
>
> DS_LEFT_LEANING_RED_BLACK_TREE_SET
>                     successor_for_removal @4
> <0000000002236D5C>  (From DS_LEFT_LEANING_RED_BLACK_TREE_CONTAINER)
>                                            Routine failure.   Fail
> ------------------------------------------------------------------------------- 
>
> DS_LEFT_LEANING_RED_BLACK_TREE_SET
>                     remove_node @11
> <0000000002236D5C>  (From DS_BINARY_SEARCH_TREE_CONTAINER)
>                                            Routine failure.   Fail
> ------------------------------------------------------------------------------- 
>
> DS_LEFT_LEANING_RED_BLACK_TREE_SET
>                     subtract @13
> <0000000002236D5C>  (From DS_BINARY_SEARCH_TREE_SET)
>                                            Routine failure.   Fail
> ------------------------------------------------------------------------------- 
>
> DS_LEFT_LEANING_RED_BLACK_TREE_SET
>                     subtraction @4
> <0000000002236B28>  (From DS_SET)          Routine failure.  Fail
> ------------------------------------------------------------------------------- 
>
> DS_TEST_LEFT_LEANING_RED_BLACK_TREE_SET
>                     test_set_operations @12
> <00000000021DAA70>                         Routine failure.  Fail
> ------------------------------------------------------------------------------- 
>
> PROCEDURE           fast_call
> <00000000021DAAD0>                         Routine failure.  Fail
> ------------------------------------------------------------------------------- 
>
> PROCEDURE           call @5
> <00000000021DAAD0>                         Routine failure.  Fail
> ------------------------------------------------------------------------------- 
>
> DS_TEST_LEFT_LEANING_RED_BLACK_TREE_SET
>                     execute_without_rescue @4
> <00000000021DAA70>  (From TS_TEST_CASE)    Routine failure.  Fail
> ------------------------------------------------------------------------------- 
>
> DS_TEST_LEFT_LEANING_RED_BLACK_TREE_SET
>                     execute_with_rescue @3
> <00000000021DAA70>  (From TS_TEST_CASE)    Routine failure.  Fail
> ------------------------------------------------------------------------------- 
>
> DS_TEST_LEFT_LEANING_RED_BLACK_TREE_SET
>                     execute @9
> <00000000021DAA70>  (From TS_TEST_CASE)    Routine failure.  Fail
> ------------------------------------------------------------------------------- 
>
> TS_TEST_SUITE       execute @5
> <00000000021D1698>                         Routine failure.  Fail
> ------------------------------------------------------------------------------- 
>
> XSTRUCTURE          execute_with_summary @7
> <00000000021D03FC>  (From TS_TESTER)       Routine failure.  Fail
> ------------------------------------------------------------------------------- 
>
> XSTRUCTURE          execute_with_output @10
> <00000000021D03FC>  (From TS_TESTER)       Routine failure.  Fail
> ------------------------------------------------------------------------------- 
>
> XSTRUCTURE          execute @9
> <00000000021D03FC>  (From TS_TESTER)       Routine failure.  Fail
> ------------------------------------------------------------------------------- 
>
> XSTRUCTURE          make @5
> <00000000021D03FC>  (From TS_TESTER)       Routine failure.  Fail
> ------------------------------------------------------------------------------- 
>
> XSTRUCTURE          root's creation
> <00000000021D03FC>                         Routine failure.  Exit
> ------------------------------------------------------------------------------- 
>
>
>
> BUILD FAILED!
>

[gobo-eiffel-develop] Bug in DS_LEFT_LEANING_RED_BLACK_TREE_SET.walk_left

[Eric B. <er...@go...>]

Hi Daniel,

When I run the test cases with all assertions enabled, I get this
loop invariant violation.



C:\gobo\test\structure>geant test_debug_ise

Testing xstructure...
Preparing Test Cases
Compiling Test Cases
Running Test Cases

Test Summary for xstructure

# Passed:     199 tests
# Failed:     0 test
# ABORTED:    1 test
# Total:      200 tests (1674 assertions)

Test Results:
ABORT:  [DS_TEST_LEFT_LEANING_RED_BLACK_TREE_SET.test_set_operations] 
Eiffel exeption
-------------------------------------------------------------------------------
Class / Object      Routine                Nature of exception 
  Effect
-------------------------------------------------------------------------------
DS_LEFT_LEANING_RED_BLACK_TREE_SET
                     walk_left @6           result_is_node_2:
<0000000002236D5C>  (From DS_LEFT_LEANING_RED_BLACK_TREE_CONTAINER)
                                            Loop invariant violated. 
   Fail
-------------------------------------------------------------------------------
DS_LEFT_LEANING_RED_BLACK_TREE_SET
                     walk_left @6
<0000000002236D5C>  (From DS_LEFT_LEANING_RED_BLACK_TREE_CONTAINER)
                                            Routine failure. 
   Fail
-------------------------------------------------------------------------------
DS_LEFT_LEANING_RED_BLACK_TREE_SET
                     successor_for_removal @4
<0000000002236D5C>  (From DS_LEFT_LEANING_RED_BLACK_TREE_CONTAINER)
                                            Routine failure. 
   Fail
-------------------------------------------------------------------------------
DS_LEFT_LEANING_RED_BLACK_TREE_SET
                     remove_node @11
<0000000002236D5C>  (From DS_BINARY_SEARCH_TREE_CONTAINER)
                                            Routine failure. 
   Fail
-------------------------------------------------------------------------------
DS_LEFT_LEANING_RED_BLACK_TREE_SET
                     subtract @13
<0000000002236D5C>  (From DS_BINARY_SEARCH_TREE_SET)
                                            Routine failure. 
   Fail
-------------------------------------------------------------------------------
DS_LEFT_LEANING_RED_BLACK_TREE_SET
                     subtraction @4
<0000000002236B28>  (From DS_SET)          Routine failure. 
  Fail
-------------------------------------------------------------------------------
DS_TEST_LEFT_LEANING_RED_BLACK_TREE_SET
                     test_set_operations @12
<00000000021DAA70>                         Routine failure. 
  Fail
-------------------------------------------------------------------------------
PROCEDURE           fast_call
<00000000021DAAD0>                         Routine failure. 
  Fail
-------------------------------------------------------------------------------
PROCEDURE           call @5
<00000000021DAAD0>                         Routine failure. 
  Fail
-------------------------------------------------------------------------------
DS_TEST_LEFT_LEANING_RED_BLACK_TREE_SET
                     execute_without_rescue @4
<00000000021DAA70>  (From TS_TEST_CASE)    Routine failure. 
  Fail
-------------------------------------------------------------------------------
DS_TEST_LEFT_LEANING_RED_BLACK_TREE_SET
                     execute_with_rescue @3
<00000000021DAA70>  (From TS_TEST_CASE)    Routine failure. 
  Fail
-------------------------------------------------------------------------------
DS_TEST_LEFT_LEANING_RED_BLACK_TREE_SET
                     execute @9
<00000000021DAA70>  (From TS_TEST_CASE)    Routine failure. 
  Fail
-------------------------------------------------------------------------------
TS_TEST_SUITE       execute @5
<00000000021D1698>                         Routine failure. 
  Fail
-------------------------------------------------------------------------------
XSTRUCTURE          execute_with_summary @7
<00000000021D03FC>  (From TS_TESTER)       Routine failure. 
  Fail
-------------------------------------------------------------------------------
XSTRUCTURE          execute_with_output @10
<00000000021D03FC>  (From TS_TESTER)       Routine failure. 
  Fail
-------------------------------------------------------------------------------
XSTRUCTURE          execute @9
<00000000021D03FC>  (From TS_TESTER)       Routine failure. 
  Fail
-------------------------------------------------------------------------------
XSTRUCTURE          make @5
<00000000021D03FC>  (From TS_TESTER)       Routine failure. 
  Fail
-------------------------------------------------------------------------------
XSTRUCTURE          root's creation
<00000000021D03FC>                         Routine failure. 
  Exit
-------------------------------------------------------------------------------


BUILD FAILED!

-- 
Eric Bezault
mailto:er...@go...
http://www.gobosoft.com

Re: [gobo-eiffel-develop] Bug in base64 encoder

[Berend de B. <be...@po...>]

>>>>> "Jann" == Jann Röder <roe...@et...> writes:

    Jann> This is the example code that reproduces the problem:

Let me just say this type prefixing looks awful.

What are you going to do when the type changes?

-- 
Cheers,

Berend de Boer

Re: [gobo-eiffel-develop] Bug in base64 encoder

[Jann R. <roe...@et...>]

Ok,
this was my fault. I forgot to call close on the encoder.
Sorry about that.

Jann

Jann Röder schrieb:
> Hello,
> I am using the gobo base64 encoder to encode the managed_pointer output 
> generated by the SED serializer. And now I found a strange bug. In some 
> cases encoding and decoding of a storable works and in others it 
> doesn't. The decoded string is one byte too short. In the example below 
> the encoding and decoding of the array <<"hello", "world">> doesn't 
> work, while the array <<"">> can be decoded just fine.
> 
> This is the example code that reproduces the problem:
> 
> test_base_64_serialization_report is
> 		-- Test the base64 encoder
> 	local
> 		l_ptr, l_ptr2: MANAGED_POINTER
> 		l_array: ARRAY[STRING]
> 		l_facilities: SED_STORABLE_FACILITIES
> 		l_rw: SED_MEMORY_READER_WRITER
> 		l_i: INTEGER
> 		l_str, l_out_str: STRING
> 		l_blob_string: UC_UTF8_STRING
> 		l_encoder: UT_BASE64_ENCODING_OUTPUT_STREAM
> 		l_input_stream: KL_STRING_INPUT_STREAM
> 		l_decoder: UT_BASE64_DECODING_INPUT_STREAM
> 	do
> 		create l_facilities
> 		create l_rw.make
> 
> 		l_array := << "hello", "world" >>
> 		--l_array := <<"">>
> 
> 		l_facilities.independent_store (l_array, l_rw, True)
> 		l_ptr := l_rw.data
> 
> 		-- Encode
> 		create l_blob_string.make (1000)
> 		create l_encoder.make (l_blob_string, False, False)
> 		from
> 			l_i := 0
> 		until
> 			l_i >= l_ptr.count
> 		loop
> 			l_encoder.put_character (l_ptr.read_character (l_i))
> 			l_i := l_i + 1
> 		end
> 		l_str := l_blob_string.to_utf8
> 
> 		-- Decode
> 		create l_input_stream.make (l_str)
> 		create l_decoder.make (l_input_stream)
> 		from
> 			create l_out_str.make (l_str.count)
> 		until
> 			l_decoder.end_of_input
> 		loop
> 			l_decoder.read_string (l_str.count)
> 			l_out_str.append (l_decoder.last_string)
> 		end
> 		create l_ptr2.make_from_pointer (l_out_str.area.base_address, 
> l_out_str.count)
> 		check
> 			l_ptr ~ l_ptr2
> 		end
> 	end
> 
> Thanks,
> Jann
> 
> ------------------------------------------------------------------------------
> Enter the BlackBerry Developer Challenge  
> This is your chance to win up to $100,000 in prizes! For a limited time, 
> vendors submitting new applications to BlackBerry App World(TM) will have
> the opportunity to enter the BlackBerry Developer Challenge. See full prize  
> details at: http://p.sf.net/sfu/Challenge
> _______________________________________________
> gobo-eiffel-develop mailing list
> gob...@li...
> https://lists.sourceforge.net/lists/listinfo/gobo-eiffel-develop

[gobo-eiffel-develop] Bug in base64 encoder

[Jann R. <roe...@et...>]

Hello,
I am using the gobo base64 encoder to encode the managed_pointer output 
generated by the SED serializer. And now I found a strange bug. In some 
cases encoding and decoding of a storable works and in others it 
doesn't. The decoded string is one byte too short. In the example below 
the encoding and decoding of the array <<"hello", "world">> doesn't 
work, while the array <<"">> can be decoded just fine.

This is the example code that reproduces the problem:

test_base_64_serialization_report is
		-- Test the base64 encoder
	local
		l_ptr, l_ptr2: MANAGED_POINTER
		l_array: ARRAY[STRING]
		l_facilities: SED_STORABLE_FACILITIES
		l_rw: SED_MEMORY_READER_WRITER
		l_i: INTEGER
		l_str, l_out_str: STRING
		l_blob_string: UC_UTF8_STRING
		l_encoder: UT_BASE64_ENCODING_OUTPUT_STREAM
		l_input_stream: KL_STRING_INPUT_STREAM
		l_decoder: UT_BASE64_DECODING_INPUT_STREAM
	do
		create l_facilities
		create l_rw.make

		l_array := << "hello", "world" >>
		--l_array := <<"">>

		l_facilities.independent_store (l_array, l_rw, True)
		l_ptr := l_rw.data

		-- Encode
		create l_blob_string.make (1000)
		create l_encoder.make (l_blob_string, False, False)
		from
			l_i := 0
		until
			l_i >= l_ptr.count
		loop
			l_encoder.put_character (l_ptr.read_character (l_i))
			l_i := l_i + 1
		end
		l_str := l_blob_string.to_utf8

		-- Decode
		create l_input_stream.make (l_str)
		create l_decoder.make (l_input_stream)
		from
			create l_out_str.make (l_str.count)
		until
			l_decoder.end_of_input
		loop
			l_decoder.read_string (l_str.count)
			l_out_str.append (l_decoder.last_string)
		end
		create l_ptr2.make_from_pointer (l_out_str.area.base_address, 
l_out_str.count)
		check
			l_ptr ~ l_ptr2
		end
	end

Thanks,
Jann

Re: [gobo-eiffel-develop] DS_BINARY_SEARCH_TREE status ?

[Jann R. <roe...@et...>]

Thanks Daniel for the quick fix.
My code works fine now!

Jann

Daniel Tuser schrieb:
> Hi Jann
> All the bugs you reported should now be corrected (in SVN). Please let 
> me now if you still have trouble with the code. Thanks for reporting the 
> bugs.
> 
> Regards,
> Daniel

Re: [gobo-eiffel-develop] DS_BINARY_SEARCH_TREE status ?

[Daniel T. <dan...@gm...>]

Hi Jann
All the bugs you reported should now be corrected (in SVN). Please let 
me now if you still have trouble with the code. Thanks for reporting the 
bugs.

Regards,
Daniel

Re: [gobo-eiffel-develop] DS_BINARY_SEARCH_TREE status ?

[Daniel T. <dan...@gm...>]

Hi Jann
The bug in `subtract' should now be corrected.

Regards
Daniel

Re: [gobo-eiffel-develop] DS_BINARY_SEARCH_TREE status ?

[Daniel T. <dan...@gm...>]

Jann Röder wrote:
> Hi,
> I was wondering what the status of the DS_BINARY_SEARCH_TREE_* classes 
> is. I was trying to use the DS_RED_BLACK_TREE_SET and found it to be 
> quite buggy. I already spent several hours trying to fix the subtract 
> feature. But fixing one bug always seems to uncover another one, which 
> is quite frustrating.
>   
Hi Jann
I am not aware of bugs in the implementation. All tests passed as far as 
I know. If you send me a reproduction, I may have a look at the problem.
> Thanks,
> Jann

Re: [gobo-eiffel-develop] DS_BINARY_SEARCH_TREE status ?

[Daniel T. <dan...@gm...>]

Eric Bezault wrote:
> Hi Daniel,
>
> You might also want to have a look at this other bug report that
> Jann sent to the Gobo users mailing list:
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Hi,
> there seems to be a bug in {DS_BINARY_SEARCH_TREE_SET}.copy .
>
> The problem is, that internal_cursor is still Void when copy is called 
> (by .twin). The Void call happens on internal_cursor.off .
>
> Jann
I missed that. I will correct it as soon as possible.

Re: [gobo-eiffel-develop] DS_BINARY_SEARCH_TREE status ?

[Daniel T. <dan...@gm...>]

Jann Röder wrote:
> Hi Daniel,
> I don't have a sample for all bugs. I will probably try to reproduce 
> it again later, but for starters:
I was able to reproduce the bug and know how to correct it. The problem 
is that one should never iterate like that through a 
DS_BINARY_SEARCH_TREE_CONTAINER and use `remove_node'. I will correct 
that as soon as possible. The problem is likely to be present in all set 
operations that remove items - certainly only in the binary search tree 
classes.
> * subtract does not reset the found_item, thus causing weird behavior 
> in has_key. The has_key implementation seems strange anyway. Why do 
> you use a class variable as iterator. Locals are faster and easier to 
> understand. I think the search feature should set found item to Void 
> in the beginning.
`found_node' is used to avoid redundant searches through the tree in 
`search_node'. E.g. if you first call `has (a_key)' and then `at 
(a_key)', then during the second call, the tree does not need to be 
traversed, as the result is already present in `found_node'.
At the time when I implemented all those classes I was not aware of the 
huge performance difference between attributes and local variables. I 
will change that as well in the loop.

Thanks for reporting the problem

Re: [gobo-eiffel-develop] DS_BINARY_SEARCH_TREE status ?

[Eric B. <er...@go...>]

Hi Daniel,

You might also want to have a look at this other bug report that
Jann sent to the Gobo users mailing list:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Hi,
there seems to be a bug in {DS_BINARY_SEARCH_TREE_SET}.copy .

The problem is, that internal_cursor is still Void when copy is called 
(by .twin). The Void call happens on internal_cursor.off .

Jann
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Jann Röder wrote:
> Hi Daniel,
> I don't have a sample for all bugs. I will probably try to reproduce it 
> again later, but for starters:
> 
> * subtract does not reset the found_item, thus causing weird behavior in 
> has_key. The has_key implementation seems strange anyway. Why do you use 
> a class variable as iterator. Locals are faster and easier to 
> understand. I think the search feature should set found_item to Void in 
> the beginning.
> 
> * subtract seems to remove the wrong node. It seems to remove the 
> successor of the node that holds the item. In my code the postcondition 
> is_disjoint fails.
> 
> * After changing subtract to remove the node which contains the item, I 
> got other weird contract violations (it claimed that l_node was not in 
> the same tree)
> 
> Jann
> 
> Daniel Tuser schrieb:
>> Jann Röder wrote:
>>> Hi,
>>> I was wondering what the status of the DS_BINARY_SEARCH_TREE_* classes 
>>> is. I was trying to use the DS_RED_BLACK_TREE_SET and found it to be 
>>> quite buggy. I already spent several hours trying to fix the subtract 
>>> feature. But fixing one bug always seems to uncover another one, which 
>>> is quite frustrating.
>>>   
>> Hi Jann
>> I am not aware of bugs in the implementation. All tests passed as far as 
>> I know. If you send me a reproduction, I may have a look at the problem.
>>> Thanks,
>>> Jann
> 


-- 
Eric Bezault
mailto:er...@go...
http://www.gobosoft.com

Re: [gobo-eiffel-develop] DS_BINARY_SEARCH_TREE status ?

[Jann R. <roe...@et...>]

Hi Daniel,
I don't have a sample for all bugs. I will probably try to reproduce it 
again later, but for starters:

* subtract does not reset the found_item, thus causing weird behavior in 
has_key. The has_key implementation seems strange anyway. Why do you use 
a class variable as iterator. Locals are faster and easier to 
understand. I think the search feature should set found_item to Void in 
the beginning.

* subtract seems to remove the wrong node. It seems to remove the 
successor of the node that holds the item. In my code the postcondition 
is_disjoint fails.

* After changing subtract to remove the node which contains the item, I 
got other weird contract violations (it claimed that l_node was not in 
the same tree)

Jann

Daniel Tuser schrieb:
> Jann Röder wrote:
>> Hi,
>> I was wondering what the status of the DS_BINARY_SEARCH_TREE_* classes 
>> is. I was trying to use the DS_RED_BLACK_TREE_SET and found it to be 
>> quite buggy. I already spent several hours trying to fix the subtract 
>> feature. But fixing one bug always seems to uncover another one, which 
>> is quite frustrating.
>>   
> Hi Jann
> I am not aware of bugs in the implementation. All tests passed as far as 
> I know. If you send me a reproduction, I may have a look at the problem.
>> Thanks,
>> Jann

[gobo-eiffel-develop] DS_BINARY_SEARCH_TREE status ?

[Jann R. <roe...@et...>]

Hi,
I was wondering what the status of the DS_BINARY_SEARCH_TREE_* classes 
is. I was trying to use the DS_RED_BLACK_TREE_SET and found it to be 
quite buggy. I already spent several hours trying to fix the subtract 
feature. But fixing one bug always seems to uncover another one, which 
is quite frustrating.

Thanks,
Jann

Re: [gobo-eiffel-develop] YY_SCANNER_SKELETON.yy_content_area could be ABSTRACT_SPECIAL ?

[Eric B. <er...@go...>]

Eric Bezault wrote:
> YY_SCANNER_SKELETON has been changed in such as way because during
> the current migration to void-safety KI_CHARACTER_BUFFER.as_special
> has been turned to return a non-void SPECIAL. I think that the problem
> comes from here. Your version of class buffer is now likely to
> violate this new postcondition in `as_special'.
> 
> I'll see what I can do to revert this change. I must say that I'm
> not a fan of all this void-safety thing. For code that was well
> equipped with assertions like the Gobo libraries, converting to
> void-safety makes the code more convoluted, hence harder to read
> and maintain.

KI_CHARACTER_BUFFER.as_special can now be void, as it used to be.
The scanner skeleton classes have been updated accordingly.

-- 
Eric Bezault
mailto:er...@go...
http://www.gobosoft.com

Re: [gobo-eiffel-develop] YY_SCANNER_SKELETON.yy_content_area could be ABSTRACT_SPECIAL ?

[Eric B. <er...@go...>]

Berend de Boer wrote:
>>>>>> "Eric" == Eric Bezault <er...@go...> writes:
> 
>     Eric> In my mind, it has always been clear that we should use
>     Eric> SPECIAL for efficency reason. The reason why this could be
>     Eric> bypassed is that at the time of writing some of the
>     Eric> supported Eiffel compilers did not support SPECIAL. Using
>     Eric> ABSTRACT_SPECIAL is not really a solution because it
>     Eric> introduces polymorphism, hence reducing the performance that
>     Eric> we want to gain by using SPECIAL directly.
> 
> But with SPECIAL comes the need for a copy. Which is a bit slow
> especially in debug mode. It's not a huge problem, but it would be
> nice if SPECIAL had a fast way to copy data from a given pointer.

In all cases that I had so far, there was no need to copy to a
SPECIAL because the SPECIAL was already available. Hence the
optimization. I had not faced the problem you have with a pointer
because I always try to write pure Eiffel, hence no pointer to
deal with.

-- 
Eric Bezault
mailto:er...@go...
http://www.gobosoft.com

Re: [gobo-eiffel-develop] YY_SCANNER_SKELETON.yy_content_area could be ABSTRACT_SPECIAL ?

[Berend de B. <be...@po...>]

>>>>> "Eric" == Eric Bezault <er...@go...> writes:

    Eric> In my mind, it has always been clear that we should use
    Eric> SPECIAL for efficency reason. The reason why this could be
    Eric> bypassed is that at the time of writing some of the
    Eric> supported Eiffel compilers did not support SPECIAL. Using
    Eric> ABSTRACT_SPECIAL is not really a solution because it
    Eric> introduces polymorphism, hence reducing the performance that
    Eric> we want to gain by using SPECIAL directly.

But with SPECIAL comes the need for a copy. Which is a bit slow
especially in debug mode. It's not a huge problem, but it would be
nice if SPECIAL had a fast way to copy data from a given pointer.


    Eric> YY_SCANNER_SKELETON has been changed in such as way because
    Eric> during the current migration to void-safety
    Eric> KI_CHARACTER_BUFFER.as_special has been turned to return a
    Eric> non-void SPECIAL. I think that the problem comes from
    Eric> here. Your version of class buffer is now likely to violate
    Eric> this new postcondition in `as_special'.

Indeed, saw the comment and I indeed inherit from KI_CHARACTER_BUFFER.


    Eric> I'll see what I can do to revert this change. I must say
    Eric> that I'm not a fan of all this void-safety thing. For code
    Eric> that was well equipped with assertions like the Gobo
    Eric> libraries, converting to void-safety makes the code more
    Eric> convoluted, hence harder to read and maintain.

Interesting perspective.

But I suppose it's a hard call. Without SPECIAL reading from the might
be a bit slower, but if you need t copy to special first + read
from SPECIAL I think the overall case has become a bit slower.
-- 
Cheers,

Berend de Boer

Re: [gobo-eiffel-develop] YY_SCANNER_SKELETON.yy_content_area could be ABSTRACT_SPECIAL ?

[Eric B. <er...@go...>]

Berend de Boer wrote:
>>>>>> "Emmanuel" == Emmanuel Stapf [ES] <ma...@ei...> writes:
> 
>     >> So the goal is to run the scanner/parser straight from a malloc
>     >> buffer, which used to work.
> 
>     Emmanuel> Having done many C/Eiffel interfaces I consider this as
>     Emmanuel> being dangerous to do. And why when using SPECIAL do you
>     Emmanuel> want to pass this to the C side? Is the C code doing to
>     Emmanuel> do some processing over the data?
> 
> Nah, it's just the other way around. The data is in a malloc buffer,
> and needs to be parsed by YY_PARSER. Previously one could write a
> wrapper where data was picked up from such buffer. Now Gobo requires
> that it is in a SPECIAL or descendant.

In my mind, it has always been clear that we should use SPECIAL
for efficency reason. The reason why this could be bypassed is
that at the time of writing some of the supported Eiffel compilers
did not support SPECIAL. Using ABSTRACT_SPECIAL is not really
a solution because it introduces polymorphism, hence reducing
the performance that we want to gain by using SPECIAL directly.

YY_SCANNER_SKELETON has been changed in such as way because during
the current migration to void-safety KI_CHARACTER_BUFFER.as_special
has been turned to return a non-void SPECIAL. I think that the problem
comes from here. Your version of class buffer is now likely to
violate this new postcondition in `as_special'.

I'll see what I can do to revert this change. I must say that I'm
not a fan of all this void-safety thing. For code that was well
equipped with assertions like the Gobo libraries, converting to
void-safety makes the code more convoluted, hence harder to read
and maintain.

-- 
Eric Bezault
mailto:er...@go...
http://www.gobosoft.com

Re: [gobo-eiffel-develop] YY_SCANNER_SKELETON.yy_content_area could be ABSTRACT_SPECIAL ?

[Berend de B. <be...@po...>]

>>>>> "Emmanuel" == Emmanuel Stapf [ES] <ma...@ei...> writes:

    >> So the goal is to run the scanner/parser straight from a malloc
    >> buffer, which used to work.

    Emmanuel> Having done many C/Eiffel interfaces I consider this as
    Emmanuel> being dangerous to do. And why when using SPECIAL do you
    Emmanuel> want to pass this to the C side? Is the C code doing to
    Emmanuel> do some processing over the data?

Nah, it's just the other way around. The data is in a malloc buffer,
and needs to be parsed by YY_PARSER. Previously one could write a
wrapper where data was picked up from such buffer. Now Gobo requires
that it is in a SPECIAL or descendant.

So I'm just trying to see if I can skip this copy or make it faster.

-- 
Cheers,

Berend de Boer

Re: [gobo-eiffel-develop] YY_SCANNER_SKELETON.yy_content_area could be ABSTRACT_SPECIAL ?

[Emmanuel S. [ES] <ma...@ei...>]

> So the goal is to run the scanner/parser straight from a malloc
> buffer, which used to work.

Having done many C/Eiffel interfaces I consider this as being dangerous to do. And
why when using SPECIAL do you want to pass this to the C side? Is the C code doing
to do some processing over the data?

Regards,
Manu

[gobo-eiffel-develop] YY_SCANNER_SKELETON.yy_content_area could be ABSTRACT_SPECIAL ?

[Berend de B. <be...@po...>]

Hi All,

A recent change in YY_COMPRESSED_SCANNER_SKELETON requires that
yy_content_area be set. Unfortunately I relied on the fact you didn't
need to set it, so I could avoid a large memory copy.

I'm not yet sure what to do. What I really need is a SPECIAL with a
valid base_address, but can I rely on ISE Eiffel not to move this? I'm
afraid I can't.

Another solution is if my C malloc wrapper (STDC_BUFFER) implements
ABSTRACT_SPECIAL, so that would require
YY_SCANNER_SKELETON.yy_content_area to be changed to use
ABSTRACT_SPECIAL.

Any thoughts?

So the goal is to run the scanner/parser straight from a malloc
buffer, which used to work.

-- 
Cheers,

Berend de Boer

Re: [gobo-eiffel-develop] Converting UC_STRING to/from C pointer to UTF8 octet sequences

[Paul C. <pa...@se...>]

Hi,

Thanks Berend for your reply. For some reason I managed to miss your
reply and didn't see it until now.

On Tue, May 19, 2009 at 8:03 AM, Berend de Boer <be...@po...> wrote:
>>>>>> "Paul" == Paul Cohen <pa...@se...> writes:
>    Paul> Hi all, I need some advice/feedback on how to convert/create
>    Paul> a UC_STRING from a C pointer to a UTF8 byte sequence (and
>    Paul> vice versa). I can't find any routines or examples in the
>    Paul> Gobo unicode library for doing it. The need for this arises
>    Paul> when writing Eiffel shared libraries (with C-style API:s).
>
>    Paul> It is unfortunate that I have to create two "string objects"
>    Paul> to perform the "conversion".
>
> You can iterate over the C pointer and do what make_from_utf8 does if
> this is a concern.

Nice. Thanks.

>    Paul> Returning a pointer to a UTF8 octet sequence is trickier.
>
> It is. I've found out that the only safe way to do this in ISE is to
> allocate memory with C (malloc), copy the string to there and give a
> pointer to that area to C.

Ok. That what was I was thinking.

> This was tested on a large production system with millions of such
> calls, and any attempt to give pointers to an Eiffel object would fail
> with weird errors after a while. There appeared to be no safe way to
> retrieve a pointer to an ISE Eiffel object, whatever tricks were
> applied.

Hmm. That's what I feared.

Berend, do you have any comment/opinion on the API design where the
caller supplies allocated memory of some size and where the supplier
(Eiffel side) can:

  1. Fill the allocated memory with the result, if the memory is large enough.

  2. Return an error code indicating that the memory provided is to
small to return the result.

Personally I think this is a rather good design principle for C API:s
and makes it easier to use threads on the client side. However it
makes it a little bit tricker to wrap the C API from Python, Ruby or
other languages.

It would be nice if one could mark given runtime objects or a class as
"uncollectable" by the garbage collector, so that they are not touched
(collected or moved) even if there are no Eiffel references to it.
Maybe something for the Gobo Eiffel compiler? Even nicer would be if
one could create such uncollectable objects on a once per thread
basis. It would make it much easier to write thread-safe C style API:s
in Eiffel!

/Paul

-- 
Paul Cohen
www.seibostudios.se
mobile: +46 730 787 035
e-mail: pau...@se...