Fnorb / Bugs / #27 TAO-Fnorb interoperability on indirection within typecode

Derek Thomson - 2005-02-09

Logged In: YES
user_id=435731

Hi Danilo,

We've received your problem report, but I personally won't
have a chance to look at this until the weekend. In the
meantime, could you try this against Fnorb 1.3?

Regards,
Derek.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Danilo Turina - 2005-02-09

Logged In: YES
user_id=232289

I also tried with 1.3, but I have the same problem:

File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/BOA.py",
line 281, in _fnorb_mainloop
protocol.start(timeout)
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/IIOPProtocol.py",
line 74, in start
self.__reactor.start_event_loop(timeout)
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/SelectReactor.py",
line 143, in start_event_loop
self.do_one_event(timeout)
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/SelectReactor.py",
line 184, in do_one_event
handler.handle_event(Reactor.READ)
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/GIOPServerWorkerReactive.py",
line 155, in handle_event
self.__handler.recv()
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/GIOPConnectionHandler.py",
line 191, in recv
self.__listener.message_received(
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/GIOPServerWorkerReactive.py",
line 198, in message_received
self.__giop_server.process_giop_message(message)
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/GIOPServer.py",
line 167, in process_giop_message
self.__oa._fnorb_request((self, request_header, cursor))
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/BOA.py",
line 355, in _fnorb_request
self.__process_request(request)
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/BOA.py",
line 479, in __process_request
apply(method, (server_request,))
File
"/opt/testenvironment/VLAN711/simu/adapt_simul/lib/AdaptationEmlNeMgr_skel/__init__.py",
line 383, in _skel_uploadTerminations
arguments = server_request.arguments()
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/DSI.py",
line 92, in arguments
arguments.append(tc._fnorb_unmarshal_value(self.__cursor))
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/TypeCode.py",
line 921, in _fnorb_unmarshal_value
result._fnorb_unmarshal(cursor)
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/Any.py",
line 85, in _fnorb_unmarshal
self.__typecode =
CORBA.TypeCodeFactory_init().unmarshal(cursor)
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/TypeCode.py",
line 3264, in unmarshal
typecode._fnorb_unmarshal(cursor)
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/TypeCode.py",
line 1676, in _fnorb_unmarshal
member_type = tcf.unmarshal(encapsulation_cursor)
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/TypeCode.py",
line 3243, in unmarshal
raise 'Invalid indirection for recursive typecode.'
Invalid indirection for recursive typecode.

N.B. I told you the wrong structure, in my first description
of the bug, the correct one is:

struct ArgsIn {
Ident id;
boolean flag;
Ident childId;
};

I tried to debug a bit the TypeCode.py portion of the code
where the problem seems to happen:

3240 # Find the repeated/recursive typecode
in the stack.
3241 for (typecode, offset) in
cursor.stack().items():
3242 if offset == mark + indirection:
3243 break
3244 else:
3245 raise 'Invalid indirection for
recursive typecode.'

and I found that, when the "childId" portion of the typecode
it's unmarshaled, in cursor.stack().items() there's one item
whose offset is 64L, while it should be equal to mark +
indirection = 412L + (-168) = 244L (I checked the typecode
with the debugger, and the replicated type is exactly at 244L).

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Derek Thomson - 2005-02-10

Logged In: YES
user_id=435731

Hi Danilo,

One more thing to try - could you check out the latest
version of Fnorb from CVS and see if it still fails?

IIRC we had some problems directly related to "long long"
(or more specifically, to do with 4/8 byte padding boundary
issues that using "long long" tends to provoke) a couple of
months ago that have since been fixed, so with any luck
that's causing your problem as well. Anyway, give it a go
and see what happens.

Regards,
Derek.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Danilo Turina - 2005-02-11

Logged In: YES
user_id=232289

I've never used CVS and, in addition, my PC is behind a very
restrictive firewall.
If there's a way to get the lastest non-released version of
Fnorb through the web access of CVS (without downloading
file per file), let me know.
Otherwise you could send a tar.gz with that very latest
version (for example using the attachment function of this
page).

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Derek Thomson - 2005-02-13

Logged In: YES
user_id=435731

Hi Danilo,

Unfortunately, I've had to spend this weekend at work, so I
couldn't try your example against TAO.

What I will do instead is package up a CVS checkout of Fnorb
from the head of the repository and attach it to this bug
report, as you suggest. Can you try that and see if it fixes
your problem?

Regards,
Derek.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Derek Thomson - 2005-02-13

Logged In: YES
user_id=435731

Danilo,

I have a source distribution packed up for you, but
SourceForge won't allow an attachment greater than 256K. The
package is about 326K. Can I simply email it to you, as it
is not really very large?

Derek.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Danilo Turina - 2005-02-13

Logged In: YES
user_id=232289

Of course Derek, I don't know if there's a way to use
SourceForge to send that package to my mailbox, so use this
address:
user: turina
domain: libero.it
(just to avoid email harvesters of spammers)

Thank you.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Danilo Turina - 2005-02-15

Logged In: YES
user_id=232289

Derek,
did you send me anything? Because I received nothing.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Derek Thomson - 2005-02-15

Logged In: YES
user_id=435731

Sorry, I've been busy. I just sent it. Tell us how it goes.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Danilo Turina - 2005-02-16

Logged In: YES
user_id=232289

Thank you for the package.
Unfortunately, now the process fails to start:

Traceback (most recent call last):
File "mysimu.py", line 1061, in ?
stringified_ior_myMgr = orb.object_to_string(myMgr)
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/CORBA.py",
line 305, in object_to_string
ior = object._fnorb_ior()
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/CORBA.py",
line 1714, in _fnorb_ior
boa = self.__boa
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/CORBA.py",
line 1617, in __getattr__
raise AttributeError, name
AttributeError: _Object_skel__boa

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Derek Thomson - 2005-02-16

Logged In: YES
user_id=435731

Urk. How annoying. I've not seen this one before. Welcome to
Fnorb 1.4 beta testing, I guess :)

I've gone to some trouble to allow the BOA to *still* work
in this new POA supporting version of Fnorb, and it seems to
be fine in the Fnorb tests and examples ...

So, either the package I put together is flakey (I just
built it with "python setup.py sdist") or you've managed to
find a problem. Can me the code that makes it do this - that
is, enough of an example so I can run it and see what's
going on? That'll be the fastest way to find the problem.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Danilo Turina - 2005-02-21

Logged In: YES
user_id=232289

I produced a simplified example that exhibits the latest
problem I encountered.

--- simple.idl BEGIN ---
module MyModule {

typedef long long Ident;

struct ArgsIn {
Ident id;
Ident childId;
};

interface Simple {
void doSomething(in any arg);
};
};
--- simple.idl END ---

--- tryout.py BEGIN ---
#!/usr/bin/env python

import sys, os, re, string

# fnorb modules
from Fnorb.orb import BOA, CORBA

import MyModule
import MyModule_skel

class SimpleMgr(MyModule_skel.Simple_skel):

def __init__(self):
print "SimpleMgr initialization"
print ''

orb = CORBA.ORB_init(sys.argv, CORBA.ORB_ID)
boa = BOA.BOA_init(sys.argv, BOA.BOA_ID)

obj1 = boa.create('SimpleMgr', SimpleMgr._FNORB_ID)
simpleMgr = SimpleMgr()
boa.obj_is_ready(obj1, simpleMgr)
stringified_ior_neMgr = orb.object_to_string(simpleMgr)
--- tryout.py END ---

Am I doing something wrong?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Derek Thomson - 2005-02-26

assigned_to: nobody --> dthomson
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Derek Thomson - 2005-02-26

Logged In: YES
user_id=435731

Thanks, I'll try this out as soon as I can (this weekend,
once I get my dev. PC working again). My goal is to allow
the BOA to still work in the new POA release ... and the old
BOA tests and examples still in fact work, so I *thought*
I'd gotten it working alright. So I'm not sure what the
problem is here. I've done something wrong no doubt. Let me
get back to you.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Derek Thomson - 2005-02-26

Logged In: YES
user_id=435731

Ah, here's your problem:

> stringified_ior_neMgr = orb.object_to_string(simpleMgr)

The issue is this - when there's only one object adapter,
you can use the servant in place of a real CORBA object
reference like this - because it's easy to get the right
object adapter.

This doesn't work when there's more than one object adapter,
so you're now effectively passing in an object of the wrong
"type".

If you do this instead:

> stringified_ior_neMgr = orb.object_to_string(obj1)

It works fine. I just tried it. So if you change your code
to only use real CORBA object references to operations like
object_to_string, it'll be fine.

On the other hand, I wonder how hard it would be to have the
ORB support the old "magic" behaviour if you're not using
the POA. I'll have to think about it, as this will break
anyone who relies on the old behavior and I'd rather not
hurt anyone who is still using the BOA if I can. I'll muck
around with it. In the meantime, you should *finally* be
able see if your typecode woes are addressed!

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Nobody/Anonymous - 2005-03-04

Logged In: NO

No success!!!!

With the modifications you suggested the servers starts,
but, anyway, I always have the "indirection" problem:

Traceback (most recent call last):
File "xxxxx.py", line xx, in ?
boa._fnorb_mainloop()
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/BOA.py",
line 278, in _fnorb_mainloop
protocol.start(timeout)
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/IIOPProtocol.py",
line 83, in start
self.__reactor.start_event_loop(timeout)
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/SelectReactor.py",
line 143, in start_event_loop
self.do_one_event(timeout)
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/SelectReactor.py",
line 184, in do_one_event
handler.handle_event(Reactor.READ)
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/GIOPServerWorkerReactive.py",
line 155, in handle_event
self.__handler.recv()
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/GIOPConnectionHandler.py",
line 191, in recv
self.__listener.message_received(
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/GIOPServerWorkerReactive.py",
line 198, in message_received
self.__giop_server.process_giop_message(message)
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/GIOPServer.py",
line 166, in process_giop_message
self.__oa._fnorb_request((self, request_header, cursor))
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/BOA.py",
line 365, in _fnorb_request
self.__process_request(request)
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/BOA.py",
line 489, in __process_request
apply(method, (server_request,))
File
"/opt/testenvironment/VLAN711/simu/adapt_simul/lib/AdaptationEmlNeMgr_skel/__init__.py",
line 383, in _skel_uploadTerminations
arguments = server_request.arguments()
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/DSI.py",
line 92, in arguments
arguments.append(tc._fnorb_unmarshal_value(self.__cursor))
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/TypeCode.py",
line 964, in _fnorb_unmarshal_value
result._fnorb_unmarshal(cursor)
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/Any.py",
line 85, in _fnorb_unmarshal
self.__typecode =
CORBA.TypeCodeFactory_init().unmarshal(cursor)
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/TypeCode.py",
line 3931, in unmarshal
typecode._fnorb_unmarshal(cursor)
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/TypeCode.py",
line 1766, in _fnorb_unmarshal
member_type = tcf.unmarshal(encapsulation_cursor)
File
"/usr/local/lib/python2.2/site-packages/Fnorb/orb/TypeCode.py",
line 3910, in unmarshal
raise 'Invalid indirection for recursive typecode.'
Invalid indirection for recursive typecode.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Derek Thomson - 2005-03-12

Logged In: YES
user_id=435731

Okay, at least the new version of Fnorb works :) As an
aside, I think we will try to support the "magic"
object-as-object-reference behaviour for people still using
the BOA. Too many people would have assumed that this will
just work. I'll have to think about this - shouldn't be too
hard.

Anyway, your problem: so we've established that it doesn't
work even with the most recent fixes. In that case, I'll
have to try your example code and try to reproduce it here!
Probably tomorrow - I'll get back to you when done.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Derek Thomson - 2005-03-13

Logged In: YES
user_id=435731

Hi Danilo,

I just quickly tried passing this struct as an any from
Fnorb client to Fnorb server, and that worked fine.

Before we go any further, I notice there was some confusion
early on about exactly what form this struct took, which
indicates it *might* just be an IDL mismatch due to a
change. Please, just to be sure, go and recompile your IDL
for *both* TAO and Fnorb, recompile all your C++, and see if
that fixes things. Do this even if you're certain you did it
already - it's *so* easy to forget to recompile the IDL on
one side, and you start getting all kinds of inexplicable
hassle. I make this mistake often, if I have to do it
manually - so before we spend more time on this please just
try that first.

The next step would be to try this with a TAO client ... and
that's getting painful. Basically, I don't use C++ at all
any more and I despise the C++/CORBA mapping. It fills me
with dread just to think about it. In short, it will be time
consuming and frustrating to have to write a TAO/C++ client
from scratch just to reproduce this myself.

Would it be possible for you to come up with a short C++
client (and a corresponding Fnorb Python server) that I can
build here against TAO to reproduce the problem?

It still won't be easy for me even *with* that code, as I'm
still not using my "real" machine (let's just say
"warranties == hassle"), and building/using TAO on this
older machine is not the most pleasurable experience
imaginable as I recall. But I will build TAO here (it *will*
take hours) so that I'll be ready to go when you lay your
example code on me :)

Regards,
Derek.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Derek Thomson - 2005-03-13

Logged In: YES
user_id=435731

Hi Danilo,

I can't compile either TAO 1.3 or 1.4 on Red Hat Fedora Core
3. It gives an obscure compiler error then dies during the
build. A quick search doesn't reveal any quick fix for this,
apart from it being related somehow to the version of the
C++ compiler shipped with Core 3 - and I'm not really keen
to muck around with building different versions of the C++
compiler just to suit TAO. Especially considering that it
will probably mean also building special versions of any
tools and libraries that the compiler uses. Sigh.

I'll keep searching for an quick answer that doesn't involve
patching compilers :) In the meantime, tell us how
recompiling the IDL on both sides goes ...

I'd honestly forgotten how painful it is to make C++ code
work on a given platform. What a nuisance. I can't really
blame the TAO people - I know it's damned hard to write
non-trivial C++ code that works across all the popular
compilers (at least, last time I used C++ circa 2001).

I'll just hold out the vague hope that it's simply
mismatched IDL for now, before I start spending *lots* of
time getting TAO working.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Derek Thomson - 2005-03-19

Logged In: YES
user_id=435731

Danilo,

The good news is I've managed to get the "beta" release of
TAO building here on FC 3. What I'll try to do is muck about
with adding your struct to the TAO interoperability tests
I've already written in C++, and see if I can't reproduce
the problem. I'll do that this afternoon (19/3), so stay
tuned ...

dt.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Derek Thomson - 2005-05-02

Logged In: YES
user_id=435731

I had to give up building TAO on my older machine, it was
painful doing *anything* TAO related on it ... but my dev
box is all patched up and I'm building TAO on it now.

I'm going to look at this problem this afternoon. My first
step will be to try to reproduce it. Sorry about the delay.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Derek Thomson - 2005-05-02

Logged In: YES
user_id=435731

Just looking back through this problem's history, I can see
that I asked you to make 100% certain you had regenerated
the stubs and skeletons from the *same* IDL for both Fnorb
and TAO. This simple oversight (that I make every now and
again, to be honest) can often be the source of such problems.

Since I've not heard anything since, can I dare hope that
you did this and the problem just went away?

I also asked for a small example that demonstrated the
problem, and I've heard nothing back ... as I said before
I'm not really keen on writing reams of C++/CORBA code due
to 1) the painful nature of the C++ mapping 2) I don't
support a *Python* ORB because I want to write lots of C++
on the weekend ;) I'll have a crack at it today but if I add
the struct you described to the TAO test code that I have
right now in the Fnorb repository, and it works, I'll
probably just log that fact here and close this as "not a bug".

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Derek Thomson - 2005-05-02

Logged In: YES
user_id=435731

Danilo,

I've got good news and bad news (I guess). The good news is
that I was able to write a little C++ TAO client and Python
Fnorb server to try your example (despite leaving
Henning+Vinoski at work!). The bad news is that it worked
fine. I packed the struct into an any on the C++ side, sent
it to the Fnorb server, extracted the struct from the any
and the values of the struct are exactly as expected.

Just to make sure, here's the IDL I'm using to do this:

typedef long long Ident;

struct ArgsIn {
Ident id;
boolean flag;
Ident childId;
};

interface TestAny
{
void do_something(in any anAny);
};

Now, my C++ client only calls this operation. It is
perfectly possible that some operation call before that is
messing up the byte alignment (GIOP is annoying that way),
so the next question is: in your code, do you call some
other operation before "do_something"?

(This is why having a small example the reproduces the
problem is so useful, I don't have to ask all these
questions but I can just look at the example instead!)

Anyway, please at least respond if you're still interested
in this problem, even if you aren't able to think about
this, just so I know if you're still alive :) If I don't
hear anything I'll close it as "not a bug" in a couple of
days. I think I've done all I can from this end.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Danilo Turina - 2005-05-02

Logged In: YES
user_id=232289

Thank you very much Derek for your time.
I can ensure that the IDL is the same for the client and the
server.

Anyway, if your test succeeded, now is up to me to produce
the smallest example that exhibit the problem.

Honestly, I haven't replied to you until now because I'm a
bit overloaded at work (and also because, for now, we have a
little abandoned Fnorb-based simulators).

For this reason I hope I'm able to produce such an example
for the end of the week.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Derek Thomson - 2005-05-03

Logged In: YES
user_id=435731

> Thank you very much Derek for your time.

No problem. I've been struggling to get time to research
this for a while now, sorry about the delay.

> I can ensure that the IDL is the same for the client and the
> server.

Okay, good! At least we can eliminate that.

> Anyway, if your test succeeded, now is up to me to produce
> the smallest example that exhibit the problem.

That would be useful!

> Honestly, I haven't replied to you until now because I'm a
> bit overloaded at work

I know the feeling :)

> For this reason I hope I'm able to produce such an example
> for the end of the week.

Okay, great. I'll leave the bug open then.

I have noticed since my last comment that when I put some
trace statements into Fnorb the typecode contained in the
"any" sent by TAO doesn't contain an indirection! I'm using
the most recent version of TAO (it's the only one that will
build for me on Fedora Core 3) so maybe I'm just not
provoking the problem at all! I will look at the options on
TAO and see if I can force it to behave the same way with
regards to indirections in the typecode of an "any". Then we
have a closer match between your example and mine.

Since you've probably used TAO more than me, do you happen
to know if there is some flexibility here and how I might
adjust it? I might also ask around (comp.object.corba etc)
about this issue ...

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

TAO-Fnorb interoperability on indirection within typecode

Group

Searches

Help

#27 TAO-Fnorb interoperability on indirection within typecode

Discussion