tipc-discussion — User questions & problem reports, project news, developer discussion

Re: [tipc-discussion] Re: queue_limit question

[Mark H. <ma...@os...>]

On Wed, 2004-06-23 at 10:18, Jon Maloy wrote:
> These messages should not be there at all. 
> 
> If you are on node <1.1.11>, and the messages are sent from <1.1.12>,
> they 
> should not be rerouted out over some other link from 1.1.11. Either
> the name 
> lookup finds node local destination ports
> and delivers the message to those, or the buffer is discarded. There
> is no sign that those
> are rejected messages, since there is no error code, and anyway
> rejected multicast
> messages is something that must not exist.

I am on node 12 and the message is being sent by 12.  The destination
was 11.


> 
> If you are _not_ on node 1.1.11 the scenario is much worse: The lookup
> may have
> found a bunch of non-local destinations, created a copy of the message
> for each of them, 
> and is trying to send them to the new destination nodes. If this
> happens again and again
> it will lead to a virtual explosion of messages, -a chain reaction
> that will eventually
> blow up the whole cluster.  An interesting scenario...
> Make sure that _only_  node local destinations are looked up in the
> secondary
> lookup on the destination node.
> 
> The importance is 4 here because the message is perceived as "routed",
> i.e. the
> originating node is different from the own node. In that case the
> queue must be more
> resilient, since it can not try to suspend the sender process (which
> is an SW interrupt).
> I therefore add 4 to the importance of all such messages.
> 
> /Jon
> Mark Haverkamp wrote:
> > On Wed, 2004-06-23 at 07:41, Jon Maloy wrote:
> >   
> > > /jon
> > > 
> > > Mark Haverkamp wrote:
> > >     
> > > > On Wed, 2004-06-23 at 06:32, Jon Maloy wrote:
> > > >   
> > > >       
> > > > > I thought a little more about this. In my own code (in link.c) I
> > > > > always
> > > > > ensure that the send conditions (for the original message type, not
> > > > > for
> > > > > FIRST_FRAGMENT) are fulfilled _before_  I call link_send_long_buf().
> > > > > This way, we never throw away FIRST_FRAGMENTS at all, the
> > > > > congestion is caught earlier, and the sending port/process is
> > > > > rescheduled
> > > > > to try again when the congestion abates.
> > > > >     
> > > > >         
> > > > I see the check for the original message before calling
> > > > link_send_long_buf but something must be happening between the time of
> > > > checking and sending out after breaking them up. Although I'm not sure
> > > > what that would be.  Looking again, shouldn't normal a message's
> > > > importance be less than TIPC_NON_REJECTABLE and  call link_schedule_port
> > > > instead of tossing them?
> > > >       
> > > Exactly. This was why I wondered which message type we are dealing
> > > with
> > > here. If (user <= TIPC_NON_REJECTABLE) the call should never enter
> > > the link_send_long_buf() call during congeston. If user is
> > > BCAST_PROTOCOL
> > > it will pass the congestin test. If that is the case the test must be
> > > changed.
> > >     
> > I put link_send_buf back the way it was (returning dsz when throwing
> > away packets).  And added some debug output.
> > 
> > The lslb(0) lines are printed from link_send_long_buf when
> > link_schedule_port is called from link_send_buf and it returns
> > TIPC_CONGESTION.
> > 
> > the lsb(0) and following lines are in link_send_buf when throwing away a
> > message.  The type is one so it must not be a first fragment.  Why is
> > the tot_importance 4 when the importance is zero?  The originating node
> > is 1001012 which is where we're sending from.  This is why it is being
> > thrown away instead of calling link_schedule_port.
> > 
> > 
> > lslb(0)
> > lslb(0)
> > lslb(0)
> > lslb(0)
> > lslb(0)
> > lslb(0)
> > lslb(0)
> > lslb(0)
> > lslb(0)
> > lslb(0)
> > lslb(0)
> > lslb(0)
> > lslb(0)
> > lslb(0)
> > lslb(0)
> > lslb(0)
> > lslb(0)
> > 
> > lsb(0) type 1 imp 0 tot_imp 4 mc 1
> > TIPC: Congestion, throwing away
> > DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(1001012:1914814476)::DEST(1001011:0):
> > lsb(0) type 1 imp 0 tot_imp 4 mc 1
> > TIPC: Congestion, throwing away
> > DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(1001012:1914814476)::DEST(1001011:0):
> > lsb(0) type 1 imp 0 tot_imp 4 mc 1
> > TIPC: Congestion, throwing away
> > DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(1001012:1914814476)::DEST(1001011:0):
> > lsb(0) type 1 imp 0 tot_imp 4 mc 1
> > TIPC: Congestion, throwing away
> > DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(1001012:1914814476)::DEST(1001011:0):
> > lsb(0) type 1 imp 0 tot_imp 4 mc 1
> > TIPC: Congestion, throwing away
> > DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(1001012:1914814476)::DEST(1001011:0):
> > lsb(0) type 1 imp 0 tot_imp 4 mc 1
> > TIPC: Congestion, throwing away
> > DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(1001012:1914814476)::DEST(1001011:0):
> > lslb(0)
> > lsb(0) type 1 imp 0 tot_imp 4 mc 1
> > TIPC: Congestion, throwing away
> > DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(1001012:1914814476)::DEST(1001011:0):
> > lsb(0) type 1 imp 0 tot_imp 4 mc 1
> > TIPC: Congestion, throwing away
> > DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(1001012:1914814476)::DEST(1001011:0):
> > lsb(0) type 1 imp 0 tot_imp 4 mc 1
> > TIPC: Congestion, throwing away
> > DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(1001012:1914814476)::DEST(1001011:0):
> > lsb(0) type 1 imp 0 tot_imp 4 mc 1
> > TIPC: Congestion, throwing away
> > DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(1001012:1914814476)::DEST(1001011:0):
> > 
> > 
> >   
-- 
Mark Haverkamp <ma...@os...>

Re: [tipc-discussion] Re: queue_limit question

[Jon M. <jon...@er...>]

These messages should not be there at all. 

If you are on node <1.1.11>, and the messages are sent from <1.1.12>,
they 
should not be rerouted out over some other link from 1.1.11. Either the
name 
lookup finds node local destination ports
and delivers the message to those, or the buffer is discarded. There is
no sign that those
are rejected messages, since there is no error code, and anyway rejected
multicast
messages is something that must not exist.

If you are _not_ on node 1.1.11 the scenario is much worse: The lookup
may have
found a bunch of non-local destinations, created a copy of the message
for each of them, 
and is trying to send them to the new destination nodes. If this happens
again and again
it will lead to a virtual explosion of messages, -a chain reaction that
will eventually
blow up the whole cluster.  An interesting scenario...
Make sure that _only_  node local destinations are looked up in the
secondary
lookup on the destination node.

The importance is 4 here because the message is perceived as "routed",
i.e. the
originating node is different from the own node. In that case the queue
must be more
resilient, since it can not try to suspend the sender process (which is
an SW interrupt).
I therefore add 4 to the importance of all such messages.

/Jon
Mark Haverkamp wrote:


On Wed, 2004-06-23 at 07:41, Jon Maloy wrote:

  

/jon



Mark Haverkamp wrote:

    

On Wed, 2004-06-23 at 06:32, Jon Maloy wrote:

  

      

I thought a little more about this. In my own code (in link.c) I

always

ensure that the send conditions (for the original message type, not

for

FIRST_FRAGMENT) are fulfilled _before_  I call link_send_long_buf().

This way, we never throw away FIRST_FRAGMENTS at all, the

congestion is caught earlier, and the sending port/process is

rescheduled

to try again when the congestion abates.

    

        

I see the check for the original message before calling

link_send_long_buf but something must be happening between the time of

checking and sending out after breaking them up. Although I'm not sure

what that would be.  Looking again, shouldn't normal a message's

importance be less than TIPC_NON_REJECTABLE and  call link_schedule_port

instead of tossing them?

      

Exactly. This was why I wondered which message type we are dealing

with

here. If (user <= TIPC_NON_REJECTABLE) the call should never enter

the link_send_long_buf() call during congeston. If user is

BCAST_PROTOCOL

it will pass the congestin test. If that is the case the test must be

changed.

    



I put link_send_buf back the way it was (returning dsz when throwing

away packets).  And added some debug output.



The lslb(0) lines are printed from link_send_long_buf when

link_schedule_port is called from link_send_buf and it returns

TIPC_CONGESTION.



the lsb(0) and following lines are in link_send_buf when throwing away a

message.  The type is one so it must not be a first fragment.  Why is

the tot_importance 4 when the importance is zero?  The originating node

is 1001012 which is where we're sending from.  This is why it is being

thrown away instead of calling link_schedule_port.





lslb(0)

lslb(0)

lslb(0)

lslb(0)

lslb(0)

lslb(0)

lslb(0)

lslb(0)

lslb(0)

lslb(0)

lslb(0)

lslb(0)

lslb(0)

lslb(0)

lslb(0)

lslb(0)

lslb(0)



lsb(0) type 1 imp 0 tot_imp 4 mc 1

TIPC: Congestion, throwing away

DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(100
1012:1914814476)::DEST(1001011:0):

lsb(0) type 1 imp 0 tot_imp 4 mc 1

TIPC: Congestion, throwing away

DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(100
1012:1914814476)::DEST(1001011:0):

lsb(0) type 1 imp 0 tot_imp 4 mc 1

TIPC: Congestion, throwing away

DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(100
1012:1914814476)::DEST(1001011:0):

lsb(0) type 1 imp 0 tot_imp 4 mc 1

TIPC: Congestion, throwing away

DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(100
1012:1914814476)::DEST(1001011:0):

lsb(0) type 1 imp 0 tot_imp 4 mc 1

TIPC: Congestion, throwing away

DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(100
1012:1914814476)::DEST(1001011:0):

lsb(0) type 1 imp 0 tot_imp 4 mc 1

TIPC: Congestion, throwing away

DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(100
1012:1914814476)::DEST(1001011:0):

lslb(0)

lsb(0) type 1 imp 0 tot_imp 4 mc 1

TIPC: Congestion, throwing away

DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(100
1012:1914814476)::DEST(1001011:0):

lsb(0) type 1 imp 0 tot_imp 4 mc 1

TIPC: Congestion, throwing away

DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(100
1012:1914814476)::DEST(1001011:0):

lsb(0) type 1 imp 0 tot_imp 4 mc 1

TIPC: Congestion, throwing away

DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(100
1012:1914814476)::DEST(1001011:0):

lsb(0) type 1 imp 0 tot_imp 4 mc 1

TIPC: Congestion, throwing away

DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(100
1012:1914814476)::DEST(1001011:0):





  

[tipc-discussion] bcast_recv/recv_bcast_data lockup

[Mark H. <ma...@os...>]

Here is another one.

In trying to use bcast (tipc_bcast_threshold = 0), I am seeing a spin
lock deadlock. 

bcast_recv gets a node lock and calls recv_bcast_data.

bcast_recv_data then can also try to get a node lock.  If those two
nodes are the same, then we'll have a deadlock.

Mark. 
-- 
Mark Haverkamp <ma...@os...>

Re: [tipc-discussion] Re: queue_limit question

[Mark H. <ma...@os...>]

On Wed, 2004-06-23 at 07:41, Jon Maloy wrote:
> /jon
> 
> Mark Haverkamp wrote:
> > On Wed, 2004-06-23 at 06:32, Jon Maloy wrote:
> >   
> > > I thought a little more about this. In my own code (in link.c) I
> > > always
> > > ensure that the send conditions (for the original message type, not
> > > for
> > > FIRST_FRAGMENT) are fulfilled _before_  I call link_send_long_buf().
> > > This way, we never throw away FIRST_FRAGMENTS at all, the
> > > congestion is caught earlier, and the sending port/process is
> > > rescheduled
> > > to try again when the congestion abates.
> > >     
> > I see the check for the original message before calling
> > link_send_long_buf but something must be happening between the time of
> > checking and sending out after breaking them up. Although I'm not sure
> > what that would be.  Looking again, shouldn't normal a message's
> > importance be less than TIPC_NON_REJECTABLE and  call link_schedule_port
> > instead of tossing them?
> Exactly. This was why I wondered which message type we are dealing
> with
> here. If (user <= TIPC_NON_REJECTABLE) the call should never enter
> the link_send_long_buf() call during congeston. If user is
> BCAST_PROTOCOL
> it will pass the congestin test. If that is the case the test must be
> changed.

I put link_send_buf back the way it was (returning dsz when throwing
away packets).  And added some debug output.

The lslb(0) lines are printed from link_send_long_buf when
link_schedule_port is called from link_send_buf and it returns
TIPC_CONGESTION.

the lsb(0) and following lines are in link_send_buf when throwing away a
message.  The type is one so it must not be a first fragment.  Why is
the tot_importance 4 when the importance is zero?  The originating node
is 1001012 which is where we're sending from.  This is why it is being
thrown away instead of calling link_schedule_port.


lslb(0)
lslb(0)
lslb(0)
lslb(0)
lslb(0)
lslb(0)
lslb(0)
lslb(0)
lslb(0)
lslb(0)
lslb(0)
lslb(0)
lslb(0)
lslb(0)
lslb(0)
lslb(0)
lslb(0)

lsb(0) type 1 imp 0 tot_imp 4 mc 1
TIPC: Congestion, throwing away
DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(1001012:1914814476)::DEST(1001011:0):
lsb(0) type 1 imp 0 tot_imp 4 mc 1
TIPC: Congestion, throwing away
DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(1001012:1914814476)::DEST(1001011:0):
lsb(0) type 1 imp 0 tot_imp 4 mc 1
TIPC: Congestion, throwing away
DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(1001012:1914814476)::DEST(1001011:0):
lsb(0) type 1 imp 0 tot_imp 4 mc 1
TIPC: Congestion, throwing away
DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(1001012:1914814476)::DEST(1001011:0):
lsb(0) type 1 imp 0 tot_imp 4 mc 1
TIPC: Congestion, throwing away
DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(1001012:1914814476)::DEST(1001011:0):
lsb(0) type 1 imp 0 tot_imp 4 mc 1
TIPC: Congestion, throwing away
DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(1001012:1914814476)::DEST(1001011:0):
lslb(0)
lsb(0) type 1 imp 0 tot_imp 4 mc 1
TIPC: Congestion, throwing away
DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(1001012:1914814476)::DEST(1001011:0):
lsb(0) type 1 imp 0 tot_imp 4 mc 1
TIPC: Congestion, throwing away
DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(1001012:1914814476)::DEST(1001011:0):
lsb(0) type 1 imp 0 tot_imp 4 mc 1
TIPC: Congestion, throwing away
DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(1001012:1914814476)::DEST(1001011:0):
lsb(0) type 1 imp 0 tot_imp 4 mc 1
TIPC: Congestion, throwing away
DAT0:MCST:HZ(44):SZ(22634):SQNO(0):ACK(0):BACK(0):PRND(1001012):ORIG(1001012:1914814476)::DEST(1001011:0):


-- 
Mark Haverkamp <ma...@os...>

Re: [tipc-discussion] Can now send messages in single node mode

[Jon M. <jon...@er...>]

Not really good either: The lookup may fail because 'destnode' is a
fully defined address to some other node.
 (I.e. in_scope(dest=1.1.2,  own=1.1.1) will fail and
nametbl_translate()
return destport=0, destnode=dest, 
while in_scope(dest=1.1.0,  own=1.1.1) will be successful
and return what it finds in the name table.

Even the first result is valid, though, it just means that the
message must be forwarded to node 1.1.2 for a new lookup.

=> the correct test is for if(destport || destnode).

I fixed it and checked in.

/jon

all...@wi... <mailto:all...@wi...>
wrote:


Hi all:
 
This morning I discovered that it isn't possible to send messages when a
node is operating in single node mode.  The problem is that
tipc_forward2name() and tipc_forward_buf2name() reject messages if the
destnode returned by nametbl_translate is 0 (which it always is in
single node mode), instead of rejecting messages if the destport
returned by nametbl_translate is 0 (indicating that the destination
doesn't exist).
 
Since the fix was trivial (testing for a non-zero "destport", instead of
a non-zero "destnode") I've already coded it up and submitted it to the
CVS repository.  Give me a shout if there are any problems.
 
Regards,
Al Stephens
all...@wi... <mailto:all...@wi...> 

Re: [tipc-discussion] Re: queue_limit question

[Jon M. <jon...@er...>]

/jon

Mark Haverkamp wrote:


On Wed, 2004-06-23 at 06:32, Jon Maloy wrote:

  

I thought a little more about this. In my own code (in link.c) I

always

ensure that the send conditions (for the original message type, not

for

FIRST_FRAGMENT) are fulfilled _before_  I call link_send_long_buf().

This way, we never throw away FIRST_FRAGMENTS at all, the

congestion is caught earlier, and the sending port/process is

rescheduled

to try again when the congestion abates.

    



I see the check for the original message before calling

link_send_long_buf but something must be happening between the time of

checking and sending out after breaking them up. Although I'm not sure

what that would be.  Looking again, shouldn't normal a message's

importance be less than TIPC_NON_REJECTABLE and  call link_schedule_port

instead of tossing them?

Exactly. This was why I wondered which message type we are dealing with
here. If (user <= TIPC_NON_REJECTABLE) the call should never enter
the link_send_long_buf() call during congeston. If user is
BCAST_PROTOCOL
it will pass the congestin test. If that is the case the test must be
changed.






I tried returning TIPC_CONGESTION inside the queue_limit check and

things got better.  It ran for a while and then got a BUG check in the

spin lock code.  I'm taking a look at that right now.



  

What is the type of message that is fragmented here? Is it a

retransmitted

message delivered over a single link ? Or is it a user message that is

sent

over a link because number of destinations is below tipc_bcast_limit ?

    



They are multicast messages to three nodes of about 32k.  They are the

replicated multicast type and I have one link.



  

If it is a BCAST_PROTOCOL message, the congestion is not caught by 

the test at the beginning og bcast_link_send_buf(), since this one

only

tests for     if (imp <= TIPC_NON_REJECTABLE) . Is this possibly what

happens ?



/Jon





Jon Maloy wrote:

    

Your theory makes sense.  It is wrong not to return 

TIPC_CONGESTION if a FIRST_FRAGMENT has to be 

thrown away in link_send_msg().  



I suggest you try this,and then test for that return value

in link_send_long_buf(). If < 0, stop the fragmentation 

and return the value. 



I am still a little puzzled that this happens. The limit for

FIRST_FRAGMENT is set to 300, which sounds a lot to me.

Maybe we should make it a function of the link window, as we

do with some other values ?



/jon



Mark Haverkamp wrote:

      

On Tue, 2004-06-22 at 12:47, Jon Maloy wrote: 

  

        

Hi,

Last question first: As you have already guessed this is a bug.

Of course DATA_NORMAL should be mapped to 

TIPC_NORMAL_IMPORTANCE and DATA_HIGH correspondingly.



Position 5 is initialized twice because at some moment I redefined

CONN_MANAGER from 8 to 5, and missed this effect.  The value

8 was taken over by BCAST_PROTOCOL, but obviously the array

was not updated correspondingly.



When looking at it now it does not make completely sense: CONN_MANAGER

should retain the value 8, since such messages may be routed,

and must not be thrown away, while BCAST_PROTOCOL can 

have the lower value 5, since they will never be routed.



I will fix this, but I need to think through it a little more first,

there may

have been some other  reasoning behind it I have forgotten about.



/jon



    

          

OK, here is my theory with the lost first fragment.  It is related to

the queue_limit.  When the large message is broken up into fragments to

send, the chain of buffers is sent to link_send_buf.  There is a check

at the start to see if the message has exceeded its queue limit for its

type.  The first fragment will have the importance of the message itself

and if low priority may be tossed if the queue is fairly full. 

Subsequent messages sent from link_send_long_buf will be for the

MSG_FRAGMENTER and be subject to to different queue_limit.  These

messages may be sent where the first fragment may have been thrown

away.  I enabled some debug output in link_send_buf and see that there

are messages being thrown away.  It seems that if the first fragment

fails to be sent, link_send_long_buf should not send the following

messages although I don't see a way to know that has happened.



What do you think?



Mark.







  

        

Re: [tipc-discussion] Re: queue_limit question

[Mark H. <ma...@os...>]

On Wed, 2004-06-23 at 06:32, Jon Maloy wrote:
> I thought a little more about this. In my own code (in link.c) I
> always
> ensure that the send conditions (for the original message type, not
> for
> FIRST_FRAGMENT) are fulfilled _before_  I call link_send_long_buf().
> This way, we never throw away FIRST_FRAGMENTS at all, the
> congestion is caught earlier, and the sending port/process is
> rescheduled
> to try again when the congestion abates.

I see the check for the original message before calling
link_send_long_buf but something must be happening between the time of
checking and sending out after breaking them up. Although I'm not sure
what that would be.  Looking again, shouldn't normal a message's
importance be less than TIPC_NON_REJECTABLE and  call link_schedule_port
instead of tossing them?

I tried returning TIPC_CONGESTION inside the queue_limit check and
things got better.  It ran for a while and then got a BUG check in the
spin lock code.  I'm taking a look at that right now.

> 
> What is the type of message that is fragmented here? Is it a
> retransmitted
> message delivered over a single link ? Or is it a user message that is
> sent
> over a link because number of destinations is below tipc_bcast_limit ?

They are multicast messages to three nodes of about 32k.  They are the
replicated multicast type and I have one link.

> If it is a BCAST_PROTOCOL message, the congestion is not caught by 
> the test at the beginning og bcast_link_send_buf(), since this one
> only
> tests for     if (imp <= TIPC_NON_REJECTABLE) . Is this possibly what
> happens ?
> 
> /Jon
> 
> 
> Jon Maloy wrote:
> > Your theory makes sense.  It is wrong not to return 
> > TIPC_CONGESTION if a FIRST_FRAGMENT has to be 
> > thrown away in link_send_msg().  
> > 
> > I suggest you try this,and then test for that return value
> > in link_send_long_buf(). If < 0, stop the fragmentation 
> > and return the value. 
> > 
> > I am still a little puzzled that this happens. The limit for
> > FIRST_FRAGMENT is set to 300, which sounds a lot to me.
> > Maybe we should make it a function of the link window, as we
> > do with some other values ?
> > 
> > /jon
> > 
> > Mark Haverkamp wrote:
> > > On Tue, 2004-06-22 at 12:47, Jon Maloy wrote: 
> > >   
> > > > Hi,
> > > > Last question first: As you have already guessed this is a bug.
> > > > Of course DATA_NORMAL should be mapped to 
> > > > TIPC_NORMAL_IMPORTANCE and DATA_HIGH correspondingly.
> > > > 
> > > > Position 5 is initialized twice because at some moment I redefined
> > > > CONN_MANAGER from 8 to 5, and missed this effect.  The value
> > > > 8 was taken over by BCAST_PROTOCOL, but obviously the array
> > > > was not updated correspondingly.
> > > > 
> > > > When looking at it now it does not make completely sense: CONN_MANAGER
> > > > should retain the value 8, since such messages may be routed,
> > > > and must not be thrown away, while BCAST_PROTOCOL can 
> > > > have the lower value 5, since they will never be routed.
> > > > 
> > > > I will fix this, but I need to think through it a little more first,
> > > > there may
> > > > have been some other  reasoning behind it I have forgotten about.
> > > > 
> > > > /jon
> > > > 
> > > >     
> > > OK, here is my theory with the lost first fragment.  It is related to
> > > the queue_limit.  When the large message is broken up into fragments to
> > > send, the chain of buffers is sent to link_send_buf.  There is a check
> > > at the start to see if the message has exceeded its queue limit for its
> > > type.  The first fragment will have the importance of the message itself
> > > and if low priority may be tossed if the queue is fairly full. 
> > > Subsequent messages sent from link_send_long_buf will be for the
> > > MSG_FRAGMENTER and be subject to to different queue_limit.  These
> > > messages may be sent where the first fragment may have been thrown
> > > away.  I enabled some debug output in link_send_buf and see that there
> > > are messages being thrown away.  It seems that if the first fragment
> > > fails to be sent, link_send_long_buf should not send the following
> > > messages although I don't see a way to know that has happened.
> > > 
> > > What do you think?
> > > 
> > > Mark.
> > > 
> > > 
> > > 
> > >   
-- 
Mark Haverkamp <ma...@os...>

Re: [tipc-discussion] Re: queue_limit question

[Jon M. <jon...@er...>]

I thought a little more about this. In my own code (in link.c) I always
ensure that the send conditions (for the original message type, not for
FIRST_FRAGMENT) are fulfilled _before_  I call link_send_long_buf().
This way, we never throw away FIRST_FRAGMENTS at all, the
congestion is caught earlier, and the sending port/process is
rescheduled
to try again when the congestion abates.

What is the type of message that is fragmented here? Is it a
retransmitted
message delivered over a single link ? Or is it a user message that is
sent
over a link because number of destinations is below tipc_bcast_limit ?
If it is a BCAST_PROTOCOL message, the congestion is not caught by 
the test at the beginning og bcast_link_send_buf(), since this one only
tests for     if (imp <= TIPC_NON_REJECTABLE) . Is this possibly what
happens ?

/Jon


Jon Maloy wrote:


Your theory makes sense.  It is wrong not to return 
TIPC_CONGESTION if a FIRST_FRAGMENT has to be 
thrown away in link_send_msg().  

I suggest you try this,and then test for that return value
in link_send_long_buf(). If < 0, stop the fragmentation 
and return the value. 

I am still a little puzzled that this happens. The limit for
FIRST_FRAGMENT is set to 300, which sounds a lot to me.
Maybe we should make it a function of the link window, as we
do with some other values ?

/jon

Mark Haverkamp wrote:


On Tue, 2004-06-22 at 12:47, Jon Maloy wrote: 

  

Hi,

Last question first: As you have already guessed this is a bug.

Of course DATA_NORMAL should be mapped to 

TIPC_NORMAL_IMPORTANCE and DATA_HIGH correspondingly.



Position 5 is initialized twice because at some moment I redefined

CONN_MANAGER from 8 to 5, and missed this effect.  The value

8 was taken over by BCAST_PROTOCOL, but obviously the array

was not updated correspondingly.



When looking at it now it does not make completely sense: CONN_MANAGER

should retain the value 8, since such messages may be routed,

and must not be thrown away, while BCAST_PROTOCOL can 

have the lower value 5, since they will never be routed.



I will fix this, but I need to think through it a little more first,

there may

have been some other  reasoning behind it I have forgotten about.



/jon



    



OK, here is my theory with the lost first fragment.  It is related to

the queue_limit.  When the large message is broken up into fragments to

send, the chain of buffers is sent to link_send_buf.  There is a check

at the start to see if the message has exceeded its queue limit for its

type.  The first fragment will have the importance of the message itself

and if low priority may be tossed if the queue is fairly full. 

Subsequent messages sent from link_send_long_buf will be for the

MSG_FRAGMENTER and be subject to to different queue_limit.  These

messages may be sent where the first fragment may have been thrown

away.  I enabled some debug output in link_send_buf and see that there

are messages being thrown away.  It seems that if the first fragment

fails to be sent, link_send_long_buf should not send the following

messages although I don't see a way to know that has happened.



What do you think?



Mark.







  

[tipc-discussion] BCAST_PROTOCOL vs CONN_MANAGER

[Jon M. <jon...@er...>]

I swapped the numbers for these two, so the initialization
of queue_limits is consistent with these values. I also
corrected the DATA_HIGH/DATA_NORMAL mismatch.

Tested and checked in:

RCS file: /cvsroot/tipc/source/unstable/net/tipc/msg.h,v
retrieving revision 1.16
diff -r1.16 msg.h
109,110c109,110
< #define DATA_HIGH                 TIPC_NORMAL_IMPORTANCE
< #define DATA_NORMAL               TIPC_HIGH_IMPORTANCE
---
> #define DATA_NORMAL               TIPC_NORMAL_IMPORTANCE
> #define DATA_HIGH                 TIPC_HIGH_IMPORTANCE
384c384
< #define  CONN_MANAGER         5
---
> #define  BCAST_PROTOCOL       5
387c387
< #define  BCAST_PROTOCOL       8
---
> #define  CONN_MANAGER         8


/jon

[tipc-discussion] SOCK_STREAM fixed and checked in

[Jon M. <jon...@er...>]

I rewrote the SOCK_STREAM send algorithm (send_stream() in
socket.c), and now its seems to work fine to send > 66000 octets.

One bug report less!

/Jon

[tipc-discussion] Re: queue_limit question

[Jon M. <jon...@er...>]

Your theory makes sense.  It is wrong not to return 
TIPC_CONGESTION if a FIRST_FRAGMENT has to be 
thrown away in link_send_msg().  

I suggest you try this,and then test for that return value
in link_send_long_buf(). If < 0, stop the fragmentation 
and return the value. 

I am still a little puzzled that this happens. The limit for
FIRST_FRAGMENT is set to 300, which sounds a lot to me.
Maybe we should make it a function of the link window, as we
do with some other values ?

/jon

Mark Haverkamp wrote:


On Tue, 2004-06-22 at 12:47, Jon Maloy wrote: 

  

Hi,

Last question first: As you have already guessed this is a bug.

Of course DATA_NORMAL should be mapped to 

TIPC_NORMAL_IMPORTANCE and DATA_HIGH correspondingly.



Position 5 is initialized twice because at some moment I redefined

CONN_MANAGER from 8 to 5, and missed this effect.  The value

8 was taken over by BCAST_PROTOCOL, but obviously the array

was not updated correspondingly.



When looking at it now it does not make completely sense: CONN_MANAGER

should retain the value 8, since such messages may be routed,

and must not be thrown away, while BCAST_PROTOCOL can 

have the lower value 5, since they will never be routed.



I will fix this, but I need to think through it a little more first,

there may

have been some other  reasoning behind it I have forgotten about.



/jon



    



OK, here is my theory with the lost first fragment.  It is related to

the queue_limit.  When the large message is broken up into fragments to

send, the chain of buffers is sent to link_send_buf.  There is a check

at the start to see if the message has exceeded its queue limit for its

type.  The first fragment will have the importance of the message itself

and if low priority may be tossed if the queue is fairly full. 

Subsequent messages sent from link_send_long_buf will be for the

MSG_FRAGMENTER and be subject to to different queue_limit.  These

messages may be sent where the first fragment may have been thrown

away.  I enabled some debug output in link_send_buf and see that there

are messages being thrown away.  It seems that if the first fragment

fails to be sent, link_send_long_buf should not send the following

messages although I don't see a way to know that has happened.



What do you think?



Mark.







  

[tipc-discussion] Re: queue_limit question

[Mark H. <ma...@os...>]

On Tue, 2004-06-22 at 12:47, Jon Maloy wrote: 
> Hi,
> Last question first: As you have already guessed this is a bug.
> Of course DATA_NORMAL should be mapped to 
> TIPC_NORMAL_IMPORTANCE and DATA_HIGH correspondingly.
> 
> Position 5 is initialized twice because at some moment I redefined
> CONN_MANAGER from 8 to 5, and missed this effect.  The value
> 8 was taken over by BCAST_PROTOCOL, but obviously the array
> was not updated correspondingly.
> 
> When looking at it now it does not make completely sense: CONN_MANAGER
> should retain the value 8, since such messages may be routed,
> and must not be thrown away, while BCAST_PROTOCOL can 
> have the lower value 5, since they will never be routed.
> 
> I will fix this, but I need to think through it a little more first,
> there may
> have been some other  reasoning behind it I have forgotten about.
> 
> /jon
> 

OK, here is my theory with the lost first fragment.  It is related to
the queue_limit.  When the large message is broken up into fragments to
send, the chain of buffers is sent to link_send_buf.  There is a check
at the start to see if the message has exceeded its queue limit for its
type.  The first fragment will have the importance of the message itself
and if low priority may be tossed if the queue is fairly full. 
Subsequent messages sent from link_send_long_buf will be for the
MSG_FRAGMENTER and be subject to to different queue_limit.  These
messages may be sent where the first fragment may have been thrown
away.  I enabled some debug output in link_send_buf and see that there
are messages being thrown away.  It seems that if the first fragment
fails to be sent, link_send_long_buf should not send the following
messages although I don't see a way to know that has happened.

What do you think?

Mark.



-- 

Mark Haverkamp <ma...@os...>

[tipc-discussion] Re: queue_limit question

[Jon M. <jon...@er...>]

Hi,
Last question first: As you have already guessed this is a bug.
Of course DATA_NORMAL should be mapped to 
TIPC_NORMAL_IMPORTANCE and DATA_HIGH correspondingly.

Position 5 is initialized twice because at some moment I redefined
CONN_MANAGER from 8 to 5, and missed this effect.  The value
8 was taken over by BCAST_PROTOCOL, but obviously the array
was not updated correspondingly.

When looking at it now it does not make completely sense: CONN_MANAGER
should retain the value 8, since such messages may be routed,
and must not be thrown away, while BCAST_PROTOCOL can 
have the lower value 5, since they will never be routed.

I will fix this, but I need to think through it a little more first,
there may
have been some other  reasoning behind it I have forgotten about.

/jon




Mark Haverkamp wrote:


I was looking at the queue_limit array and noticed that location 5 is

getting initialized twice and location 8 isn't getting initialized at

all.  Is there something wrong there?



Also out of curiosity, why is DATA_HIGH set to TIPC_NORMAL_IMPORTANCE

and DATA_NORMAL set to TIPC_HIGH_IMPORTANCE?



Mark.



(0)	this->queue_limit[DATA_LOW] = window;

(2)	this->queue_limit[DATA_NORMAL] = (window / 3) * 4;

(1)	this->queue_limit[DATA_HIGH] = (window / 3) * 5;

(3)	this->queue_limit[DATA_NON_REJECTABLE] = (window / 3) * 6;

	/* Transiting data messages,inclusive FIRST_FRAGM */

(4)	this->queue_limit[DATA_LOW + 4] = 300;

(6)	this->queue_limit[DATA_NORMAL + 4] = 600;

(5)	this->queue_limit[DATA_HIGH + 4] = 900;

(7)	this->queue_limit[DATA_NON_REJECTABLE + 4] = 1200;

(5)	this->queue_limit[CONN_MANAGER] = 1200;

(9)	this->queue_limit[ROUTE_DISTRIBUTOR] = 1200;

(10)	this->queue_limit[CHANGEOVER_PROTOCOL] = 2500;

(11)	this->queue_limit[NAME_DISTRIBUTOR] = 3000;

	/* FRAGMENT and LAST_FRAGMENT packets */

(12)	this->queue_limit[MSG_FRAGMENTER] = 4000;



#define DATA_LOW                  TIPC_LOW_IMPORTANCE

#define DATA_HIGH                 TIPC_NORMAL_IMPORTANCE

#define DATA_NORMAL               TIPC_HIGH_IMPORTANCE

#define DATA_NON_REJECTABLE       TIPC_NON_REJECTABLE



#define TIPC_LOW_IMPORTANCE     0  /* default */

#define TIPC_NORMAL_IMPORTANCE	1

#define TIPC_HIGH_IMPORTANCE	2

#define TIPC_NON_REJECTABLE	3



#define  CONN_MANAGER         5

#define  MSG_BUNDLER          6

#define  LINK_PROTOCOL        7

#define  BCAST_PROTOCOL       8

#define  ROUTE_DISTRIBUTOR    9

#define  CHANGEOVER_PROTOCOL  10

#define  NAME_DISTRIBUTOR     11

#define  MSG_FRAGMENTER       12



  

[tipc-discussion] queue_limit question

[Mark H. <ma...@os...>]

I was looking at the queue_limit array and noticed that location 5 is
getting initialized twice and location 8 isn't getting initialized at
all.  Is there something wrong there?

Also out of curiosity, why is DATA_HIGH set to TIPC_NORMAL_IMPORTANCE
and DATA_NORMAL set to TIPC_HIGH_IMPORTANCE?

Mark.

(0)	this->queue_limit[DATA_LOW] = window;
(2)	this->queue_limit[DATA_NORMAL] = (window / 3) * 4;
(1)	this->queue_limit[DATA_HIGH] = (window / 3) * 5;
(3)	this->queue_limit[DATA_NON_REJECTABLE] = (window / 3) * 6;
	/* Transiting data messages,inclusive FIRST_FRAGM */
(4)	this->queue_limit[DATA_LOW + 4] = 300;
(6)	this->queue_limit[DATA_NORMAL + 4] = 600;
(5)	this->queue_limit[DATA_HIGH + 4] = 900;
(7)	this->queue_limit[DATA_NON_REJECTABLE + 4] = 1200;
(5)	this->queue_limit[CONN_MANAGER] = 1200;
(9)	this->queue_limit[ROUTE_DISTRIBUTOR] = 1200;
(10)	this->queue_limit[CHANGEOVER_PROTOCOL] = 2500;
(11)	this->queue_limit[NAME_DISTRIBUTOR] = 3000;
	/* FRAGMENT and LAST_FRAGMENT packets */
(12)	this->queue_limit[MSG_FRAGMENTER] = 4000;

#define DATA_LOW                  TIPC_LOW_IMPORTANCE
#define DATA_HIGH                 TIPC_NORMAL_IMPORTANCE
#define DATA_NORMAL               TIPC_HIGH_IMPORTANCE
#define DATA_NON_REJECTABLE       TIPC_NON_REJECTABLE

#define TIPC_LOW_IMPORTANCE     0  /* default */
#define TIPC_NORMAL_IMPORTANCE	1
#define TIPC_HIGH_IMPORTANCE	2
#define TIPC_NON_REJECTABLE	3

#define  CONN_MANAGER         5
#define  MSG_BUNDLER          6
#define  LINK_PROTOCOL        7
#define  BCAST_PROTOCOL       8
#define  ROUTE_DISTRIBUTOR    9
#define  CHANGEOVER_PROTOCOL  10
#define  NAME_DISTRIBUTOR     11
#define  MSG_FRAGMENTER       12

-- 
Mark Haverkamp <ma...@os...>

Re: [tipc-discussion] LKML announcement ?

[Jon M. <jon...@er...>]

That won't be  a problem :-)=20

/Jon

Ling, Xiaofeng wrote:


From experience of some of my colleagues, peoples in LKML don't like to
see a project with everything ok to be announced.

They like to see a draft project with a long TODO list or even just
draft idea. So it should not matter it there are still bugs or undone
features.

More TODO will attract more people to contribute to the project. So
maybe the work before announce is to give out a TODO list that is long
enough.







 =20

-----Original Message-----

From:  tip...@li...
<mailto:tip...@li...>=20

[ mailto:tip...@li...
<mailto:tip...@li...> ] On Behalf

Of Jon Maloy

Sent: 2004=C4=EA6=D4=C222=C8=D5 10:11

To: tipc

Subject: [tipc-discussion] LKML announcement ?



Status, June 21:

I have reduced the number of open bug reports from 10 to 6

today, by fixing a number of minor issues: the capability test

for management commands, replaced the macro k_malloc()

with kmalloc(X,GFP_ATOMIC), got rid of the typedefs

tipc_net_addr_t and tipc_ref_t, etc.



I tried another one: to remove the caching of sk_buffs,

but performance got so lousy that I backed out of it. Transfer

time for short node internal messages increased from 6 us

to 9 us, i.e. 50 %, which I find very difficult to accept, no matter

what S. Hemminger and others may say.



Now there remains only four known bugs: the 66k stream

message problem, the 64-process problem, the changeover

problem, and the ORPHAN fragment problem.



Some of these should be possible to fix within the next days,

so now we are back to the old question:

Is TIPC ready to be announced at LKML ?

My company really wants me to do this by June 28 (next Monday),

and I can not  any longer see that there are any showstoppers

in the code.



What is the opinion of you others working with TIPC.  Do you

see any strong reasons (anything that can not be fixed until the 28th)

for not publishing TIPC now ?  Does OSDL want to make another

code review first ?



If you still think we should wait, or if you think we are ready to

go, please give me some feedback about this tomorrow or Wednesday.

(Thursday is holiday here in Quebec, and I will be out of office

even on Friday.)



/Jon





-------------------------------------------------------

This SF.Net email sponsored by Black Hat Briefings & Training.

Attend Black Hat Briefings & Training, Las Vegas July 24-29 -

digital self defense, top technical experts, no vendor pitches,

unmatched networking opportunities. Visit  www.blackhat.com <http://www.
blackhat.com>=20

_______________________________________________

tipc-discussion mailing list

tip...@li...
<mailto:tip...@li...>=20

https://lists.sourceforge.net/lists/listinfo/tipc-discussion
<https://lists.sourceforge.net/lists/listinfo/tipc-discussion>=20





   =20













 =20

RE: [tipc-discussion] LKML announcement ?

[Ling, X. <xia...@in...>]

From experience of some of my colleagues, peoples in LKML don't like to =
see a project with everything ok to be announced.
They like to see a draft project with a long TODO list or even just =
draft idea. So it should not matter it there are still bugs or undone =
features.
More TODO will attract more people to contribute to the project. So =
maybe the work before announce is to give out a TODO list that is long =
enough.



>-----Original Message-----
>From: tip...@li...
>[mailto:tip...@li...] On Behalf
>Of Jon Maloy
>Sent: 2004=C4=EA6=D4=C222=C8=D5 10:11
>To: tipc
>Subject: [tipc-discussion] LKML announcement ?
>
>Status, June 21:
>I have reduced the number of open bug reports from 10 to 6
>today, by fixing a number of minor issues: the capability test
>for management commands, replaced the macro k_malloc()
>with kmalloc(X,GFP_ATOMIC), got rid of the typedefs
>tipc_net_addr_t and tipc_ref_t, etc.
>
>I tried another one: to remove the caching of sk_buffs,
>but performance got so lousy that I backed out of it. Transfer
>time for short node internal messages increased from 6 us
>to 9 us, i.e. 50 %, which I find very difficult to accept, no matter
>what S. Hemminger and others may say.
>
>Now there remains only four known bugs: the 66k stream
>message problem, the 64-process problem, the changeover
>problem, and the ORPHAN fragment problem.
>
>Some of these should be possible to fix within the next days,
>so now we are back to the old question:
>Is TIPC ready to be announced at LKML ?
>My company really wants me to do this by June 28 (next Monday),
>and I can not  any longer see that there are any showstoppers
>in the code.
>
>What is the opinion of you others working with TIPC.  Do you
>see any strong reasons (anything that can not be fixed until the 28th)
>for not publishing TIPC now ?  Does OSDL want to make another
>code review first ?
>
>If you still think we should wait, or if you think we are ready to
>go, please give me some feedback about this tomorrow or Wednesday.
>(Thursday is holiday here in Quebec, and I will be out of office
>even on Friday.)
>
>/Jon
>
>
>-------------------------------------------------------
>This SF.Net email sponsored by Black Hat Briefings & Training.
>Attend Black Hat Briefings & Training, Las Vegas July 24-29 -
>digital self defense, top technical experts, no vendor pitches,
>unmatched networking opportunities. Visit www.blackhat.com
>_______________________________________________
>tipc-discussion mailing list
>tip...@li...
>https://lists.sourceforge.net/lists/listinfo/tipc-discussion
>
>

RE: [tipc-discussion] Checked in

[Ling, X. <xia...@in...>]

I've checked in two fix of node_lock.
I have not found other node_locks, Is there other lock problems?
________________________________

	From: Jon Maloy [mailto:jon...@er...]=20
	Sent: 2004=C4=EA6=D4=C218=C8=D5 21:16
	To: Ling, Xiaofeng
	Cc: tipc
	Subject: Re: [tipc-discussion] Checked in
=09
=09
	It is ok, except that the net_lock must be taken before the
	node_lock and released after the node_lock has been released.
	The net_lock is there to ensure that you can trust the pnode-
	pointer, and it is only within the scope of net_lock that
	node and link pointers can be used. (There is a description
	of the lock policy in net.c).
=09
	I saw this design at several places in the code; =
bcast_link_retransmit(),
	bcast_deferred_queue_add(),recaculate_gap(),recv_bcast_data() so
	you should go through it and eliminate all such cases.
=09
	Regards /jon
=09
=09
=09
	Ling, Xiaofeng wrote:
=09

		=20
	=09
		 =20

			-----Original Message-----
			From: tip...@li...=20
			[mailto:tip...@li...] On Behalf=20
			Of Jon Maloy
			Sent: 2004=C4=EA6=D4=C218=C8=D5 8:26
			To: tipc
			Subject: [tipc-discussion] Checked in
		=09
			Ling: I noted that at a couple of places in your code you
			grab the node lock, get hold of a link pointer, release the
			node lock, and then continue to use the obtained pointer.
			This must be changed,since a link pointer must never be accessed
			if its owner node is not locked.
			I made a comment in the code about this.
			   =20

	=09
		So is the lock releas shall be moved down until the "this" is not =
used?
		Is this fix ok? Is the net_lock needed?
	=09
		Index: sendbcast.c
		=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
		RCS file: /cvsroot/tipc/source/unstable/net/tipc/sendbcast.c,v
		retrieving revision 1.21
		diff -u -r1.21 sendbcast.c
		--- sendbcast.c	17 Jun 2004 23:58:36 -0000	1.21
		+++ sendbcast.c	18 Jun 2004 07:44:29 -0000
		@@ -606,22 +606,22 @@
		 			if (!likely(msg))
		 				return;
		=20
		-			read_lock_bh(&net_lock);
		                         /*=20
		                          * xfling,this is no good. The this-pointer =
must
		                          * not be touched after the node-lock is =
released
		                          * Keep the node-lock and net-lock until =
retransmit is done
		                          * /jon
		                         */
		-                        node_lock(pnode);
		+            node_lock(pnode);
		+			read_lock_bh(&net_lock);
		 			this =3D pnode->active_links[msg_link_selector(msg)];
		-                        node_unlock(pnode);
		 			read_unlock_bh(&net_lock);
		 			if (unlikely(!this)) {
		 				dbg("no link found\n");
		 				continue;
		 			}
		 			bcast_link_retransmit(this, buf, lastseq);
		+            node_unlock(pnode);
		=20
		 		}
		 	}
	=09
	=09
	=09
		 =20

			For the same reason the call to blink_outqueue_release()
			directly from tipc_recv_msg() should be removed. It is
			sufficient to check for release of the queue at any broadcast
			send and reception. If some buffers stay a few extra milliseconds
			in the queue before the are released does not matter, -it is the
			loss detection time that is critical.
			   =20

	=09
		Yes, I agree, the place is added for debuging convient at that time, =
and should be deleted.
		 =20

[tipc-discussion] LKML announcement ?

[Jon M. <jon...@er...>]

Status, June 21:
I have reduced the number of open bug reports from 10 to 6
today, by fixing a number of minor issues: the capability test
for management commands, replaced the macro k_malloc()
with kmalloc(X,GFP_ATOMIC), got rid of the typedefs
tipc_net_addr_t and tipc_ref_t, etc.

I tried another one: to remove the caching of sk_buffs,
but performance got so lousy that I backed out of it. Transfer
time for short node internal messages increased from 6 us
to 9 us, i.e. 50 %, which I find very difficult to accept, no matter
what S. Hemminger and others may say.

Now there remains only four known bugs: the 66k stream
message problem, the 64-process problem, the changeover
problem, and the ORPHAN fragment problem.

Some of these should be possible to fix within the next days,
so now we are back to the old question:
Is TIPC ready to be announced at LKML ?
My company really wants me to do this by June 28 (next Monday),
and I can not  any longer see that there are any showstoppers in the code.

What is the opinion of you others working with TIPC.  Do you
see any strong reasons (anything that can not be fixed until the 28th)
for not publishing TIPC now ?  Does OSDL want to make another
code review first ?  

If you still think we should wait, or if you think we are ready to
go, please give me some feedback about this tomorrow or Wednesday.
(Thursday is holiday here in Quebec, and I will be out of office
even on Friday.)

/Jon

Re: [tipc-discussion] cvs files checked in

[Jon M. <jon...@er...>]

I just checked in the capable() patch as suggested by Ling, for all
configuration commands capable of changing configuration
settings. I even found an acceptable solution for message commands,
by looking into that type messages at sending, and check capability
if the command is a set-command. (I had to renumber the command
macros in tipc.h to achieve this.) From now on all set-commands
must be done from root processes.

It is not a neat solution, but much better than having nothing.

/Jon

Mark Haverkamp wrote:

>I checked in files that fix a multicast deadlock.
>port.h
>port.c
>sendbcast.c
>
>Mark.
>
>  
>

[tipc-discussion] cvs files checked in

[Mark H. <ma...@os...>]

I checked in files that fix a multicast deadlock.
port.h
port.c
sendbcast.c

Mark.

-- 
Mark Haverkamp <ma...@os...>

Re: [tipc-discussion] Checked in

[Jon M. <jon...@er...>]

No problem. Go ahead.
/jon

Mark Haverkamp wrote:


On Fri, 2004-06-18 at 12:18, Jon Maloy wrote:



[...]

  

Mark Haverkamp wrote:

    

On Fri, 2004-06-18 at 11:07, Jon Maloy wrote:

  

      

You don't need  to lock the port at all when sending a message. 

If it is a user call the socket semaphore will protect it from 

any parallel access, and if it is a driver call the driver must

himself ensure that the port is not released while sending messages.

(I.e. the sound approach of not accessing the same port from different

threads/tasklets.)

Similarly, the incoming message data path does not access the same 

data fields in the port as the outgoing data path. 

The reason why incoming messages need to lock the port is to protect 

against inadvertent _release_ of the port from the user. So,

tipc_deleteport()  

grabs the port lock, but none of the tipc_send() functions.



=> Change port_lock() to port_deref() in tipc_multicast(), and it

should

work fine.

    

        

OK, that did it.

      



I plan on checking this in if there are no objections.



cvs diff -u port.h port.c sendbcast.c

Index: port.h

===================================================================

RCS file: /cvsroot/tipc/source/unstable/net/tipc/port.h,v

retrieving revision 1.13

diff -u -r1.13 port.h

--- port.h	16 Jun 2004 17:29:33 -0000	1.13

+++ port.h	21 Jun 2004 18:31:14 -0000

@@ -205,6 +205,11 @@

 	return (struct port*) ref_lock(ref);

 }

 

+static inline struct port* port_deref(tipc_ref_t ref)

+{

+	return (struct port*) ref_deref(ref);

+}

+

 /* 

  * port_unlock(): Unlock a port instance. Safe (and faster) to

  * use pointer instead of ref_unlock() since port already locked.

Index: port.c

===================================================================

RCS file: /cvsroot/tipc/source/unstable/net/tipc/port.c,v

retrieving revision 1.26

diff -u -r1.26 port.c

--- port.c	16 Jun 2004 17:29:33 -0000	1.26

+++ port.c	21 Jun 2004 18:31:14 -0000

@@ -246,11 +246,6 @@

 static void port_timeout(tipc_ref_t);

 

 

-static inline struct port* port_deref(tipc_ref_t ref)

-{

-	return (struct port*) ref_deref(ref);

-}

-

 static inline void port_set_timer(struct port* this,uint time) 

 {

 	if (this->timer_ref)

Index: sendbcast.c

===================================================================

RCS file: /cvsroot/tipc/source/unstable/net/tipc/sendbcast.c,v

retrieving revision 1.21

diff -u -r1.21 sendbcast.c

--- sendbcast.c	17 Jun 2004 23:58:36 -0000	1.21

+++ sendbcast.c	21 Jun 2004 18:31:14 -0000

@@ -221,7 +221,7 @@

 	struct tipc_msg *hdr;

 	struct tipc_portid orig;

 	struct sk_buff *b;

-	struct port *this = port_lock(ref);

+	struct port *this = port_deref(ref);

 

 

 	if (!this)



  

[tipc-discussion] Can now send messages in single node mode

[<all...@wi...>]

Hi all:
 
This morning I discovered that it isn't possible to send messages when a
node is operating in single node mode.  The problem is that
tipc_forward2name() and tipc_forward_buf2name() reject messages if the
destnode returned by nametbl_translate is 0 (which it always is in single
node mode), instead of rejecting messages if the destport returned by
nametbl_translate is 0 (indicating that the destination doesn't exist).
 
Since the fix was trivial (testing for a non-zero "destport", instead of a
non-zero "destnode") I've already coded it up and submitted it to the CVS
repository.  Give me a shout if there are any problems.
 
Regards,
Al Stephens
all...@wi...

Re: [tipc-discussion] Checked in

[Mark H. <ma...@os...>]

On Fri, 2004-06-18 at 12:18, Jon Maloy wrote:

[...]
> 
> Mark Haverkamp wrote:
> > On Fri, 2004-06-18 at 11:07, Jon Maloy wrote:
> >   
> > > You don't need  to lock the port at all when sending a message. 
> > > If it is a user call the socket semaphore will protect it from 
> > > any parallel access, and if it is a driver call the driver must
> > > himself ensure that the port is not released while sending messages.
> > > (I.e. the sound approach of not accessing the same port from different
> > > threads/tasklets.)
> > > Similarly, the incoming message data path does not access the same 
> > > data fields in the port as the outgoing data path. 
> > > The reason why incoming messages need to lock the port is to protect 
> > > against inadvertent _release_ of the port from the user. So,
> > > tipc_deleteport()  
> > > grabs the port lock, but none of the tipc_send() functions.
> > > 
> > > => Change port_lock() to port_deref() in tipc_multicast(), and it
> > > should
> > > work fine.
> > >     
> > OK, that did it.

I plan on checking this in if there are no objections.

cvs diff -u port.h port.c sendbcast.c
Index: port.h
===================================================================
RCS file: /cvsroot/tipc/source/unstable/net/tipc/port.h,v
retrieving revision 1.13
diff -u -r1.13 port.h
--- port.h	16 Jun 2004 17:29:33 -0000	1.13
+++ port.h	21 Jun 2004 18:31:14 -0000
@@ -205,6 +205,11 @@
 	return (struct port*) ref_lock(ref);
 }
 
+static inline struct port* port_deref(tipc_ref_t ref)
+{
+	return (struct port*) ref_deref(ref);
+}
+
 /* 
  * port_unlock(): Unlock a port instance. Safe (and faster) to
  * use pointer instead of ref_unlock() since port already locked.
Index: port.c
===================================================================
RCS file: /cvsroot/tipc/source/unstable/net/tipc/port.c,v
retrieving revision 1.26
diff -u -r1.26 port.c
--- port.c	16 Jun 2004 17:29:33 -0000	1.26
+++ port.c	21 Jun 2004 18:31:14 -0000
@@ -246,11 +246,6 @@
 static void port_timeout(tipc_ref_t);
 
 
-static inline struct port* port_deref(tipc_ref_t ref)
-{
-	return (struct port*) ref_deref(ref);
-}
-
 static inline void port_set_timer(struct port* this,uint time) 
 {
 	if (this->timer_ref)
Index: sendbcast.c
===================================================================
RCS file: /cvsroot/tipc/source/unstable/net/tipc/sendbcast.c,v
retrieving revision 1.21
diff -u -r1.21 sendbcast.c
--- sendbcast.c	17 Jun 2004 23:58:36 -0000	1.21
+++ sendbcast.c	21 Jun 2004 18:31:14 -0000
@@ -221,7 +221,7 @@
 	struct tipc_msg *hdr;
 	struct tipc_portid orig;
 	struct sk_buff *b;
-	struct port *this = port_lock(ref);
+	struct port *this = port_deref(ref);
 
 
 	if (!this)

-- 
Mark Haverkamp <ma...@os...>

Re: [tipc-discussion] Checked in

[Jon M. <jon...@er...>]

Yes, there is obviously something fishy there. I hope you and Ling
can spend some time on this. I will have to do some heavy-duty 
trouble-shooting on the 64-process problem, among other things.

/Jon

Mark Haverkamp wrote:


On Fri, 2004-06-18 at 11:07, Jon Maloy wrote:

  

You don't need  to lock the port at all when sending a message. 

If it is a user call the socket semaphore will protect it from 

any parallel access, and if it is a driver call the driver must

himself ensure that the port is not released while sending messages.

(I.e. the sound approach of not accessing the same port from different

threads/tasklets.)

Similarly, the incoming message data path does not access the same 

data fields in the port as the outgoing data path. 

The reason why incoming messages need to lock the port is to protect 

against inadvertent _release_ of the port from the user. So,

tipc_deleteport()  

grabs the port lock, but none of the tipc_send() functions.



=> Change port_lock() to port_deref() in tipc_multicast(), and it

should

work fine.

    



OK, that did it.



I see that you put some debug output in the link_recv_fragment

function.  I am now seeing lots of ORPHAN messages on the console when

sending large multicast messages.



Mark.



  

Re: [tipc-discussion] Checked in

[Mark H. <ma...@os...>]

On Fri, 2004-06-18 at 11:07, Jon Maloy wrote:
> You don't need  to lock the port at all when sending a message. 
> If it is a user call the socket semaphore will protect it from 
> any parallel access, and if it is a driver call the driver must
> himself ensure that the port is not released while sending messages.
> (I.e. the sound approach of not accessing the same port from different
> threads/tasklets.)
> Similarly, the incoming message data path does not access the same 
> data fields in the port as the outgoing data path. 
> The reason why incoming messages need to lock the port is to protect 
> against inadvertent _release_ of the port from the user. So,
> tipc_deleteport()  
> grabs the port lock, but none of the tipc_send() functions.
> 
> => Change port_lock() to port_deref() in tipc_multicast(), and it
> should
> work fine.

OK, that did it.

I see that you put some debug output in the link_recv_fragment
function.  I am now seeing lots of ORPHAN messages on the console when
sending large multicast messages.

Mark.

-- 
Mark Haverkamp <ma...@os...>