mpls-linux-devel — Mailig list for discussing the development of MPLS on Linux

Re: [mpls-linux-devel] Current state of dst stacking on davem implementation

[James R. L. <jl...@mi...>]

Just an FYI, for some reason my P4 server is not reachable.  I
may head home at lunchtime (CST) and check it out.

On Mon, Mar 29, 2004 at 10:06:08AM -0500, Jamal Hadi Salim wrote:
> On Mon, 2004-03-29 at 09:43, James R. Leu wrote:
> 
> > > Good enough; my experiences are it is on the exception side however
> > > if you guys say it aint then it aint.
> > 
> > I have never seen nor heard of a RFC or even a draft which describs
> > a protocol which distributes more then one level of label.
> 
> True, but you can leave the handling of how the stacking is done to user
> space and make the kernel portion simpler. I think this is a moot point
> for now so lets move on. 
>  
> > > This is fine for fast failover; challenge is it may end up being an
> > > expensive thing when trying to scale (2 lookups instead of 1 for one
> > > that says "goto c" - the cost could really add up at high speeds).
> > > Also, from a semantic view - doesnt the view of what a nhlfe change now
> > > with "goto c" syntax.
> > > We should still strive to have the MIB modelling as example apply
> > > (NHLFE table, ILM table etc). 
> > 
> > The data stored with the FWD instruction would be a pointer the the
> > next NHLFE.  Therefore no lookup is done.
> 
> I would have to wait and see the implementation piece. If i understood
> correctly though:
> - theres policy which chains together these little "commands" like
> "goto"
> - at policy creation time you will have a chaining of these commands;
> maybe a linked list of what applies (your reference to a "pointer to
> next NHLFE" above)
> - each of these commands will i suppose have a refcount which will be
> incremented for each policy that holds them
> - each of these commands will be separately editable; hence changing one
> changes multiple policies at the same time.
> 
> If i did understand correctly, then shouldnt the "commands" be a
> separate table? And the NHLFE holds the policies of what connects to
> what. No?
> Similarly the ILM could reference this "commands" table.
> 
> 
> > I have created a seperate branch off of 'mpls-kernel-davem' called
> > 'mpls-kernel-merger'.  It is where we can submit changes.  Every once in a
> > while we can look at the diff between 'mpls-kernel-davem' and
> > 'mpls-kernel-merger' and decide what gets propogated to 'mpls-kernel-davem'.
> 
> Sounds like a plan
> 
> > This next week is going to be very busy for me, so I will not be able to
> > do much development.  
> 
> Should give me opportunity to get started on setting up the environment.
> Im officialy first time in sane state at the office, so i should at
> least be able to set up the UML and the P4 setup
> 
> > Here is how I propose we start:
> > 
> > -I will submit the dst stacking code.
> > -rename all structures to proper names
> >     s/mpls_nhlfe_route/mpls_nhlfe/
> >     s/ltable/mpls_ilm/
> > -add instruction list to ILM and NHLFE
> > -move NHLFE label and nexthop to instructions
> > -move ILM -> NHLFE connection to instructions
> > -modify netlink to better take advantage of the new structure
> 
> All above sound reasonable. If you want i could help with the last one
> if you document the new structuring
> 
> > I was planning to take much of the code from our implementation and
> > modify it to work in the DaveM code.
> 
> This is also acceptable.
> 
> cheers,
> jamal
> 

-- 
James R. Leu
jl...@mi...

Re: [mpls-linux-devel] (1) mpls for linux-2.6 and (2) access to archives

[Ramon C. <cas...@in...>]

Hi Murthy,

On Mon, 29 Mar 2004, Murthy Andukuri wrote:

> Hi,
> I have newly joined the group and wondering if there is a version of
> mpls-linux for linux-2.6 .

There are two actually, but for now, you should follow James instructions
and set up p4 clients. Instructions on the project webpage
http://mpls-linux.sf.net.

James will help you with this.





> For this and other questions, I have been trying to access the archives
> on sourceforge but keep failing 'cause I get the message that access to the

I'm sorry, I don't manage the mailing lists.


Best regards,

Ramon

[mpls-linux-devel] (1) mpls for linux-2.6 and (2) access to archives

[Murthy A. <mur...@ho...>]

Hi,
I have newly joined the group and wondering if there is a version of 
mpls-linux
for linux-2.6 .
For this and other questions, I have been trying to access the archives
on sourceforge but keep failing 'cause I get the message that access to the
archives is limited to members - which I am. What am I missing ?

Thanks

_________________________________________________________________
Get tax tips, tools and access to IRS forms – all in one place at MSN Money! 
http://moneycentral.msn.com/tax/home.asp

Re: [mpls-linux-devel] Current state of dst stacking on davem implementation

[Jamal H. S. <ha...@zn...>]

On Mon, 2004-03-29 at 09:43, James R. Leu wrote:

> > Good enough; my experiences are it is on the exception side however
> > if you guys say it aint then it aint.
> 
> I have never seen nor heard of a RFC or even a draft which describs
> a protocol which distributes more then one level of label.

True, but you can leave the handling of how the stacking is done to user
space and make the kernel portion simpler. I think this is a moot point
for now so lets move on. 
 
> > This is fine for fast failover; challenge is it may end up being an
> > expensive thing when trying to scale (2 lookups instead of 1 for one
> > that says "goto c" - the cost could really add up at high speeds).
> > Also, from a semantic view - doesnt the view of what a nhlfe change now
> > with "goto c" syntax.
> > We should still strive to have the MIB modelling as example apply
> > (NHLFE table, ILM table etc). 
> 
> The data stored with the FWD instruction would be a pointer the the
> next NHLFE.  Therefore no lookup is done.

I would have to wait and see the implementation piece. If i understood
correctly though:
- theres policy which chains together these little "commands" like
"goto"
- at policy creation time you will have a chaining of these commands;
maybe a linked list of what applies (your reference to a "pointer to
next NHLFE" above)
- each of these commands will i suppose have a refcount which will be
incremented for each policy that holds them
- each of these commands will be separately editable; hence changing one
changes multiple policies at the same time.

If i did understand correctly, then shouldnt the "commands" be a
separate table? And the NHLFE holds the policies of what connects to
what. No?
Similarly the ILM could reference this "commands" table.


> I have created a seperate branch off of 'mpls-kernel-davem' called
> 'mpls-kernel-merger'.  It is where we can submit changes.  Every once in a
> while we can look at the diff between 'mpls-kernel-davem' and
> 'mpls-kernel-merger' and decide what gets propogated to 'mpls-kernel-davem'.

Sounds like a plan

> This next week is going to be very busy for me, so I will not be able to
> do much development.  

Should give me opportunity to get started on setting up the environment.
Im officialy first time in sane state at the office, so i should at
least be able to set up the UML and the P4 setup

> Here is how I propose we start:
> 
> -I will submit the dst stacking code.
> -rename all structures to proper names
>     s/mpls_nhlfe_route/mpls_nhlfe/
>     s/ltable/mpls_ilm/
> -add instruction list to ILM and NHLFE
> -move NHLFE label and nexthop to instructions
> -move ILM -> NHLFE connection to instructions
> -modify netlink to better take advantage of the new structure

All above sound reasonable. If you want i could help with the last one
if you document the new structuring

> I was planning to take much of the code from our implementation and
> modify it to work in the DaveM code.

This is also acceptable.

cheers,
jamal

Re: [mpls-linux-devel] Current state of dst stacking on davem implementation

[James R. L. <jl...@mi...>]

On Mon, Mar 29, 2004 at 09:04:14AM -0500, Jamal Hadi Salim wrote:
> Hi guys,
> 
> I am gonna stop apologizing for not responding on timely fashion. 
> Currently, this is beyond my control with the new baby.
> 
> On Fri, 2004-03-26 at 09:10, Ramon Casellas wrote:
> 
> > On Fri, 26 Mar 2004, Jamal Hadi Salim wrote:
> > 
> > > 
> > > Is this _always_ the case or mostly the exception?
> > 
> > To the best of my knowledge, it happens quite often,
> 
> Good enough; my experiences are it is on the exception side however
> if you guys say it aint then it aint.

I have never seen nor heard of a RFC or even a draft which describs
a protocol which distributes more then one level of label.

> > > > 3) The layer of indirection cause by the 'stacked' NHLFE allows
> > > > for fast fail-over by modifying only the bottom most NHLFE.
> > > 
> > > Isnt editing that NHLFE entry sufficient?
> > 
> > I have always thought that being able to specify the last instruction 
> > of a NHLFE 'a' to be "and now go follow NHLFE 'c'" provides a great amount 
> > of flexibility. I vote for being able to "chain" opcodes. In two cases:
> > 
> > - In instructions are chained to a NHLFE instructions
> > 
> > - NHLFEntries may refer to another NHLFE
> > 
> > NHLFE a:
> >  push 10
> >  goto c
> > 
> > NHLFE b:
> >  push 11
> >  goto c
> > 
> > NHLFE c:
> >  push 40
> >  fwd
> > 
> > by changing c I can engineer at the same time a and b. As James stated: 
> > path protection, LSP modification, even if it is not *strictly* required I 
> > think it may be good design and does not impact (noticiably) performance.
> > 
> 
> This is fine for fast failover; challenge is it may end up being an
> expensive thing when trying to scale (2 lookups instead of 1 for one
> that says "goto c" - the cost could really add up at high speeds).
> Also, from a semantic view - doesnt the view of what a nhlfe change now
> with "goto c" syntax.
> We should still strive to have the MIB modelling as example apply
> (NHLFE table, ILM table etc). 

The data stored with the FWD instruction would be a pointer the the
next NHLFE.  Therefore no lookup is done.

> > > Ok, generally i understand what you are proposing (i think) - to
> > > reiterate 
> > > 1) both the ILM and NHLFE should have opcodes in them 
> > 
> > yes. In instructions and out instructions.
> > 
> 
> For the sake of progress i would say lets move on and have this in.
> So where do we go next?
> I am going ot setup the uml thing on my laptop. Actually i need
> to install Fedora on a PC at home and use that as the real environment
> for this.
> 
> > 
> > > 2) a tunnel device
> > 
> > nice to have.
> > 
> > So DaveM code + In/Out instructions + tunnels + stacking entries = JLeu 
> > code + DaveM improvements. I agree that some parts are indeed not required 
> > (procfs, sysfs) but they are there for historical reasons and in the 
> > process of getting migrated.
> 
> Lets leave sysfs and profs last; i think we need to iron out the issue
> listed. We should really shoot to integrate something soon into the
> 2.6.x kernel. Lets make small releases - any suggestions?
> Ramon, do you wanna list all outstanding issues discussed so far - so we
> can see which one we attack incrementally?
> I dont think Davem will have any issues with the dst stacking code so
> that could be one of the first things that goes in;

I have created a seperate branch off of 'mpls-kernel-davem' called
'mpls-kernel-merger'.  It is where we can submit changes.  Every once in a
while we can look at the diff between 'mpls-kernel-davem' and
'mpls-kernel-merger' and decide what gets propogated to 'mpls-kernel-davem'.

This next week is going to be very busy for me, so I will not be able to
do much development.  Here is how I propose we start:

-I will submit the dst stacking code.
-rename all structures to proper names
    s/mpls_nhlfe_route/mpls_nhlfe/
    s/ltable/mpls_ilm/
-add instruction list to ILM and NHLFE
-move NHLFE label and nexthop to instructions
-move ILM -> NHLFE connection to instructions
-modify netlink to better take advantage of the new structure

I was planning to take much of the code from our implementation and
modify it to work in the DaveM code.

> cheers,
> jamal
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: IBM Linux Tutorials
> Free Linux tutorial presented by Daniel Robbins, President and CEO of
> GenToo technologies. Learn everything from fundamentals to system
> administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
> _______________________________________________
> mpls-linux-devel mailing list
> mpl...@li...
> https://lists.sourceforge.net/lists/listinfo/mpls-linux-devel

-- 
James R. Leu
jl...@mi...

Re: [mpls-linux-devel] Current state of dst stacking on davem implementation

[Jamal H. S. <ha...@zn...>]

Hi guys,

I am gonna stop apologizing for not responding on timely fashion. 
Currently, this is beyond my control with the new baby.

On Fri, 2004-03-26 at 09:10, Ramon Casellas wrote:

> On Fri, 26 Mar 2004, Jamal Hadi Salim wrote:
> 
> > 
> > Is this _always_ the case or mostly the exception?
> 
> To the best of my knowledge, it happens quite often,

Good enough; my experiences are it is on the exception side however
if you guys say it aint then it aint.

> > 
> > > 3) The layer of indirection cause by the 'stacked' NHLFE allows
> > > for fast fail-over by modifying only the bottom most NHLFE.
> > 
> > Isnt editing that NHLFE entry sufficient?
> 
> I have always thought that being able to specify the last instruction 
> of a NHLFE 'a' to be "and now go follow NHLFE 'c'" provides a great amount 
> of flexibility. I vote for being able to "chain" opcodes. In two cases:
> 
> - In instructions are chained to a NHLFE instructions
> 
> - NHLFEntries may refer to another NHLFE
> 
> NHLFE a:
>  push 10
>  goto c
> 
> NHLFE b:
>  push 11
>  goto c
> 
> NHLFE c:
>  push 40
>  fwd
> 
> by changing c I can engineer at the same time a and b. As James stated: 
> path protection, LSP modification, even if it is not *strictly* required I 
> think it may be good design and does not impact (noticiably) performance.
> 

This is fine for fast failover; challenge is it may end up being an
expensive thing when trying to scale (2 lookups instead of 1 for one
that says "goto c" - the cost could really add up at high speeds).
Also, from a semantic view - doesnt the view of what a nhlfe change now
with "goto c" syntax.
We should still strive to have the MIB modelling as example apply
(NHLFE table, ILM table etc). 


> > Ok, generally i understand what you are proposing (i think) - to
> > reiterate 
> > 1) both the ILM and NHLFE should have opcodes in them 
> 
> yes. In instructions and out instructions.
> 

For the sake of progress i would say lets move on and have this in.
So where do we go next?
I am going ot setup the uml thing on my laptop. Actually i need
to install Fedora on a PC at home and use that as the real environment
for this.

> 
> > 2) a tunnel device
> 
> nice to have.
> 
> So DaveM code + In/Out instructions + tunnels + stacking entries = JLeu 
> code + DaveM improvements. I agree that some parts are indeed not required 
> (procfs, sysfs) but they are there for historical reasons and in the 
> process of getting migrated.

Lets leave sysfs and profs last; i think we need to iron out the issue
listed. We should really shoot to integrate something soon into the
2.6.x kernel. Lets make small releases - any suggestions?
Ramon, do you wanna list all outstanding issues discussed so far - so we
can see which one we attack incrementally?
I dont think Davem will have any issues with the dst stacking code so
that could be one of the first things that goes in;

cheers,
jamal

Re: [mpls-linux-devel] Current state of dst stacking on davem implementation

[Ramon C. <cas...@in...>]

Hi Jamal,



See comments below,


On Fri, 26 Mar 2004, Jamal Hadi Salim wrote:

> 
> Is this _always_ the case or mostly the exception?

To the best of my knowledge, it happens quite often, and it may happen
even more often in the future with GMPLS. Different signalling protcols
may be used. You don't even need to be aware of GMPLS to need this (e.g.  
RSVP-TE for Traffic Engineered core and LDP/MP-iBGP for client route/pweid
signaling). Basically, I see it like an entity that pushes a label may not 
always know the stack depth. Anyway, James is more knowledgeable that I 
am, so he ay correct me if I'm wrong.







> 
> ok.
> 
> > 3) The layer of indirection cause by the 'stacked' NHLFE allows
> > for fast fail-over by modifying only the bottom most NHLFE.
> 
> Isnt editing that NHLFE entry sufficient?

I have always thought that being able to specify the last instruction 
of a NHLFE 'a' to be "and now go follow NHLFE 'c'" provides a great amount 
of flexibility. I vote for being able to "chain" opcodes. In two cases:

- In instructions are chained to a NHLFE instructions

- NHLFEntries may refer to another NHLFE

NHLFE a:
 push 10
 goto c

NHLFE b:
 push 11
 goto c

NHLFE c:
 push 40
 fwd

by changing c I can engineer at the same time a and b. As James stated: 
path protection, LSP modification, even if it is not *strictly* required I 
think it may be good design and does not impact (noticiably) performance.








> 
> Ok, generally i understand what you are proposing (i think) - to
> reiterate 
> 1) both the ILM and NHLFE should have opcodes in them 

yes. In instructions and out instructions.



> 2) a tunnel device

nice to have.



So DaveM code + In/Out instructions + tunnels + stacking entries = JLeu 
code + DaveM improvements. I agree that some parts are indeed not required 
(procfs, sysfs) but they are there for historical reasons and in the 
process of getting migrated.



Regards,
R.

Re: [mpls-linux-devel] Current state of dst stacking on davem implementation

[Jamal H. S. <ha...@zn...>]

James,
My std apologies for late response ;-<

On Wed, 2004-03-24 at 00:36, James R. Leu wrote:

> The very nature of LSPs (uni-directional) means that an ingress LER
> only TXs on a LSP. (ie only needs a NHLFE).  A separate LSP must be
> setup going in the reverse direction (with completely independent
> label values), and the egress LER will only RX on that LSP (ie only has
> an ILM).  Does that make sense?

Yes it does make sense; the value i saw of NHLFE knowing all details of 
the NH is so that in a purely ingress LER, there will be no ILM
table setup but now i am realizing even in your case there is no need
for any ILM setup.

> > I think we did touch on this - and i did see the value of the tunnel
> > device. For stacking though, why would it be an issue to have all the
> > labels in the same NHLFE entry? You seem to indicate that you need to
> > separate them?
> 
> 1) Separate signaling protocols distribute the label for each layer
> of the hierarchy.  No one protocol has all of the label information
> available to build the entire stack.

Is this _always_ the case or mostly the exception?

> 2) Each layer of the hierarchy can act on its own. (ie the outer most LSP
> can send traffic for the inner LSPs and IPv4 traffic.) Thus the outer
> LSP (bottom NHLFE) might be TXing packets with any number of labels, all
> of which were pushed at this LSR.  The number of labels pushed depends
> on the FEC to NHLFE mapping.

ok.

> 3) The layer of indirection cause by the 'stacked' NHLFE allows
> for fast fail-over by modifying only the bottom most NHLFE.

Isnt editing that NHLFE entry sufficient?

Ok, generally i understand what you are proposing (i think) - to
reiterate 
1) both the ILM and NHLFE should have opcodes in them 
2) a tunnel device

#1 i think was the controvesial part (re the current code) but you
explained the practicality of it. Did i get this correct?

cheers,
jamal

Re: [mpls-linux-devel] Current state of dst stacking on davem implementation

[James R. L. <jl...@mi...>]

Please see comments in-line.

On Wed, Mar 24, 2004 at 12:09:48AM -0500, Jamal Hadi Salim wrote:
> James,
> On Tue, 2004-03-23 at 23:47, James R. Leu wrote:
> > Jamal,
> > 
> > Before we can talk about LSP hierarchy we need to discuss the
> > single array of instructions on the NHLFE versus an array of
> > instructions on the ILM and the NHLFE.
> > 
> > Their is one main reason for the 'dual' model, NHLFE re-use.
> > NHLFE re-use comes into play for an LSP which an LSR is ingress
> > and transit, and for LSP hierarchy.
> > 
> > In the case where a LSR is ingress and transit for an LSP, a single set
> > of instructions on the NHLFE results in needing to create two NHLFE,
> > one with an ADD instruction, which is used by the ingress code path, and
> > one which contains a XCHANGE instruction for the transit code path.  Where
> > as with a 'dual' model, the instructions on the ILM contain POP and the
> > instructions on the NHLFE contain a PUSH.  The ingress code path only hits
> > the PUSH in the NHLFE and the transit code path hits the POP on the ILM
> > and the PUSH on the NHLFE.
> 
> Ok, I think i understand what you are saying.
> Question: Is it possible that a device could be only interfacing 
> as an ingress LER; i.e it sends MPLS packets into the MPLS cloud,
> but it never receives back MPLS packets? In other words such a device
> may not have any need for a ILM but will need a NHLFE. If thats a yes,
> then there is clear value for the separation, no? I have never heard
> of a setup like this so just curious.

(Forgive me if I'm stating the obvious, it was the only way I could think
of answering your question.  If I missed you point, please elaborate.)

The very nature of LSPs (uni-directional) means that an ingress LER
only TXs on a LSP. (ie only needs a NHLFE).  A separate LSP must be
setup going in the reverse direction (with completely independent
label values), and the egress LER will only RX on that LSP (ie only has
an ILM).  Does that make sense?  (ie LSPs are not like ATM VCs or ethernet
VLANs where TX and RX occur on the same entity).

> > The reasoning for the 'dual' model with respect to LSP hierarchy is similar,
> > but pertains to NHLFE which are 'stacked'.  (an LSR which adds hierarchy
> > really is just a special case of an ingress LER/LSR, remember that hierarchy
> > can be added at any point in the MPLS domain, not just the edge) The NHLFE
> > for the hierarchy label (an example of a hierarchy label is a VPN label)
> > contains a PUSH and FWD to another NHLFE, which contains a PUSH for the
> > tunnel label.  With this separation the tunnel NHLFE can be used for a
> > transit LSP as well.  This separation also lends itself well to fast
> > fail over between primary and backup LSPs.  Image 1000s of VPN NHLFEs
> > pointing to the same tunnel NHLFE, if the tunnel NHLFE needs to be
> > changed (to fail over to a backup tunnel) we can change just one NHLFE
> > and all of the VPN NHLFEs start using the backup tunnel.
> 
> I think we did touch on this - and i did see the value of the tunnel
> device. For stacking though, why would it be an issue to have all the
> labels in the same NHLFE entry? You seem to indicate that you need to
> separate them?

1) Separate signaling protocols distribute the label for each layer
of the hierarchy.  No one protocol has all of the label information
available to build the entire stack.

2) Each layer of the hierarchy can act on its own. (ie the outer most LSP
can send traffic for the inner LSPs and IPv4 traffic.) Thus the outer
LSP (bottom NHLFE) might be TXing packets with any number of labels, all
of which were pushed at this LSR.  The number of labels pushed depends
on the FEC to NHLFE mapping.

3) The layer of indirection cause by the 'stacked' NHLFE allows
for fast fail-over by modifying only the bottom most NHLFE.

> > That is enough for now.  I'm not ignoring the rest of your email, but
> > we can address the other issues/questions/points after we talk about
> > this one.
> 
> np.
> 
> cheers,
> jamal

-- 
James R. Leu
jl...@mi...

Re: [mpls-linux-devel] Current state of dst stacking on davem implementation

[Jamal H. S. <ha...@zn...>]

James,
On Tue, 2004-03-23 at 23:47, James R. Leu wrote:
> Jamal,
> 
> Before we can talk about LSP hierarchy we need to discuss the
> single array of instructions on the NHLFE versus an array of
> instructions on the ILM and the NHLFE.
> 
> Their is one main reason for the 'dual' model, NHLFE re-use.
> NHLFE re-use comes into play for an LSP which an LSR is ingress
> and transit, and for LSP hierarchy.
> 
> In the case where a LSR is ingress and transit for an LSP, a single set
> of instructions on the NHLFE results in needing to create two NHLFE,
> one with an ADD instruction, which is used by the ingress code path, and
> one which contains a XCHANGE instruction for the transit code path.  Where
> as with a 'dual' model, the instructions on the ILM contain POP and the
> instructions on the NHLFE contain a PUSH.  The ingress code path only hits
> the PUSH in the NHLFE and the transit code path hits the POP on the ILM
> and the PUSH on the NHLFE.

Ok, I think i understand what you are saying.
Question: Is it possible that a device could be only interfacing 
as an ingress LER; i.e it sends MPLS packets into the MPLS cloud,
but it never receives back MPLS packets? In other words such a device
may not have any need for a ILM but will need a NHLFE. If thats a yes,
then there is clear value for the separation, no? I have never heard
of a setup like this so just curious.

> The reasoning for the 'dual' model with respect to LSP hierarchy is similar,
> but pertains to NHLFE which are 'stacked'.  (an LSR which adds hierarchy
> really is just a special case of an ingress LER/LSR, remember that hierarchy
> can be added at any point in the MPLS domain, not just the edge) The NHLFE
> for the hierarchy label (an example of a hierarchy label is a VPN label)
> contains a PUSH and FWD to another NHLFE, which contains a PUSH for the
> tunnel label.  With this separation the tunnel NHLFE can be used for a
> transit LSP as well.  This separation also lends itself well to fast
> fail over between primary and backup LSPs.  Image 1000s of VPN NHLFEs
> pointing to the same tunnel NHLFE, if the tunnel NHLFE needs to be
> changed (to fail over to a backup tunnel) we can change just one NHLFE
> and all of the VPN NHLFEs start using the backup tunnel.

I think we did touch on this - and i did see the value of the tunnel
device. For stacking though, why would it be an issue to have all the
labels in the same NHLFE entry? You seem to indicate that you need to
separate them?

> That is enough for now.  I'm not ignoring the rest of your email, but
> we can address the other issues/questions/points after we talk about
> this one.

np.

cheers,
jamal

Re: [mpls-linux-devel] Current state of dst stacking on davem implementation

[James R. L. <jl...@mi...>]

Jamal,

Before we can talk about LSP hierarchy we need to discuss the
single array of instructions on the NHLFE versus an array of
instructions on the ILM and the NHLFE.

Their is one main reason for the 'dual' model, NHLFE re-use.
NHLFE re-use comes into play for an LSP which an LSR is ingress
and transit, and for LSP hierarchy.

In the case where a LSR is ingress and transit for an LSP, a single set
of instructions on the NHLFE results in needing to create two NHLFE,
one with an ADD instruction, which is used by the ingress code path, and
one which contains a XCHANGE instruction for the transit code path.  Where
as with a 'dual' model, the instructions on the ILM contain POP and the
instructions on the NHLFE contain a PUSH.  The ingress code path only hits
the PUSH in the NHLFE and the transit code path hits the POP on the ILM
and the PUSH on the NHLFE.

The reasoning for the 'dual' model with respect to LSP hierarchy is similar,
but pertains to NHLFE which are 'stacked'.  (an LSR which adds hierarchy
really is just a special case of an ingress LER/LSR, remember that hierarchy
can be added at any point in the MPLS domain, not just the edge) The NHLFE
for the hierarchy label (an example of a hierarchy label is a VPN label)
contains a PUSH and FWD to another NHLFE, which contains a PUSH for the
tunnel label.  With this separation the tunnel NHLFE can be used for a
transit LSP as well.  This separation also lends itself well to fast
fail over between primary and backup LSPs.  Image 1000s of VPN NHLFEs
pointing to the same tunnel NHLFE, if the tunnel NHLFE needs to be
changed (to fail over to a backup tunnel) we can change just one NHLFE
and all of the VPN NHLFEs start using the backup tunnel.

That is enough for now.  I'm not ignoring the rest of your email, but
we can address the other issues/questions/points after we talk about
this one.

-- 
James R. Leu
jl...@mi...

On Tue, Mar 23, 2004 at 10:15:06PM -0500, Jamal Hadi Salim wrote:
> Hi James,
> 
> You will have to forgive my delayed responses; seems like time is not
> my best friend right now.
> 
> On Mon, 2004-03-22 at 12:29, James R. Leu wrote:
> > See comments in line.
> > 
> 
> [..] 
> > > To be fair, all the above are resolvable issues. Some are even
> > > mentioned in the TODO list.
> > 
> > Some are quite trivial, but some require changes to the architecture.
> 
> Ok.
> 
> > > > -no support for LSP hierarchy (ingress or egress)
> > > >
> > > > The "no support for LSP hierarchy" is only one element in the list, but
> > > > it is a fairly large issue.
> > > 
> > > I didnt understand this one. Did we discuss this?
> > 
> > I mentioned in a previous email that evaluating the current architecture
> > without considering LSP hierarchy was a bit foolish.
> 
> So this seems to be the big one. I am trying to grasp me it myself so if
> you can explain it we can take it from there.
> 
> 
> > > Maybe we can do parallel approach and have
> > > both coming in together?
> > 
> > I reality the implementations are already converging because I'm flexible
> > with my implementation and I'm willing to learn from others and have thus
> > implemented what I see as the positive points of the DaveM code.
> 
> This is good.
> 
> > The remaining areas of difference are:
> > 
> > -instructions - I have 'ILM' and 'NHLFE' instructions, the DaveM code only
> >  has 'NHLFE' instructions.  Separating 'ILM' and 'NHLFE' processing in my
> >  mind is one of the key requirements for supporting scalable LSP hierarchy.
> >  If the 'single instruction' model is desirable for some configurations, it
> >  can be accomplished with my 'dual' instructions model, by making all of the
> >  'ILMs' have one instruction, which FWDs it to a NHLFE for processing.
> 
> Isnt the separation as it is right now in the Davem code ok?
> 
> > -storage - I use a generated key for storing 'NHLFE' info, the DaveM code
> >  uses the label value itself (see above list of issues as to why this
> >  it bad).  I store the 'ILM' and 'NHLFE' info in a radix tree, the DaveM
> >  code uses a hash table.
> 
> Radix tree, i see no issues changing to. Its probably a lesser concern
> right away.
> I think we discussed the nhlfe key a while back; can you elaborate more?
> 
> 
> > -user input - my implementation does not have a netlink interface (yet),
> >  I use IOCTLs().  The DaveM code uses netlink.
> 
> This is my doamin. Netlink is definetely the way to go. I am working on
> some distributed stuff and i really dont see anything else going in.
> 
> > -labelspace - I have thought out the issues with respect to labelspace and
> >  and have implemented a 'best of both worlds' scheme which allows the users
> >  application to implement whatever type of labelspace management they choose.
> >  The DaveM code only has labelspace 0.
> 
> That can be fixed; on a slightly different topic, theres some huge push
> to virtualize the linux stack (actually all of linux, but i care about
> the stack). I know you have some interest in VRs; i will ping you on
> this.
>  
> > -labeling types - I have support for all labeling types in my code (ATM, FR,
> >  and generic) and support for multiple interface type (ethernet, PPP, GRE).
> >  At one point in time I even had direct support for ATM interfaces (has
> >  since been removed because the ATM stack for linux is not designed for
> >  routers or switches).  The DaveM code only supports ethernet.
> 
> Yes, this is also in the TODO.
> 
> > -availability of kernel information - My implementation has implemented
> >  crude yet effective PROCFS and SYSFS interfaces, the DaveM code has neither.
> 
> I puke on both of them. Actually i was supposed to add procfs to it but
> didnt see the need for it. I think having them is gravy, but not a huge
> value differentiator. Maintaining large tables using sysfs is a
> guarnateed disaster.
> I could see use for sysfs for something like "turn on the debug flag for
> nhlfe"..
> 
> > Basically, if you rip out support for LSP hierarchy, PROCFS, SYSFS, and
> > a couple other features and add netlink support and convert from radix
> > to a hash table, you could convert my implementation into the DaveM
> > implementation.
> 
> Lets do that.
> The way i see it at this point is as follows: 
> We need to have a convergence between your code and Daves. DaveM will
> take care of making sure things move smoothly from one kernel to a new
> one (eg 2.6->2.7->2.8) etc him being the maintainer of the net code. You
> will be the other guy that owns this code. You know about MPLS more than
> Dave does - the architecture convergence is the marriage of the two
> styles (yours and Daves). Both have to be comfortable with the changes.
> There should really be a convergence otherwise this is going to become
> another freeswan divergence.  My role here is not a lot more than to
> shepherd and make sure that the code makes it in. My interest is really
> in the netlink code. I can make calls on any of the netlink stuff.
> 
> 
> > I think we should identify the desirable features of the DaveM code and
> > my code and decide whether it is easier to propagate the changes from mine
> > to DaveM or vice versa.  That is just my $.02.
> >
> > > Thoughts?
> > 
> > If we are still dead set on enhancing the DaveM code, lets decide
> > what is the first area to focus on and fix it.  I think the most
> > important area to look at is support for LSP hierarchy.
> 
> Ok, lets discuss the LSP hierachy thing. Note i am not dead set on
> anything. I wanna make sure that we do whats best for linux. I think
> that in the interest of a fast merge, we need to take the best of both
> and fuse into one. As an example the dst stacking thing from your code
> was great change. Radix tree we can make - although i would think that
> would be a lower priority compared to say the hierachy issue. Lets
> discuss the changes, make the code changes, stick your name in the
> authorship and lets get the code merged.
> 
> cheers,
> jamal

Re: [mpls-linux-devel] Current state of dst stacking on davem implementation

[Jamal H. S. <ha...@zn...>]

Hi James,

You will have to forgive my delayed responses; seems like time is not
my best friend right now.

On Mon, 2004-03-22 at 12:29, James R. Leu wrote:
> See comments in line.
> 

[..] 
> > To be fair, all the above are resolvable issues. Some are even
> > mentioned in the TODO list.
> 
> Some are quite trivial, but some require changes to the architecture.

Ok.

> > > -no support for LSP hierarchy (ingress or egress)
> > >
> > > The "no support for LSP hierarchy" is only one element in the list, but
> > > it is a fairly large issue.
> > 
> > I didnt understand this one. Did we discuss this?
> 
> I mentioned in a previous email that evaluating the current architecture
> without considering LSP hierarchy was a bit foolish.

So this seems to be the big one. I am trying to grasp me it myself so if
you can explain it we can take it from there.


> > Maybe we can do parallel approach and have
> > both coming in together?
> 
> I reality the implementations are already converging because I'm flexible
> with my implementation and I'm willing to learn from others and have thus
> implemented what I see as the positive points of the DaveM code.

This is good.

> The remaining areas of difference are:
> 
> -instructions - I have 'ILM' and 'NHLFE' instructions, the DaveM code only
>  has 'NHLFE' instructions.  Separating 'ILM' and 'NHLFE' processing in my
>  mind is one of the key requirements for supporting scalable LSP hierarchy.
>  If the 'single instruction' model is desirable for some configurations, it
>  can be accomplished with my 'dual' instructions model, by making all of the
>  'ILMs' have one instruction, which FWDs it to a NHLFE for processing.

Isnt the separation as it is right now in the Davem code ok?

> -storage - I use a generated key for storing 'NHLFE' info, the DaveM code
>  uses the label value itself (see above list of issues as to why this
>  it bad).  I store the 'ILM' and 'NHLFE' info in a radix tree, the DaveM
>  code uses a hash table.

Radix tree, i see no issues changing to. Its probably a lesser concern
right away.
I think we discussed the nhlfe key a while back; can you elaborate more?


> -user input - my implementation does not have a netlink interface (yet),
>  I use IOCTLs().  The DaveM code uses netlink.

This is my doamin. Netlink is definetely the way to go. I am working on
some distributed stuff and i really dont see anything else going in.

> -labelspace - I have thought out the issues with respect to labelspace and
>  and have implemented a 'best of both worlds' scheme which allows the users
>  application to implement whatever type of labelspace management they choose.
>  The DaveM code only has labelspace 0.

That can be fixed; on a slightly different topic, theres some huge push
to virtualize the linux stack (actually all of linux, but i care about
the stack). I know you have some interest in VRs; i will ping you on
this.
 
> -labeling types - I have support for all labeling types in my code (ATM, FR,
>  and generic) and support for multiple interface type (ethernet, PPP, GRE).
>  At one point in time I even had direct support for ATM interfaces (has
>  since been removed because the ATM stack for linux is not designed for
>  routers or switches).  The DaveM code only supports ethernet.

Yes, this is also in the TODO.

> -availability of kernel information - My implementation has implemented
>  crude yet effective PROCFS and SYSFS interfaces, the DaveM code has neither.

I puke on both of them. Actually i was supposed to add procfs to it but
didnt see the need for it. I think having them is gravy, but not a huge
value differentiator. Maintaining large tables using sysfs is a
guarnateed disaster.
I could see use for sysfs for something like "turn on the debug flag for
nhlfe"..

> Basically, if you rip out support for LSP hierarchy, PROCFS, SYSFS, and
> a couple other features and add netlink support and convert from radix
> to a hash table, you could convert my implementation into the DaveM
> implementation.

Lets do that.
The way i see it at this point is as follows: 
We need to have a convergence between your code and Daves. DaveM will
take care of making sure things move smoothly from one kernel to a new
one (eg 2.6->2.7->2.8) etc him being the maintainer of the net code. You
will be the other guy that owns this code. You know about MPLS more than
Dave does - the architecture convergence is the marriage of the two
styles (yours and Daves). Both have to be comfortable with the changes.
There should really be a convergence otherwise this is going to become
another freeswan divergence.  My role here is not a lot more than to
shepherd and make sure that the code makes it in. My interest is really
in the netlink code. I can make calls on any of the netlink stuff.


> I think we should identify the desirable features of the DaveM code and
> my code and decide whether it is easier to propagate the changes from mine
> to DaveM or vice versa.  That is just my $.02.
>
> > Thoughts?
> 
> If we are still dead set on enhancing the DaveM code, lets decide
> what is the first area to focus on and fix it.  I think the most
> important area to look at is support for LSP hierarchy.

Ok, lets discuss the LSP hierachy thing. Note i am not dead set on
anything. I wanna make sure that we do whats best for linux. I think
that in the interest of a fast merge, we need to take the best of both
and fuse into one. As an example the dst stacking thing from your code
was great change. Radix tree we can make - although i would think that
would be a lower priority compared to say the hierachy issue. Lets
discuss the changes, make the code changes, stick your name in the
authorship and lets get the code merged.

cheers,
jamal

Re: [mpls-linux-devel] Current state of dst stacking on davem implementation

[James R. L. <jl...@mi...>]

See comments in line.

On Mon, Mar 22, 2004 at 08:38:29AM -0500, Jamal Hadi Salim wrote:
> James,
> I really meant to respond to this sooner, but i suppose i am still faced
> with glitches.
> 
> On Fri, 2004-03-19 at 11:40, James R. Leu wrote:
> > Hey guys,
> > 
> > Here is my list of "issues" with the davem implementation:
> > 
> > -no handling of exceeding MTU, or to changes in devices MTU
> > -reserved label values are not restricted nor handled
> > -no support for flushing all ILM/NHLFE
> > -no handling of PUSH when SKB has no space
> > -tcpdump of egress LSR shows packet twice, once with MPLS header once without
> > -NHLFE structure has issues
> >     -same NHLFE cannot be used by transit LSP and locally originated traffic
> >         -Think of LDP with ordered control where an LSR could be ingress for
> >          a FEC as well as transit. (LSR received label mapping, it decides it
> >          can be ingress for the FEC, and then propagates the mapping, result
> >          it that the NHLFE is tied to a route entry and well as a ILM.  ILM
> >          needs XCHANGE, route entry needs ADD).
> >     -there is no such thing as a labelspace for NHLFE, it should be removed.
> >     -NHLFE hashing does not allow for same label mapping to be received
> >      from multiple peers in the same subnet
> >         -Think of an LSR speaking RSVP-TE connected to a ethernet subnet
> >          with 2 other RSVP-TE speakers.  RSVP-TE LSPs are requested from
> >          each of the other RSVP-TE speakers, they both advertise back the
> >          same label value.
> >     -does not allow for re-use of NHLFE when stacking
> >     -extra fields which would not be used when stacking
> 
> To be fair, all the above are resolvable issues. Some are even
> mentioned in the TODO list.

Some are quite trivial, but some require changes to the architecture.

> > -no support for LSP hierarchy (ingress or egress)
> >
> > The "no support for LSP hierarchy" is only one element in the list, but
> > it is a fairly large issue.
> 
> I didnt understand this one. Did we discuss this?

I mentioned in a previous email that evaluating the current architecture
without considering LSP hierarchy was a bit foolish.

> > Since I didn't want to make large changes to the davem code without discussion
> > I have been spending my time integrating the positive features of the davem
> > implementation into mine.  This consisted of adding the protocol driver
> > model and making better use of the the dst_entry to guide skb's through
> > the MPLS stack.
> >
> > In the process I've:
> > - implemented MPLS-ICMP for IPv4 (which has been implemented by vendors,
> >   but is only in draft form. It works with the NANOG patch to traceroute
> >   to show the labeling info.)  It is kind of a hack because I build the
> >   entire ICMP payload by hand.  I'm looking at ways to utilize the existing
> >   IPv4 ICMP code. (as an aside, it is cool to use an MPLS enable traceroute
> >   and see all of the providers who are using MPLS :-)
> > - the ICMP path has been modified to allow for offending messages to be
> >   replace by ICMP messages and forwarded along to the end of the LSP where
> >   the egress can handle it.
> > - the refcnt'ing code has been modified to disallow removing in/out
> >   information when some external entity is holding a ref (IPv4/IPv6/MPLS
> >   tunnel interface)
> > - Beginning of separation of sysfs code.  I love the sysfs code, but it needed
> >   to be separated so MPLS can be compiled without sysfs support.  The sysfs
> >   support also makes the implementation look more complicated then it really
> >   is.
> > 
> > I've just started adding netlink support.  My goal is to utilize all of the
> > netlink infrastructure Jamal added to the Davem implementation.  I want to
> > be user land API compliant between the implementations.
> > 
> > After netlink comes LSP load balancing (ILM based) using some of the schemes
> > we discussed previously.
> > 
> > In addition I've spent a lot of time working on a generic MPLS infrastructure
> > for quagga.  I'm to the point where I now need to add OS level support for
> > the various MPLS components.  I will make sure to write versions for
> > my code as well and the davem code.
> > 
> > I still think the best path going forward is to enhance my implementation
> > to be more agreeable to DaveM (I'm not sure what, if any, issues remain).
> > If you guys still think that the easiest path to adoption of MPLS code into
> > the Linux distribution is by modifying the DaveM code, I'll continue to help
> > out as much as possible, but we need to starting talking about making some
> > big changes.
> 
> The problem i am faced with is to come up with some strong reasons to go
> back to Dave. It will be unfair to him if we go back now - after all the
> time we passed - and say we decided to junk all the code he has written
> (and is willing to maintain). We need some strong reasons to make
> progress for inclusion. What you have listed seems to me as something
> someone with time can fix. Maybe we can do parallel approach and have
> both coming in together?

I reality the implementations are already converging because I'm flexible
with my implementation and I'm willing to learn from others and have thus
implemented what I see as the positive points of the DaveM code.

The remaining areas of difference are:

-instructions - I have 'ILM' and 'NHLFE' instructions, the DaveM code only
 has 'NHLFE' instructions.  Separating 'ILM' and 'NHLFE' processing in my
 mind is one of the key requirements for supporting scalable LSP hierarchy.
 If the 'single instruction' model is desirable for some configurations, it
 can be accomplished with my 'dual' instructions model, by making all of the
 'ILMs' have one instruction, which FWDs it to a NHLFE for processing.
-storage - I use a generated key for storing 'NHLFE' info, the DaveM code
 uses the label value itself (see above list of issues as to why this
 it bad).  I store the 'ILM' and 'NHLFE' info in a radix tree, the DaveM
 code uses a hash table.
-user input - my implementation does not have a netlink interface (yet),
 I use IOCTLs().  The DaveM code uses netlink.
-labelspace - I have thought out the issues with respect to labelspace and
 and have implemented a 'best of both worlds' scheme which allows the users
 application to implement whatever type of labelspace management they choose.
 The DaveM code only has labelspace 0.
-labeling types - I have support for all labeling types in my code (ATM, FR,
 and generic) and support for multiple interface type (ethernet, PPP, GRE).
 At one point in time I even had direct support for ATM interfaces (has
 since been removed because the ATM stack for linux is not designed for
 routers or switches).  The DaveM code only supports ethernet.
-availability of kernel information - My implementation has implemented
 crude yet effective PROCFS and SYSFS interfaces, the DaveM code has neither.

Basically, if you rip out support for LSP hierarchy, PROCFS, SYSFS, and
a couple other features and add netlink support and convert from radix
to a hash table, you could convert my implementation into the DaveM
implementation.

> > No matter what. I'll still be working on my implementation, if nothing
> > else it is an easy place for me to implement new features which could then
> > be ported to the DaveM code.
> > 
> 
> Maybe we can do this. To me the most valuable thing is to have the code
> merged. And the approach of improving over Davems code seems to be the
> most logical compromise. Perhaps your code can serve as an area for more
> experimental stuff which later gets merged. 

I think we should identify the desirable features of the DaveM code and
my code and decide whether it is easier to propagate the changes from mine
to DaveM or vice versa.  That is just my $.02.

> Thoughts?

If we are still dead set on enhancing the DaveM code, lets decide
what is the first area to focus on and fix it.  I think the most
important area to look at is support for LSP hierarchy.

> cheers,
> jamal
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: IBM Linux Tutorials
> Free Linux tutorial presented by Daniel Robbins, President and CEO of
> GenToo technologies. Learn everything from fundamentals to system
> administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
> _______________________________________________
> mpls-linux-devel mailing list
> mpl...@li...
> https://lists.sourceforge.net/lists/listinfo/mpls-linux-devel

-- 
James R. Leu
jl...@mi...

Re: [mpls-linux-devel] Current state of dst stacking on davem implementation

[Jamal H. S. <ha...@zn...>]

James,
I really meant to respond to this sooner, but i suppose i am still faced
with glitches.

On Fri, 2004-03-19 at 11:40, James R. Leu wrote:
> Hey guys,
> 
> Here is my list of "issues" with the davem implementation:
> 
> -no handling of exceeding MTU, or to changes in devices MTU
> -reserved label values are not restricted nor handled
> -no support for flushing all ILM/NHLFE
> -no handling of PUSH when SKB has no space
> -tcpdump of egress LSR shows packet twice, once with MPLS header once without
> -NHLFE structure has issues
>     -same NLFE cannot be used by transit LSP and locally originated traffic
>         -Think of LDP with ordered control where an LSR could be ingress for
>          a FEC as well as transit. (LSR received label mapping, it decides it
>          can be ingress for the FEC, and then propogates the mapping, result
>          it that the NHLFE is tied to a route entry and well as a ILM.  ILM
>          needs XCHANGE, route entry needs ADD).
>     -there is no such thing as a labelspace for NHLFE, it should be removed.
>     -NHLFE hashing does not allow for same label mapping to be received
>      from multiple peers in the same subnet
>         -Think of an LSR speaking RSVP-TE connected to a ethernet subnet
>          with 2 other RSVP-TE speakers.  RSVP-TE LSPs are requested from
>          each of the other RSVP-TE speakers, they both advertise back the
>          same label value.
>     -does not allow for re-use of NHLFE when stacking
>     -extra fields which would not be used when stacking

To be fair, all the above are resolvable issues. Some are even
mentioned in the TODO list.

> -no support for LSP hierarchy (ingress or egress)
>
> The "no support for LSP hierarchy" is only one element in the list, but
> it is a fairly large issue.

I didnt understand this one. Did we discuss this?

> Since I didn't want to make large changes to the davem code without discussion
> I have been spending my time integrating the positive features of the davem
> implementation into mine.  This consisted of adding the protocol driver
> model and making better use of the the dst_entry to guide skb's through
> the MPLS stack.
>
> In the process I've:
> - implemented MPLS-ICMP for IPv4 (which has been implemented by vendors,
>   but is only in draft form. It works with the NANOG patch to traceroute
>   to show the labelling info.)  It is kind of a hack because I build the
>   entire ICMP payload by hand.  I'm looking at ways to utilize the existing
>   IPv4 ICMP code. (as an aside, it is cool to use an MPLS enable traceroute
>   and see all of the providers who are using MPLS :-)
> - the ICMP path has been modified to allow for offending messages to be
>   replace by ICMP messages and forwarded along to the end of the LSP where
>   the egress can handle it.
> - the refcnt'ing code has been modified to disallow removing in/out
>   information when some external entitiy is holding a ref (IPv4/IPv6/MPLS
>   tunnel interface)
> - Begining of seperation of sysfs code.  I love the sysfs code, but it needed
>   to be seperated so MPLS can be compiled without sysfs support.  The sysfs
>   support also makes the implementation look more complicated then it really
>   is.
> 
> I've just started adding netlink support.  My goal is to utilize all of the
> netlink infrastructure Jamal added to the Davem implementation.  I want to
> be userland API compliant between the implementations.
> 
> After netlink comes LSP loadbalancing (ILM based) using some of the schemes
> we discussed previously.
> 
> In addition I've spent a lot of time working on a generic MPLS infrastructure
> for quagga.  I'm to the point where I now need to add OS level support for
> the various MPLS components.  I will make sure to write versions for
> my code as well and the davem code.
> 
> I still think the best path going forward is to enhance my implementation
> to be more agreeable to DaveM (I'm not sure what, if any, issues remain).
> If you guys still think that the easiest path to adoption of MPLS code into
> the Linux distribution is by modifing the DaveM code, I'll continue to help
> out as much as possible, but we need to starting talking about making some
> big changes.

The problem i am faced with is to come up with some strong reasons to go
back to Dave. It will be unfair to him if we go back now - after all the
time we passed - and say we decided to junk all the code he has written
(and is willing to maintain). We need some strong reasons to make
progress for inclusion. What you have listed seems to me as something
someone with time can fix. Maybe we can do parallel approach and have
both coming in together?

> No matter what. I'll still be working on my implementation, if nothing
> else it is an easy place for me to implement new features which could then
> be ported to the DaveM code.
> 

Maybe we can do this. To me the most valuable thing is to have the code
merged. And the approach of improving over Davems code seems to be the
most logical compromise. Perhaps your code can serve as an area for more
experimental stuff which later gets merged. 

Thoughts?

cheers,
jamal

Re: [mpls-linux-devel] Current state of dst stacking on davem implementation

[James R. L. <jl...@mi...>]

Hey guys,

Here is my list of "issues" with the davem implementation:

-no handling of exceeding MTU, or to changes in devices MTU
-reserved label values are not restricted nor handled
-no support for flushing all ILM/NHLFE
-no handling of PUSH when SKB has no space
-tcpdump of egress LSR shows packet twice, once with MPLS header once without
-NHLFE structure has issues
    -same NLFE cannot be used by transit LSP and locally originated traffic
        -Think of LDP with ordered control where an LSR could be ingress for
         a FEC as well as transit. (LSR received label mapping, it decides it
         can be ingress for the FEC, and then propogates the mapping, result
         it that the NHLFE is tied to a route entry and well as a ILM.  ILM
         needs XCHANGE, route entry needs ADD).
    -there is no such thing as a labelspace for NHLFE, it should be removed.
    -NHLFE hashing does not allow for same label mapping to be received
     from multiple peers in the same subnet
        -Think of an LSR speaking RSVP-TE connected to a ethernet subnet
         with 2 other RSVP-TE speakers.  RSVP-TE LSPs are requested from
         each of the other RSVP-TE speakers, they both advertise back the
         same label value.
    -does not allow for re-use of NHLFE when stacking
    -extra fields which would not be used when stacking
-no support for LSP hierarchy (ingress or egress)

The "no support for LSP hierarchy" is only one element in the list, but
it is a fairly large issue.

Since I didn't want to make large changes to the davem code without discussion
I have been spending my time integrating the positive features of the davem
implementation into mine.  This consisted of adding the protocol driver
model and making better use of the the dst_entry to guide skb's through
the MPLS stack.

In the process I've:
- implemented MPLS-ICMP for IPv4 (which has been implemented by vendors,
  but is only in draft form. It works with the NANOG patch to traceroute
  to show the labelling info.)  It is kind of a hack because I build the
  entire ICMP payload by hand.  I'm looking at ways to utilize the existing
  IPv4 ICMP code. (as an aside, it is cool to use an MPLS enable traceroute
  and see all of the providers who are using MPLS :-)
- the ICMP path has been modified to allow for offending messages to be
  replace by ICMP messages and forwarded along to the end of the LSP where
  the egress can handle it.
- the refcnt'ing code has been modified to disallow removing in/out
  information when some external entitiy is holding a ref (IPv4/IPv6/MPLS
  tunnel interface)
- Begining of seperation of sysfs code.  I love the sysfs code, but it needed
  to be seperated so MPLS can be compiled without sysfs support.  The sysfs
  support also makes the implementation look more complicated then it really
  is.

I've just started adding netlink support.  My goal is to utilize all of the
netlink infrastructure Jamal added to the Davem implementation.  I want to
be userland API compliant between the implementations.

After netlink comes LSP loadbalancing (ILM based) using some of the schemes
we discussed previously.

In addition I've spent a lot of time working on a generic MPLS infrastructure
for quagga.  I'm to the point where I now need to add OS level support for
the various MPLS components.  I will make sure to write versions for
my code as well and the davem code.

I still think the best path going forward is to enhance my implementation
to be more agreeable to DaveM (I'm not sure what, if any, issues remain).
If you guys still think that the easiest path to adoption of MPLS code into
the Linux distribution is by modifing the DaveM code, I'll continue to help
out as much as possible, but we need to starting talking about making some
big changes.

No matter what. I'll still be working on my implementation, if nothing
else it is an easy place for me to implement new features which could then
be ported to the DaveM code.

On Fri, Mar 19, 2004 at 10:52:33AM -0500, Jamal Hadi Salim wrote:
> hi gents,
> I am semi back - hopefully we can get this done soon so we shoot 
> to submit?
> Where were we? ;-> 
> 
> cheers,
> jamal
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: IBM Linux Tutorials
> Free Linux tutorial presented by Daniel Robbins, President and CEO of
> GenToo technologies. Learn everything from fundamentals to system
> administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
> _______________________________________________
> mpls-linux-devel mailing list
> mpl...@li...
> https://lists.sourceforge.net/lists/listinfo/mpls-linux-devel

-- 
James R. Leu
jl...@mi...

Re: [mpls-linux-devel] Current state of dst stacking on davem implementation

[Jamal H. S. <ha...@zn...>]

hi gents,
I am semi back - hopefully we can get this done soon so we shoot 
to submit?
Where were we? ;-> 

cheers,
jamal

Re: [mpls-linux-devel] Current state of dst stacking on davem implementation

[Jamal H. S. <ha...@zn...>]

Hi James,
I am in recovery mode. Kid one week old. I am not sure if i ever
responded to this.

On Wed, 2004-03-03 at 10:24, James R. Leu wrote:

> I wonder if there is a way to change the nhlfe options so that they make more
> sense when defining a local delivery NHLFE.

What do you have in mind?

cheers,
jamal

Re: [mpls-linux-devel] Current state of dst stacking on davem implementation

[James R. L. <jl...@mi...>]

On Wed, Mar 03, 2004 at 06:21:08AM -0500, Jamal Hadi Salim wrote:
> On Mon, 2004-03-01 at 00:03, James R. Leu wrote:
> 
> > Items left to be resolved:
> > -correct l2c command line for input definition?  (are we open for discussing
> >  changes?)
> 
> I will look at it and get back to you.

I wonder if there is a way to change the nhlfe options so that they make more
sense when defining a local delivery NHLFE.

> > -correct model for refcnting nhlfe's.  I have an idea:
> >    -'imagined' refcnt for holding nhlfe in hash table, no actual
> >     inc or dec when adding/deleting to hash table.  This way a simple
> >     check for refcnt != 0 will tell if some other layer is using the
> >     nhlfe (ILM, IPv4 rt_cache/route table, IPv6 route table, etc).  Deletes
> >     are denied until refcnt == 0, at which time a dst_free is executed.
> >    -question: OK to let a dst_entry have refcnt of 0?  Do I need to handle
> >     any special cases in the dst_ops to account for the refcnt != 0 cases?
> >    -I have this implemented on my code and am doing long term testing (>24hrs)
> >     to see if anything weird happens to ths refcnting.
> 
> I havent looked at code, but:
> As a general rule, unless you have no choice, it is safer to just use
> refcounting for atomicity purposes (eg SMP safety - the stack is pretty
> much reentrant and could run concurently on multiple processors) 

I think I have a scheme that works, I'll post an updated diff this weekend.

> > BTW I had to use '-p6' to apply the patch
> 
> That should be fixable.

This is from using p4's diff to generate the patch.

> 
> cheers,
> jamal
> 
> 
> 
> -------------------------------------------------------
> SF.Net is sponsored by: Speed Start Your Linux Apps Now.
> Build and deploy apps & Web services for Linux with
> a free DVD software kit from IBM. Click Now!
> http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
> _______________________________________________
> mpls-linux-devel mailing list
> mpl...@li...
> https://lists.sourceforge.net/lists/listinfo/mpls-linux-devel

-- 
James R. Leu
jl...@mi...

Re: [mpls-linux-devel] Current state of dst stacking on davem implementation

[Jamal H. S. <ha...@zn...>]

Hi James,
I didnt apply the patch or test it but it looks clean. I like some of
the cleanups.
I wasnt sure about a few things (which maybe harder to see because i am
reading a patch);
1) ipv4/fib_semantics.c, you got rid of the check:

--
if (fi->fib_nh && fi->fib_nh->nh_mpls_fec) {
---

Is this in the spirit of avoiding refcount inc/dec?
BTW, i think it is a good idea to probably keep the refcounts inc/dec;
What is the main reason you are trying to get rid of them?

2) You no longer do a mpls_bind_neighbour(); is this covered elsewhere?

3) The check:
     if (NULL != mpls) {
               int usrs = atomic_read(&mpls->mr_ref);

is to ensure that no user apace deleted a nhlfe entry while some
other place is using it.

4) I suppose mpls_nhlfe_release() is no longer needed now because
of the refcounting changes?

Anyways, overall i would say this is an improvement and we should
have no challenges getting past Davem.

On Mon, 2004-03-01 at 00:03, James R. Leu wrote:
> I've attached a patch against the head of line of the p4 depot for the davem
> implementation.  I tested the output path and it correctly generates MPLS
> packets.  I was unable to figure out the correct combination of l2c
> commands on the input side.  So I gave up testing the input path for
> tonight.  I thought I'd send out what I have so you can look it over and
> get an understanding of where I'm heading with this change.  Of course you
> could always look at the latest changes to my implementation and you'd
> probably get an even better idea ;-)
> 
> Items left to be resolved:
> -correct l2c command line for input definition?  (are we open for discussing
>  changes?)

This is my domain and i am open. Lets just make sure that the ideas
have minimal impact on Daves code and 2 they are guided by useful
changes as opposed to sentimental ones (example "this is how we did it
before").

cheers,
jamal

Re: [mpls-linux-devel] Current state of dst stacking on davem implementation

[Jamal H. S. <ha...@zn...>]

On Mon, 2004-03-01 at 00:03, James R. Leu wrote:

> Items left to be resolved:
> -correct l2c command line for input definition?  (are we open for discussing
>  changes?)

I will look at it and get back to you.

> -correct model for refcnting nhlfe's.  I have an idea:
>    -'imagined' refcnt for holding nhlfe in hash table, no actual
>     inc or dec when adding/deleting to hash table.  This way a simple
>     check for refcnt != 0 will tell if some other layer is using the
>     nhlfe (ILM, IPv4 rt_cache/route table, IPv6 route table, etc).  Deletes
>     are denied until refcnt == 0, at which time a dst_free is executed.
>    -question: OK to let a dst_entry have refcnt of 0?  Do I need to handle
>     any special cases in the dst_ops to account for the refcnt != 0 cases?
>    -I have this implemented on my code and am doing long term testing (>24hrs)
>     to see if anything weird happens to ths refcnting.

I havent looked at code, but:
As a general rule, unless you have no choice, it is safer to just use
refcounting for atomicity purposes (eg SMP safety - the stack is pretty
much reentrant and could run concurently on multiple processors) 

> BTW I had to use '-p6' to apply the patch

That should be fixable.

cheers,
jamal

Re: [mpls-linux-devel] Current state of dst stacking on davem implementation

[Jamal H. S. <ha...@zn...>]

Gents,
I apologize for dissapearing one more time; iam having a more
than usual busy schedule (baby probaly coming by this weekend
and work overloaded with new product deadlines).
James, I am going to look at what you sent and give you feedback.
I also hope to setup a environment for testing etc.

cheers,
jamal

[mpls-linux-devel] Current state of dst stacking on davem implementation

[James R. L. <jl...@mi...>]

I've attached a patch against the head of line of the p4 depot for the davem
implementation.  I tested the output path and it correctly generates MPLS
packets.  I was unable to figure out the correct combination of l2c
commands on the input side.  So I gave up testing the input path for
tonight.  I thought I'd send out what I have so you can look it over and
get an understanding of where I'm heading with this change.  Of course you
could always look at the latest changes to my implementation and you'd
probably get an even better idea ;-)

Items left to be resolved:
-correct l2c command line for input definition?  (are we open for discussing
 changes?)
-correct model for refcnting nhlfe's.  I have an idea:
    -'imagined' refcnt for holding nhlfe in hash table, no actual
     inc or dec when adding/deleting to hash table.  This way a simple
     check for refcnt != 0 will tell if some other layer is using the
     nhlfe (ILM, IPv4 rt_cache/route table, IPv6 route table, etc).  Deletes
     are denied until refcnt == 0, at which time a dst_free is executed.
   -question: OK to let a dst_entry have refcnt of 0?  Do I need to handle
    any special cases in the dst_ops to account for the refcnt != 0 cases?
   -I have this implemented on my code and am doing long term testing (>24hrs)
    to see if anything weird happens to ths refcnting.

BTW I had to use '-p6' to apply the patch

-- 
James R. Leu
jl...@mi...

Re: [mpls-linux-devel] Hi all

[Ramon C. <cas...@in...>]

On 23 Feb 2004, Jamal Hadi Salim wrote:

> On Mon, 2004-02-23 at 07:43, Ramon Casellas wrote:
>
> [..]


> Sounds reasonable. Did you end up changing the fecid variable as well?

Yes it's almost done. I'll submit the changeset today.


> It seems to me a global search and replace on the patch itself would be
> the best option. Then you apply the patch.

Umm, since the code is now in James' p4 repo. (not that there have been a
lot of changes, though) I think I'll stick to p4

Thanks,

R.

Re: [mpls-linux-devel] Hi all

[Jamal H. S. <ha...@zn...>]

On Mon, 2004-02-23 at 07:43, Ramon Casellas wrote:

[..]

> 
> Update:
> ******
> Nahhh, I bet 'today salary' that you do not have CONFIG_IP_ROUTE_MULTIPATH
> .that would explain it... :)

Ok you can have this months salary because this is the main reason
it wasnt compiling ;->
Note, I did not test multi path.

> Other things that I will cleanup (as soon as James server is online)
> net/mpls/mpls_fib.c: In function `mpls_ilm_lookup':
> net/mpls/mpls_fib.c:212: warning: ISO C90 forbids mixed declarations and
> code
> net/mpls/mpls_fib.c: In function `lt_fill_ilm':
> net/mpls/mpls_fib.c:340: warning: implicit declaration of function
> `gen_copy_stats'
> net/mpls/mpls_fib.c: In function `mpls_get_ilm':
> net/mpls/mpls_fib.c:446: warning: unused variable `lt'
> net/mpls/mpls_fib.c:447: warning: unused variable `i'
> net/netlink/af_netlink.c: In function `netlink_proto_init':
> net/netlink/af_netlink.c:1136: warning: implicit declaration of function
> `l2cnetlink_init'
> 

Sounds reasonable. Did you end up changing the fecid variable as well?
It seems to me a global search and replace on the patch itself would be
the best option. Then you apply the patch.

cheers,
jamal

Re: [mpls-linux-devel] Hi all

[Ramon C. <cas...@in...>]

On 23 Feb 2004, Jamal Hadi Salim wrote:

> didnt mean to desert you - things have been extremely hectic. Still are
> (both at work and home). So i hope you wont mind me being intermittent.
> I will try my best to respond when i believe its high priority like in
> this case.

I understand... I will also have some other projects that may impact my
productivity :)


>
> Scary - It compiles just fine for me.

Uhm...


> Substitute that nhl to be nhp - its what it should be.

Well, I _know_ that :)



> ------
> [root@jzny 261-mod]# gcc -v

gandalf casellas$ gcc -v
Reading specs from /usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.3/specs
Configured with: /var/tmp/portage/gcc-3.3.3/work/gcc-3.3.3/configure
--prefix=/usr --bindir=/usr/i686-pc-linux-gnu/gcc-bin/3.3
--includedir=/usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.3/include
--datadir=/usr/share/gcc-data/i686-pc-linux-gnu/3.3
--mandir=/usr/share/gcc-data/i686-pc-linux-gnu/3.3/man
--infodir=/usr/share/gcc-data/i686-pc-linux-gnu/3.3/info --enable-shared
--host=i686-pc-linux-gnu --target=i686-pc-linux-gnu --with-system-zlib
--enable-languages=c,c++,f77,objc,java --enable-threads=posix
--enable-long-long --disable-checking --enable-cstdio=stdio
--enable-clocale=generic --enable-__cxa_atexit
--enable-version-specific-runtime-libs
--with-gxx-include-dir=/usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.3/include/g++-v3
--with-local-prefix=/usr/local --enable-shared --enable-nls
--without-included-gettext --x-includes=/usr/X11R6/include
--x-libraries=/usr/X11R6/lib --enable-interpreter --enable-java-awt=xlib
--with-x --disable-multilib
Thread model: posix
gcc version 3.3.3 20040217


>
> I susbtituted that nhl with nhx and it still compiled.

gandalf mpls-kernel-davem# grep -bn MPLS .config
583:12615:CONFIG_IP_MPLS=y
603:12997:CONFIG_INET6_MPLS=m
706:15439:CONFIG_NET_MPLS=y


> ...
> Why did yours work?

What happens in your tree if you put a #error directive in other parts of
the file/other files? I'm afraid that you just did not compile MPLS
support in...




> Can you replace with the above #error and see if still compiles?

  CC      net/ipv4/fib_semantics.o
net/ipv4/fib_semantics.c:351:2: #error "I bet this month salary that this
message is displayed"
make[2]: *** [net/ipv4/fib_semantics.o] Error 1
make[1]: *** [net/ipv4] Error 2
make: *** [net] Error 2


Well, what did you expect ? :))



Update:
******
Nahhh, I bet 'today salary' that you do not have CONFIG_IP_ROUTE_MULTIPATH
.that would explain it... :)


Other things that I will cleanup (as soon as James server is online)
net/mpls/mpls_fib.c: In function `mpls_ilm_lookup':
net/mpls/mpls_fib.c:212: warning: ISO C90 forbids mixed declarations and
code
net/mpls/mpls_fib.c: In function `lt_fill_ilm':
net/mpls/mpls_fib.c:340: warning: implicit declaration of function
`gen_copy_stats'
net/mpls/mpls_fib.c: In function `mpls_get_ilm':
net/mpls/mpls_fib.c:446: warning: unused variable `lt'
net/mpls/mpls_fib.c:447: warning: unused variable `i'
net/netlink/af_netlink.c: In function `netlink_proto_init':
net/netlink/af_netlink.c:1136: warning: implicit declaration of function
`l2cnetlink_init'



Regards,
R.