linux-vrf-core — Mailing list for core developers

Re: [linux-vrf-core] First code release

[Nick E. <ni...@dc...>]

Jim,

Could you take a few minutes and type up a note that describes from a
system level the way you have implemented this?  How are you
conceptualizing the kernel mods to support the vrfs?

Thanks!

Nick

On Wed, 6 Mar 2002, James R. Leu wrote:

> Hey guys,
>
> I know its been a LONG time since we talked about this, but I have been
> thinking about it, and I've done some work.  I placed my first crack at a
> VRF implementation for the kernel.  Take a look at the README inside it
> then post questions :-)
>
> Jim
>

[linux-vrf-core] First code release

[James R. L. <jl...@mi...>]

Hey guys,

I know its been a LONG time since we talked about this, but I have been
thinking about it, and I've done some work.  I placed my first crack at a
VRF implementation for the kernel.  Take a look at the README inside it
then post questions :-)

Jim
-- 
James R. Leu

Re: [linux-vrf-core] VRF for Linux (MPLS IP VPNs)

[Nick E. <ni...@dc...>]

Jim,

I noticed that this mailing list is not visible to anonymous users
browsing sourceforge.  Is this by design?

--nick

Re: [linux-vrf-core] VRF for Linux (MPLS IP VPNs)

[Nick E. <ni...@dc...>]

Actually, the allegro "Virtual Router" approach is very similar to what we
are looking at, or rather our recent topics can be considered a blending
of the two approaches.

Allegro's approach involves dedicated hardware resources for each virtual
router, which we have not addressed.

Out first approach, using UML, is currently working.  The problem is that
UML imposes very servere performance penalties, but each UML actually is a
virtual router.

The second approach involves pushing the separation or virualization down
into the kernel.  This is obviously a lot more work, but the recently
discovered "server context" work (http://www.solucorp.qc.ca/miscprj/s_context.hc)
goes a good way toward making this a reality.

(The code is good, but really weak in the networking department.  We need
to locate all of the roots in the net subsystem and push them into the
s_context structure.  At least, that is what I think right now)

If we get this right, we will get vrfs in the "default" s-context, which
will be the basis for the "virtual servers" in the "non-default" contexts.

Comments?

--Nick

On Mon, 12 Nov 2001, Manish Karir wrote:

>
> As we are still feeling around for scope here...
> Maybe we should consider the approach that Allegro is
> taking....
>
> http://www.allegronetworks.com/technology/
>
> is this not a cleaner nicer way of doing VRF's??
> or am I missing something quite fundamental here?
>
> manish
>
>
>
> On Thu, 1 Nov 2001, Nick Eggleston wrote:
>
>
>
> _______________________________________________
> linux-vrf-core mailing list
> lin...@li...
> https://lists.sourceforge.net/lists/listinfo/linux-vrf-core
>

Re: [linux-vrf-core] VRF for Linux (MPLS IP VPNs)

[Manish K. <ka...@is...>]

As we are still feeling around for scope here...
Maybe we should consider the approach that Allegro is 
taking....

http://www.allegronetworks.com/technology/

is this not a cleaner nicer way of doing VRF's??
or am I missing something quite fundamental here?

manish



On Thu, 1 Nov 2001, Nick Eggleston wrote:

Re: [linux-vrf-core] VRF for Linux (MPLS IP VPNs)

[James R. L. <jl...@mi...>]

New stuff to play with, it has the right ideas for process/security/memory
but lack much of what we need for the IP forwarding.

I've contacted the author to see if there is a chance for colaoration.

http://www.solucorp.qc.ca/miscprj/s_context.hc

Jim

On Thu, Oct 25, 2001 at 08:51:44AM -0500, Nick Eggleston wrote:
> I've been doing more thinking about this.
> 
> I think the first thing we should tackle is the UML approach.  It provides
> a while virtual router interface, and shoud be fairly easy to set up.  I
> am not too familiar with it, so I will definately need help
> interconnecting the UMLs for networking with VLANs (and MPLS, for you).
> 
> The other design will involve changes to the routing, user/process socket
> layers to support VRF separation in the kernel, which is going to take a
> while to implement.  I still think it would be useful to implement such a
> system, but let's get things working with UML, first.
> 
> What do you think?
> 
> --Nick
> 
> On Thu, 11 Oct 2001, James R. Leu wrote:
> 
> > On Wed, Oct 10, 2001 at 09:35:47AM -0500, Nick Eggleston wrote:
> > <snip>
> > > Please read the attached references.  Here are some very sketchy ideas on
> > > potential implemenations approaches for Linux:
> > >
> > >
> > > 1. minimal changes - keep existing rules and tables, use fwmarks to
> > > distinguish traffic internally.  This CAN work for non-local traffic,
> > > and it is even possible with the OWNER netfilter target to set the
> > > fwmark for localally GENERATED traffic.  However, how to handle
> > > local reception is not understood.  I particular, how RIP, OSPF, BGP
> > > and the user process subsystem need to be modified is incomplete, and
> > > these would need to be modified to push their routes into specific tables.
> >
> > Interesting.  I hadn't thought about using fwmark.  I will look around
> > about how that would work.
> >
> > > How do we fix up local subsystem to mark sockets (so gated, zebra, routed)
> > > work right.  What about ping, traceroute, telnet, etc?  /etc/hosts, dns?
> >
> > I think if we provide a means for a socket to say which VRF it wishes
> > to use we cover all of the cases. By default a socket would be in the default
> > VRF.  Any application that would need to take advantage of VFRs would have to
> > be modified (alternativly by hacking libc to recognize a environment varible
> > no application would HAVE to be changed)
> >
> > > 2. separate, virtual netfilters, rt, rules, with extensions to break out
> > > of the virtual spaces for particular routing magic.  Process subsystem
> > > would be modified to set it into a default VRF context (a'la chroot).
> > > Special mechanisms should be included so VRF-aware processes can change
> > > VRF membership for particular sockets.
> >
> > If I understand correctly you want to make sure that existing rules and
> > netwfilters continue to work, but have the notion of which VFR they belong to
> > so that a VRF A cannot create a rule which forwards traffic to a table owned
> > by VRF B.
> >
> > > 3. virtual linux "machines".  I know there is a project to virtualize the
> > > whole linux system running atop linux (name?).  The issue them would be
> > > how to interconnect the routing, how to manage the system as a while, and
> > > how to handle PVCs (Frame, ATM) and VLANs.
> >
> > user-mode-linux.  It works pretty good, I use it for a lot of my MPLS testing.
> > They already have a scheme to network between the "UMLs".  This is a very
> > interesting point.  We need to have the ability to route traffic between
> > VRFs as well.  In particular the "Internet VRF" (the VRF that knows how
> > to get the real world (non VPN), needs to be reachbale via other VPNs.  Can
> > a rule be created to handle this?
> >
> > So I think we've agreed (maybe implicitly) on the following:
> > -sockets need to be VRF aware
> > -interfaces need to be VRF aware
> > -exiting functionality should not be limited just becuase a VRF is being used
> >
> > Jim
> >
> 
> 
> _______________________________________________
> linux-vrf-core mailing list
> lin...@li...
> https://lists.sourceforge.net/lists/listinfo/linux-vrf-core

-- 
James R. Leu

Re: [linux-vrf-core] VRF for Linux (MPLS IP VPNs)

[Nick E. <ni...@dc...>]

On Thu, 1 Nov 2001, James R. Leu wrote:

> On Thu, Nov 01, 2001 at 11:23:05AM -0500, Nick Eggleston wrote:
> > Regarding UML:
> >
> > I have been trying to set it up for testing (using VLANs as the private
> > interfaces).  What I was thinking is that on the host, I would bridge the
> > vlan interface and the tap interface, and make sure that IP was not
> > running on either of them.  However, the UML code for tuntap and ethertap
> > seems to insist on having an IP address to hand the host.  I need the host
> > tap interface to have NO ip running on it at all.  Can you offer any
> > advice?
> >
>
> You should beable to simply add a:
>
> eth0=tuntap,tap1
>
> or
>
> eth0=ethertap,tap1
>
> to your uml command line and it should work.  You'll probably get a couple
> of error messages, but it should still work.  I have modified the tuntap
> version to not give me the error messages.  I then use 'brctl' (i'm on a 2.4.13
> host) to bridge multiple tapXs or bridgex ethX and tapX.

Okay, I'll try it again.  When I do it, it complains and then just hangs.
Do you have any advice on troubleshooting this?

>
>
> > Regarding native VRF:
> >

<snip>

> > This doesn't even address issues like having separate hosts and other
> > user-level thingies that one would like to be different on a per-vrf
> > basis.
>
> ... I don;t follow this last statement.  Maybe an example would help me
> understand ...
>

For example, you have two different VRFs.  Each one has a machine with ip
10.0.0.1.  You would like to have the hosts file or DNS lookups be able to
resolve this address within the respective VRF namespaces.  Think about
having a telnet session running in each of the VRFs trying to telnet to
machines that have the same IP address but are in completely different
address spaces.

Maybe CHROOT can take care of this, in some small way.  Or maybe we can
get copy-on-write or stackable filesystems working in the native kernel.


--Nick

> > This seems like lots of work!
>
> Heh heh, yep, nobody said it would be easy :-)  But I think it is doable.
>
> Jim
>
> > --Nick
> >
> > On Wed, 31 Oct 2001,
> > James R. Leu wrote:
> >
> > > On Thu, Oct 25, 2001 at 08:51:44AM -0500, Nick Eggleston wrote:
> > > > I've been doing more thinking about this.
> > > >
> > > > I think the first thing we should tackle is the UML approach.  It provides
> > > > a while virtual router interface, and shoud be fairly easy to set up.  I
> > > > am not too familiar with it, so I will definately need help
> > > > interconnecting the UMLs for networking with VLANs (and MPLS, for you).
> > >
> > > This might be a good idea to work on so we can understand how a "virtual"
> > > should work.  But for a long term solution it might not be a good idea
> > > because the network performance on a UML is not the greatest.
> > >
> > > I use UML quite extensivly to provide a test best for my MPLS work.  I start
> > > 3 UMLs up, network them together, then run my MPLS regression test on them.
> > >
> > > > The other design will involve changes to the routing, user/process socket
> > > > layers to support VRF separation in the kernel, which is going to take a
> > > > while to implement.  I still think it would be useful to implement such a
> > > > system, but let's get things working with UML, first.
> > >
> > > As far as the changing the routing/socket/process idea goes.  THis can be
> > > broken into four parts:
> > >
> > > -transit processing
> > > -locally terminating traffic
> > > -locally originating traffic
> > > -populating VRF tables/rules
> > >
> > > Transit processing it turns out will be pretty simple.  Right now all IP
> > > routing goes through fib_lookup.  fib_lookup looks in it's rules list.
> > > There exists default rules which say look in the default routing table
> > > and if it's not there drop it on the floor.  Other rules that would show
> > > up in the ruls list would be NAT, policy routing, etc.  If we create an
> > > array or rules lists, indexed by the VRF number.  Then each route lookup
> > > just needs to know which VRF (or rules list) it needs to look in.
> > > For transit processing the VRF number is stored on the interface
> > > on which the packet arrived.  I may be over simplifing this a bit,
> > > but it looks promising. (this requires an IOCTL to assign a VRF number
> > > to an interface)
> > >
> > > Locally originating traffic will be very similar as above, execpt the VRF
> > > number will come from the socket the packet was generated on.
> > > (this requires an IOCTL to assign a socket to an interface)
> > >
> > > Locally terminating traffic might be a bit more work.  When a scoket is assigned
> > > to a VRF it needs to be added to a list of sockets associated with the VRF.
> > > When ip_local_route runs it needs to look in the VRFs list of sockets.
> > > I'm not sure how to make this work yet.
> > >
> > > Populating VRF table rules will require extending the 'ip' utility
> > > and/or the ifconfig/route utilities.  The kernel side will just grab the
> > > appropriate table/rules before doing any actions.
> > >
> > > > What do you think?
> > >
> > > I think the UML idea is a good sand box to figure out how this really should
> > > work.  Ideally the results of this project should be that a single linux
> > > box will behave like multiple linux boxes (from the routing and forwarding
> > > standpoint).
> > >
> > > Jim
> > >
> > > > --Nick
> > > >
> > > > On Thu, 11 Oct 2001, James R. Leu wrote:
> > > >
> > > > > On Wed, Oct 10, 2001 at 09:35:47AM -0500, Nick Eggleston wrote:
> > > > > <snip>
> > > > > > Please read the attached references.  Here are some very sketchy ideas on
> > > > > > potential implemenations approaches for Linux:
> > > > > >
> > > > > >
> > > > > > 1. minimal changes - keep existing rules and tables, use fwmarks to
> > > > > > distinguish traffic internally.  This CAN work for non-local traffic,
> > > > > > and it is even possible with the OWNER netfilter target to set the
> > > > > > fwmark for localally GENERATED traffic.  However, how to handle
> > > > > > local reception is not understood.  I particular, how RIP, OSPF, BGP
> > > > > > and the user process subsystem need to be modified is incomplete, and
> > > > > > these would need to be modified to push their routes into specific tables.
> > > > >
> > > > > Interesting.  I hadn't thought about using fwmark.  I will look around
> > > > > about how that would work.
> > > > >
> > > > > > How do we fix up local subsystem to mark sockets (so gated, zebra, routed)
> > > > > > work right.  What about ping, traceroute, telnet, etc?  /etc/hosts, dns?
> > > > >
> > > > > I think if we provide a means for a socket to say which VRF it wishes
> > > > > to use we cover all of the cases. By default a socket would be in the default
> > > > > VRF.  Any application that would need to take advantage of VFRs would have to
> > > > > be modified (alternativly by hacking libc to recognize a environment varible
> > > > > no application would HAVE to be changed)
> > > > >
> > > > > > 2. separate, virtual netfilters, rt, rules, with extensions to break out
> > > > > > of the virtual spaces for particular routing magic.  Process subsystem
> > > > > > would be modified to set it into a default VRF context (a'la chroot).
> > > > > > Special mechanisms should be included so VRF-aware processes can change
> > > > > > VRF membership for particular sockets.
> > > > >
> > > > > If I understand correctly you want to make sure that existing rules and
> > > > > netwfilters continue to work, but have the notion of which VFR they belong to
> > > > > so that a VRF A cannot create a rule which forwards traffic to a table owned
> > > > > by VRF B.
> > > > >
> > > > > > 3. virtual linux "machines".  I know there is a project to virtualize the
> > > > > > whole linux system running atop linux (name?).  The issue them would be
> > > > > > how to interconnect the routing, how to manage the system as a while, and
> > > > > > how to handle PVCs (Frame, ATM) and VLANs.
> > > > >
> > > > > user-mode-linux.  It works pretty good, I use it for a lot of my MPLS testing.
> > > > > They already have a scheme to network between the "UMLs".  This is a very
> > > > > interesting point.  We need to have the ability to route traffic between
> > > > > VRFs as well.  In particular the "Internet VRF" (the VRF that knows how
> > > > > to get the real world (non VPN), needs to be reachbale via other VPNs.  Can
> > > > > a rule be created to handle this?
> > > > >
> > > > > So I think we've agreed (maybe implicitly) on the following:
> > > > > -sockets need to be VRF aware
> > > > > -interfaces need to be VRF aware
> > > > > -exiting functionality should not be limited just becuase a VRF is being used
> > > > >
> > > > > Jim
> > > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > linux-vrf-core mailing list
> > > > lin...@li...
> > > > https://lists.sourceforge.net/lists/listinfo/linux-vrf-core
> > >
> > >
> >
> >
> > _______________________________________________
> > linux-vrf-core mailing list
> > lin...@li...
> > https://lists.sourceforge.net/lists/listinfo/linux-vrf-core
>
>

Re: [linux-vrf-core] VRF for Linux (MPLS IP VPNs)

[James R. L. <jl...@mi...>]

On Thu, Nov 01, 2001 at 11:23:05AM -0500, Nick Eggleston wrote:
> Regarding UML:
> 
> I have been trying to set it up for testing (using VLANs as the private
> interfaces).  What I was thinking is that on the host, I would bridge the
> vlan interface and the tap interface, and make sure that IP was not
> running on either of them.  However, the UML code for tuntap and ethertap
> seems to insist on having an IP address to hand the host.  I need the host
> tap interface to have NO ip running on it at all.  Can you offer any
> advice?
> 

You should beable to simply add a:

eth0=tuntap,tap1

or

eth0=ethertap,tap1

to your uml command line and it should work.  You'll probably get a couple
of error messages, but it should still work.  I have modified the tuntap
version to not give me the error messages.  I then use 'brctl' (i'm on a 2.4.13
host) to bridge multiple tapXs or bridgex ethX and tapX.


> Regarding native VRF:
> 
> My thought is that we need to add a vrf identifier (like the route
> distinguisher id) to interfaces, sockets, skbuffs, processes, routing and
> arp tables.
> 
> The model I have is that processes would all be assigned to a VRF.  It
> would be 0 by default, but could be changed.  Then, all sockets that the
> process created would inherit this VRF identifier.  If the program were
> VRF-aware, and the user had the right credentials, then it could change
> the VRF tag for any given socket.
> 
> Interfaces (devices) would also have a VRF identifier associated with
> them.  Incoming packets would get tagged with the VRF, which would have to
> follow them through the various kernel subsystems.  Possibly, the
> netfilter code could check/modify these identifiers for routing magic.
> 
> The routing/forwarding tables, too should be split to support the vrf
> identifier.  Thay way, each VRF could have the same functionalty in terms
> of the use of rules and routing tables, etc.  Possibly the netfilter
> subsystem should be virtualized as well.
> 
> Basically, all of the internal comparison/lookup routings would have to be
> extended to match the vrf id before their current algorighms run.

I follow everthing so far ....

> This doesn't even address issues like having separate hosts and other
> user-level thingies that one would like to be different on a per-vrf
> basis.

... I don;t follow this last statement.  Maybe an example would help me
understand ...

> This seems like lots of work!

Heh heh, yep, nobody said it would be easy :-)  But I think it is doable.

Jim

> --Nick
> 
> On Wed, 31 Oct 2001,
> James R. Leu wrote:
> 
> > On Thu, Oct 25, 2001 at 08:51:44AM -0500, Nick Eggleston wrote:
> > > I've been doing more thinking about this.
> > >
> > > I think the first thing we should tackle is the UML approach.  It provides
> > > a while virtual router interface, and shoud be fairly easy to set up.  I
> > > am not too familiar with it, so I will definately need help
> > > interconnecting the UMLs for networking with VLANs (and MPLS, for you).
> >
> > This might be a good idea to work on so we can understand how a "virtual"
> > should work.  But for a long term solution it might not be a good idea
> > because the network performance on a UML is not the greatest.
> >
> > I use UML quite extensivly to provide a test best for my MPLS work.  I start
> > 3 UMLs up, network them together, then run my MPLS regression test on them.
> >
> > > The other design will involve changes to the routing, user/process socket
> > > layers to support VRF separation in the kernel, which is going to take a
> > > while to implement.  I still think it would be useful to implement such a
> > > system, but let's get things working with UML, first.
> >
> > As far as the changing the routing/socket/process idea goes.  THis can be
> > broken into four parts:
> >
> > -transit processing
> > -locally terminating traffic
> > -locally originating traffic
> > -populating VRF tables/rules
> >
> > Transit processing it turns out will be pretty simple.  Right now all IP
> > routing goes through fib_lookup.  fib_lookup looks in it's rules list.
> > There exists default rules which say look in the default routing table
> > and if it's not there drop it on the floor.  Other rules that would show
> > up in the ruls list would be NAT, policy routing, etc.  If we create an
> > array or rules lists, indexed by the VRF number.  Then each route lookup
> > just needs to know which VRF (or rules list) it needs to look in.
> > For transit processing the VRF number is stored on the interface
> > on which the packet arrived.  I may be over simplifing this a bit,
> > but it looks promising. (this requires an IOCTL to assign a VRF number
> > to an interface)
> >
> > Locally originating traffic will be very similar as above, execpt the VRF
> > number will come from the socket the packet was generated on.
> > (this requires an IOCTL to assign a socket to an interface)
> >
> > Locally terminating traffic might be a bit more work.  When a scoket is assigned
> > to a VRF it needs to be added to a list of sockets associated with the VRF.
> > When ip_local_route runs it needs to look in the VRFs list of sockets.
> > I'm not sure how to make this work yet.
> >
> > Populating VRF table rules will require extending the 'ip' utility
> > and/or the ifconfig/route utilities.  The kernel side will just grab the
> > appropriate table/rules before doing any actions.
> >
> > > What do you think?
> >
> > I think the UML idea is a good sand box to figure out how this really should
> > work.  Ideally the results of this project should be that a single linux
> > box will behave like multiple linux boxes (from the routing and forwarding
> > standpoint).
> >
> > Jim
> >
> > > --Nick
> > >
> > > On Thu, 11 Oct 2001, James R. Leu wrote:
> > >
> > > > On Wed, Oct 10, 2001 at 09:35:47AM -0500, Nick Eggleston wrote:
> > > > <snip>
> > > > > Please read the attached references.  Here are some very sketchy ideas on
> > > > > potential implemenations approaches for Linux:
> > > > >
> > > > >
> > > > > 1. minimal changes - keep existing rules and tables, use fwmarks to
> > > > > distinguish traffic internally.  This CAN work for non-local traffic,
> > > > > and it is even possible with the OWNER netfilter target to set the
> > > > > fwmark for localally GENERATED traffic.  However, how to handle
> > > > > local reception is not understood.  I particular, how RIP, OSPF, BGP
> > > > > and the user process subsystem need to be modified is incomplete, and
> > > > > these would need to be modified to push their routes into specific tables.
> > > >
> > > > Interesting.  I hadn't thought about using fwmark.  I will look around
> > > > about how that would work.
> > > >
> > > > > How do we fix up local subsystem to mark sockets (so gated, zebra, routed)
> > > > > work right.  What about ping, traceroute, telnet, etc?  /etc/hosts, dns?
> > > >
> > > > I think if we provide a means for a socket to say which VRF it wishes
> > > > to use we cover all of the cases. By default a socket would be in the default
> > > > VRF.  Any application that would need to take advantage of VFRs would have to
> > > > be modified (alternativly by hacking libc to recognize a environment varible
> > > > no application would HAVE to be changed)
> > > >
> > > > > 2. separate, virtual netfilters, rt, rules, with extensions to break out
> > > > > of the virtual spaces for particular routing magic.  Process subsystem
> > > > > would be modified to set it into a default VRF context (a'la chroot).
> > > > > Special mechanisms should be included so VRF-aware processes can change
> > > > > VRF membership for particular sockets.
> > > >
> > > > If I understand correctly you want to make sure that existing rules and
> > > > netwfilters continue to work, but have the notion of which VFR they belong to
> > > > so that a VRF A cannot create a rule which forwards traffic to a table owned
> > > > by VRF B.
> > > >
> > > > > 3. virtual linux "machines".  I know there is a project to virtualize the
> > > > > whole linux system running atop linux (name?).  The issue them would be
> > > > > how to interconnect the routing, how to manage the system as a while, and
> > > > > how to handle PVCs (Frame, ATM) and VLANs.
> > > >
> > > > user-mode-linux.  It works pretty good, I use it for a lot of my MPLS testing.
> > > > They already have a scheme to network between the "UMLs".  This is a very
> > > > interesting point.  We need to have the ability to route traffic between
> > > > VRFs as well.  In particular the "Internet VRF" (the VRF that knows how
> > > > to get the real world (non VPN), needs to be reachbale via other VPNs.  Can
> > > > a rule be created to handle this?
> > > >
> > > > So I think we've agreed (maybe implicitly) on the following:
> > > > -sockets need to be VRF aware
> > > > -interfaces need to be VRF aware
> > > > -exiting functionality should not be limited just becuase a VRF is being used
> > > >
> > > > Jim
> > > >
> > >
> > >
> > > _______________________________________________
> > > linux-vrf-core mailing list
> > > lin...@li...
> > > https://lists.sourceforge.net/lists/listinfo/linux-vrf-core
> >
> >
> 
> 
> _______________________________________________
> linux-vrf-core mailing list
> lin...@li...
> https://lists.sourceforge.net/lists/listinfo/linux-vrf-core

-- 
James R. Leu

Re: [linux-vrf-core] VRF for Linux (MPLS IP VPNs)

[Nick E. <ni...@dc...>]

Regarding UML:

I have been trying to set it up for testing (using VLANs as the private
interfaces).  What I was thinking is that on the host, I would bridge the
vlan interface and the tap interface, and make sure that IP was not
running on either of them.  However, the UML code for tuntap and ethertap
seems to insist on having an IP address to hand the host.  I need the host
tap interface to have NO ip running on it at all.  Can you offer any
advice?

Regarding native VRF:

My thought is that we need to add a vrf identifier (like the route
distinguisher id) to interfaces, sockets, skbuffs, processes, routing and
arp tables.

The model I have is that processes would all be assigned to a VRF.  It
would be 0 by default, but could be changed.  Then, all sockets that the
process created would inherit this VRF identifier.  If the program were
VRF-aware, and the user had the right credentials, then it could change
the VRF tag for any given socket.

Interfaces (devices) would also have a VRF identifier associated with
them.  Incoming packets would get tagged with the VRF, which would have to
follow them through the various kernel subsystems.  Possibly, the
netfilter code could check/modify these identifiers for routing magic.

The routing/forwarding tables, too should be split to support the vrf
identifier.  Thay way, each VRF could have the same functionalty in terms
of the use of rules and routing tables, etc.  Possibly the netfilter
subsystem should be virtualized as well.

Basically, all of the internal comparison/lookup routings would have to be
extended to match the vrf id before their current algorighms run.

This doesn't even address issues like having separate hosts and other
user-level thingies that one would like to be different on a per-vrf
basis.

This seems like lots of work!

--Nick

On Wed, 31 Oct 2001,
James R. Leu wrote:

> On Thu, Oct 25, 2001 at 08:51:44AM -0500, Nick Eggleston wrote:
> > I've been doing more thinking about this.
> >
> > I think the first thing we should tackle is the UML approach.  It provides
> > a while virtual router interface, and shoud be fairly easy to set up.  I
> > am not too familiar with it, so I will definately need help
> > interconnecting the UMLs for networking with VLANs (and MPLS, for you).
>
> This might be a good idea to work on so we can understand how a "virtual"
> should work.  But for a long term solution it might not be a good idea
> because the network performance on a UML is not the greatest.
>
> I use UML quite extensivly to provide a test best for my MPLS work.  I start
> 3 UMLs up, network them together, then run my MPLS regression test on them.
>
> > The other design will involve changes to the routing, user/process socket
> > layers to support VRF separation in the kernel, which is going to take a
> > while to implement.  I still think it would be useful to implement such a
> > system, but let's get things working with UML, first.
>
> As far as the changing the routing/socket/process idea goes.  THis can be
> broken into four parts:
>
> -transit processing
> -locally terminating traffic
> -locally originating traffic
> -populating VRF tables/rules
>
> Transit processing it turns out will be pretty simple.  Right now all IP
> routing goes through fib_lookup.  fib_lookup looks in it's rules list.
> There exists default rules which say look in the default routing table
> and if it's not there drop it on the floor.  Other rules that would show
> up in the ruls list would be NAT, policy routing, etc.  If we create an
> array or rules lists, indexed by the VRF number.  Then each route lookup
> just needs to know which VRF (or rules list) it needs to look in.
> For transit processing the VRF number is stored on the interface
> on which the packet arrived.  I may be over simplifing this a bit,
> but it looks promising. (this requires an IOCTL to assign a VRF number
> to an interface)
>
> Locally originating traffic will be very similar as above, execpt the VRF
> number will come from the socket the packet was generated on.
> (this requires an IOCTL to assign a socket to an interface)
>
> Locally terminating traffic might be a bit more work.  When a scoket is assigned
> to a VRF it needs to be added to a list of sockets associated with the VRF.
> When ip_local_route runs it needs to look in the VRFs list of sockets.
> I'm not sure how to make this work yet.
>
> Populating VRF table rules will require extending the 'ip' utility
> and/or the ifconfig/route utilities.  The kernel side will just grab the
> appropriate table/rules before doing any actions.
>
> > What do you think?
>
> I think the UML idea is a good sand box to figure out how this really should
> work.  Ideally the results of this project should be that a single linux
> box will behave like multiple linux boxes (from the routing and forwarding
> standpoint).
>
> Jim
>
> > --Nick
> >
> > On Thu, 11 Oct 2001, James R. Leu wrote:
> >
> > > On Wed, Oct 10, 2001 at 09:35:47AM -0500, Nick Eggleston wrote:
> > > <snip>
> > > > Please read the attached references.  Here are some very sketchy ideas on
> > > > potential implemenations approaches for Linux:
> > > >
> > > >
> > > > 1. minimal changes - keep existing rules and tables, use fwmarks to
> > > > distinguish traffic internally.  This CAN work for non-local traffic,
> > > > and it is even possible with the OWNER netfilter target to set the
> > > > fwmark for localally GENERATED traffic.  However, how to handle
> > > > local reception is not understood.  I particular, how RIP, OSPF, BGP
> > > > and the user process subsystem need to be modified is incomplete, and
> > > > these would need to be modified to push their routes into specific tables.
> > >
> > > Interesting.  I hadn't thought about using fwmark.  I will look around
> > > about how that would work.
> > >
> > > > How do we fix up local subsystem to mark sockets (so gated, zebra, routed)
> > > > work right.  What about ping, traceroute, telnet, etc?  /etc/hosts, dns?
> > >
> > > I think if we provide a means for a socket to say which VRF it wishes
> > > to use we cover all of the cases. By default a socket would be in the default
> > > VRF.  Any application that would need to take advantage of VFRs would have to
> > > be modified (alternativly by hacking libc to recognize a environment varible
> > > no application would HAVE to be changed)
> > >
> > > > 2. separate, virtual netfilters, rt, rules, with extensions to break out
> > > > of the virtual spaces for particular routing magic.  Process subsystem
> > > > would be modified to set it into a default VRF context (a'la chroot).
> > > > Special mechanisms should be included so VRF-aware processes can change
> > > > VRF membership for particular sockets.
> > >
> > > If I understand correctly you want to make sure that existing rules and
> > > netwfilters continue to work, but have the notion of which VFR they belong to
> > > so that a VRF A cannot create a rule which forwards traffic to a table owned
> > > by VRF B.
> > >
> > > > 3. virtual linux "machines".  I know there is a project to virtualize the
> > > > whole linux system running atop linux (name?).  The issue them would be
> > > > how to interconnect the routing, how to manage the system as a while, and
> > > > how to handle PVCs (Frame, ATM) and VLANs.
> > >
> > > user-mode-linux.  It works pretty good, I use it for a lot of my MPLS testing.
> > > They already have a scheme to network between the "UMLs".  This is a very
> > > interesting point.  We need to have the ability to route traffic between
> > > VRFs as well.  In particular the "Internet VRF" (the VRF that knows how
> > > to get the real world (non VPN), needs to be reachbale via other VPNs.  Can
> > > a rule be created to handle this?
> > >
> > > So I think we've agreed (maybe implicitly) on the following:
> > > -sockets need to be VRF aware
> > > -interfaces need to be VRF aware
> > > -exiting functionality should not be limited just becuase a VRF is being used
> > >
> > > Jim
> > >
> >
> >
> > _______________________________________________
> > linux-vrf-core mailing list
> > lin...@li...
> > https://lists.sourceforge.net/lists/listinfo/linux-vrf-core
>
>

Re: [linux-vrf-core] VRF for Linux (MPLS IP VPNs)

[James R. L. <jl...@mi...>]

On Thu, Oct 25, 2001 at 08:51:44AM -0500, Nick Eggleston wrote:
> I've been doing more thinking about this.
> 
> I think the first thing we should tackle is the UML approach.  It provides
> a while virtual router interface, and shoud be fairly easy to set up.  I
> am not too familiar with it, so I will definately need help
> interconnecting the UMLs for networking with VLANs (and MPLS, for you).

This might be a good idea to work on so we can understand how a "virtual"
should work.  But for a long term solution it might not be a good idea
because the network performance on a UML is not the greatest.

I use UML quite extensivly to provide a test best for my MPLS work.  I start
3 UMLs up, network them together, then run my MPLS regression test on them.

> The other design will involve changes to the routing, user/process socket
> layers to support VRF separation in the kernel, which is going to take a
> while to implement.  I still think it would be useful to implement such a
> system, but let's get things working with UML, first.

As far as the changing the routing/socket/process idea goes.  THis can be
broken into four parts:

-transit processing
-locally terminating traffic
-locally originating traffic
-populating VRF tables/rules

Transit processing it turns out will be pretty simple.  Right now all IP
routing goes through fib_lookup.  fib_lookup looks in it's rules list.
There exists default rules which say look in the default routing table
and if it's not there drop it on the floor.  Other rules that would show
up in the ruls list would be NAT, policy routing, etc.  If we create an 
array or rules lists, indexed by the VRF number.  Then each route lookup
just needs to know which VRF (or rules list) it needs to look in.
For transit processing the VRF number is stored on the interface
on which the packet arrived.  I may be over simplifing this a bit,
but it looks promising. (this requires an IOCTL to assign a VRF number
to an interface)

Locally originating traffic will be very similar as above, execpt the VRF
number will come from the socket the packet was generated on.
(this requires an IOCTL to assign a socket to an interface)

Locally terminating traffic might be a bit more work.  When a scoket is assigned
to a VRF it needs to be added to a list of sockets associated with the VRF.
When ip_local_route runs it needs to look in the VRFs list of sockets.
I'm not sure how to make this work yet.

Populating VRF table rules will require extending the 'ip' utility
and/or the ifconfig/route utilities.  The kernel side will just grab the
appropriate table/rules before doing any actions.

> What do you think?

I think the UML idea is a good sand box to figure out how this really should
work.  Ideally the results of this project should be that a single linux
box will behave like multiple linux boxes (from the routing and forwarding
standpoint).

Jim

> --Nick
> 
> On Thu, 11 Oct 2001, James R. Leu wrote:
> 
> > On Wed, Oct 10, 2001 at 09:35:47AM -0500, Nick Eggleston wrote:
> > <snip>
> > > Please read the attached references.  Here are some very sketchy ideas on
> > > potential implemenations approaches for Linux:
> > >
> > >
> > > 1. minimal changes - keep existing rules and tables, use fwmarks to
> > > distinguish traffic internally.  This CAN work for non-local traffic,
> > > and it is even possible with the OWNER netfilter target to set the
> > > fwmark for localally GENERATED traffic.  However, how to handle
> > > local reception is not understood.  I particular, how RIP, OSPF, BGP
> > > and the user process subsystem need to be modified is incomplete, and
> > > these would need to be modified to push their routes into specific tables.
> >
> > Interesting.  I hadn't thought about using fwmark.  I will look around
> > about how that would work.
> >
> > > How do we fix up local subsystem to mark sockets (so gated, zebra, routed)
> > > work right.  What about ping, traceroute, telnet, etc?  /etc/hosts, dns?
> >
> > I think if we provide a means for a socket to say which VRF it wishes
> > to use we cover all of the cases. By default a socket would be in the default
> > VRF.  Any application that would need to take advantage of VFRs would have to
> > be modified (alternativly by hacking libc to recognize a environment varible
> > no application would HAVE to be changed)
> >
> > > 2. separate, virtual netfilters, rt, rules, with extensions to break out
> > > of the virtual spaces for particular routing magic.  Process subsystem
> > > would be modified to set it into a default VRF context (a'la chroot).
> > > Special mechanisms should be included so VRF-aware processes can change
> > > VRF membership for particular sockets.
> >
> > If I understand correctly you want to make sure that existing rules and
> > netwfilters continue to work, but have the notion of which VFR they belong to
> > so that a VRF A cannot create a rule which forwards traffic to a table owned
> > by VRF B.
> >
> > > 3. virtual linux "machines".  I know there is a project to virtualize the
> > > whole linux system running atop linux (name?).  The issue them would be
> > > how to interconnect the routing, how to manage the system as a while, and
> > > how to handle PVCs (Frame, ATM) and VLANs.
> >
> > user-mode-linux.  It works pretty good, I use it for a lot of my MPLS testing.
> > They already have a scheme to network between the "UMLs".  This is a very
> > interesting point.  We need to have the ability to route traffic between
> > VRFs as well.  In particular the "Internet VRF" (the VRF that knows how
> > to get the real world (non VPN), needs to be reachbale via other VPNs.  Can
> > a rule be created to handle this?
> >
> > So I think we've agreed (maybe implicitly) on the following:
> > -sockets need to be VRF aware
> > -interfaces need to be VRF aware
> > -exiting functionality should not be limited just becuase a VRF is being used
> >
> > Jim
> >
> 
> 
> _______________________________________________
> linux-vrf-core mailing list
> lin...@li...
> https://lists.sourceforge.net/lists/listinfo/linux-vrf-core

-- 
James R. Leu

Re: [linux-vrf-core] VRF for Linux (MPLS IP VPNs)

[Nick E. <ni...@dc...>]

I've been doing more thinking about this.

I think the first thing we should tackle is the UML approach.  It provides
a while virtual router interface, and shoud be fairly easy to set up.  I
am not too familiar with it, so I will definately need help
interconnecting the UMLs for networking with VLANs (and MPLS, for you).

The other design will involve changes to the routing, user/process socket
layers to support VRF separation in the kernel, which is going to take a
while to implement.  I still think it would be useful to implement such a
system, but let's get things working with UML, first.

What do you think?

--Nick

On Thu, 11 Oct 2001, James R. Leu wrote:

> On Wed, Oct 10, 2001 at 09:35:47AM -0500, Nick Eggleston wrote:
> <snip>
> > Please read the attached references.  Here are some very sketchy ideas on
> > potential implemenations approaches for Linux:
> >
> >
> > 1. minimal changes - keep existing rules and tables, use fwmarks to
> > distinguish traffic internally.  This CAN work for non-local traffic,
> > and it is even possible with the OWNER netfilter target to set the
> > fwmark for localally GENERATED traffic.  However, how to handle
> > local reception is not understood.  I particular, how RIP, OSPF, BGP
> > and the user process subsystem need to be modified is incomplete, and
> > these would need to be modified to push their routes into specific tables.
>
> Interesting.  I hadn't thought about using fwmark.  I will look around
> about how that would work.
>
> > How do we fix up local subsystem to mark sockets (so gated, zebra, routed)
> > work right.  What about ping, traceroute, telnet, etc?  /etc/hosts, dns?
>
> I think if we provide a means for a socket to say which VRF it wishes
> to use we cover all of the cases. By default a socket would be in the default
> VRF.  Any application that would need to take advantage of VFRs would have to
> be modified (alternativly by hacking libc to recognize a environment varible
> no application would HAVE to be changed)
>
> > 2. separate, virtual netfilters, rt, rules, with extensions to break out
> > of the virtual spaces for particular routing magic.  Process subsystem
> > would be modified to set it into a default VRF context (a'la chroot).
> > Special mechanisms should be included so VRF-aware processes can change
> > VRF membership for particular sockets.
>
> If I understand correctly you want to make sure that existing rules and
> netwfilters continue to work, but have the notion of which VFR they belong to
> so that a VRF A cannot create a rule which forwards traffic to a table owned
> by VRF B.
>
> > 3. virtual linux "machines".  I know there is a project to virtualize the
> > whole linux system running atop linux (name?).  The issue them would be
> > how to interconnect the routing, how to manage the system as a while, and
> > how to handle PVCs (Frame, ATM) and VLANs.
>
> user-mode-linux.  It works pretty good, I use it for a lot of my MPLS testing.
> They already have a scheme to network between the "UMLs".  This is a very
> interesting point.  We need to have the ability to route traffic between
> VRFs as well.  In particular the "Internet VRF" (the VRF that knows how
> to get the real world (non VPN), needs to be reachbale via other VPNs.  Can
> a rule be created to handle this?
>
> So I think we've agreed (maybe implicitly) on the following:
> -sockets need to be VRF aware
> -interfaces need to be VRF aware
> -exiting functionality should not be limited just becuase a VRF is being used
>
> Jim
>

Re: [linux-vrf-core] VRF for Linux (MPLS IP VPNs)

[James R. L. <jl...@mi...>]

On Wed, Oct 10, 2001 at 09:35:47AM -0500, Nick Eggleston wrote:
<snip>
> Please read the attached references.  Here are some very sketchy ideas on
> potential implemenations approaches for Linux:
> 
> 
> 1. minimal changes - keep existing rules and tables, use fwmarks to
> distinguish traffic internally.  This CAN work for non-local traffic,
> and it is even possible with the OWNER netfilter target to set the
> fwmark for localally GENERATED traffic.  However, how to handle
> local reception is not understood.  I particular, how RIP, OSPF, BGP
> and the user process subsystem need to be modified is incomplete, and
> these would need to be modified to push their routes into specific tables.

Interesting.  I hadn't thought about using fwmark.  I will look around
about how that would work.

> How do we fix up local subsystem to mark sockets (so gated, zebra, routed)
> work right.  What about ping, traceroute, telnet, etc?  /etc/hosts, dns?

I think if we provide a means for a socket to say which VRF it wishes
to use we cover all of the cases. By default a socket would be in the default
VRF.  Any application that would need to take advantage of VFRs would have to
be modified (alternativly by hacking libc to recognize a environment varible
no application would HAVE to be changed)

> 2. separate, virtual netfilters, rt, rules, with extensions to break out
> of the virtual spaces for particular routing magic.  Process subsystem
> would be modified to set it into a default VRF context (a'la chroot).
> Special mechanisms should be included so VRF-aware processes can change
> VRF membership for particular sockets.

If I understand correctly you want to make sure that existing rules and
netwfilters continue to work, but have the notion of which VFR they belong to
so that a VRF A cannot create a rule which forwards traffic to a table owned
by VRF B.

> 3. virtual linux "machines".  I know there is a project to virtualize the
> whole linux system running atop linux (name?).  The issue them would be
> how to interconnect the routing, how to manage the system as a while, and
> how to handle PVCs (Frame, ATM) and VLANs.

user-mode-linux.  It works pretty good, I use it for a lot of my MPLS testing.
They already have a scheme to network between the "UMLs".  This is a very
interesting point.  We need to have the ability to route traffic between
VRFs as well.  In particular the "Internet VRF" (the VRF that knows how
to get the real world (non VPN), needs to be reachbale via other VPNs.  Can
a rule be created to handle this?

So I think we've agreed (maybe implicitly) on the following:
-sockets need to be VRF aware
-interfaces need to be VRF aware
-exiting functionality should not be limited just becuase a VRF is being used

Jim
-- 
James R. Leu

[linux-vrf-core] VRF for Linux (MPLS IP VPNs)

[Nick E. <ni...@dc...>]

Welcome to VRF for Linux!

Please read this document to familiarize yourself with what we are trying
to accomplish.

[Initial blathering... (revised)]

VRF - Virtual (or VPN) Routing/Forwarding instances are at the heart of
aggregating multiple customers onto a single machine, while
keeping their address spaces separate.  VRFs allow two (or
more) customers to have identical IP address spaces in use, while keeping
them entirely separate withing the router.  This is accomplished by
maintaining separate L2/L3 tables for each customer (VPN).

RFC 2547 describes the requirements for these MPLS VPNs in more detail.
VRFs are requred in the Provider's Edge (PE) routers, which is what we are
trying to build for Linux.  The only issue not directly dealt with is how
local traffic (processes running on the router) interace with the VRFs.
(TODO: check this)

Please read the attached references.  Here are some very sketchy ideas on
potential implemenations approaches for Linux:


1. minimal changes - keep existing rules and tables, use fwmarks to
distinguish traffic internally.  This CAN work for non-local traffic,
and it is even possible with the OWNER netfilter target to set the
fwmark for localally GENERATED traffic.  However, how to handle
local reception is not understood.  I particular, how RIP, OSPF, BGP
and the user process subsystem need to be modified is incomplete, and
these would need to be modified to push their routes into specific tables.

How do we fix up local subsystem to mark sockets (so gated, zebra, routed)
work right.  What about ping, traceroute, telnet, etc?  /etc/hosts, dns?

2. separate, virtual netfilters, rt, rules, with extensions to break out
of the virtual spaces for particular routing magic.  Process subsystem
would be modified to set it into a default VRF context (a'la chroot).
Special mechanisms should be included so VRF-aware processes can change
VRF membership for particular sockets.

3. virtual linux "machines".  I know there is a project to virtualize the
whole linux system running atop linux (name?).  The issue them would be
how to interconnect the routing, how to manage the system as a while, and
how to handle PVCs (Frame, ATM) and VLANs.

This is just a start.  References below:

http://www.juniper.net/techcenter/techpapers/200012.html

This is a white paper by Juniper: RFC 2547bis: BGP/MPLS VPN Fundamentals
It is a good introduction and has nice illustrations which quickly convey
the essence of this concept.


http://www.juniper.net/techcenter/techpapers/200014.html

Another Junpier White Paper: RFC 2547bis: BGP/MPLS VPN Hierarchical and
Recursive Applications


http://www.cisco.com/univercd/cc/td/doc/product/software/ios120/120newft/120limit/120st/120st17/vpnid.htm

Cisco: Discusses MPLS VPN ID (RFC 2685), which is used in dialup (RADIUS)
and dhcp applications to identify to the servers to which VPN the request
belongs.


http://www.cisco.com/univercd/cc/td/doc/product/rtrmgmt/vpnsc/mpls/1_2/prov_gd/vpn_ug1.htm

Cisco: Indtroduction to MPLS VPN Technology
Includes Cisco definiton of VRFs.


http://www.mentortech.com/learn/welcher/papers/mplsvpn.html

Another Intro to MPLS VPNs (RFC 2547).  This one by MentorTech.  Short,
and clear.


http://community.roxen.com/developers/idocs/rfc/rfc2917.html

RFC 2917: another MPLS VPN RFC.  They take a different approach to some of
the VRF route-distributon issues, using Multicast to implement a virtual
LAN between all the VRFs belonging to a VPN.  Good reading.