[tipc-discussion] vTIPC-1.5.12 patch.

[Randy M. <mac...@no...>]

Hi,

Attached is a patch to tipc-1.5.12 that enables
multiple independent TIPC stacks to run on a single Linux OS.

This patch is referred to as virtualized TIPC or vTIPC.
The typical usage is an ATCA chassis that has separate
control and data networks (aka base and data in ATCA lingo).

Please take a look at the patch and the notes below and
let us know what you think of the design and implementation.

We've tested on:
ppc64, linux-2.6.10
ppc32, linux-2.6.14
x86,   linux-2.6.20-16-generic (Ubuntu)

There are some issues with 2.6.22 (Ubuntu 7.10) so
unfortunately testing on that system will require some work.

Signed-off-by: Randy MacLeod <mac...@no...>

Thanks,
   Ravi Rao      - TCS
   Randy MacLeod - Nortel



How this works
==============

We've logically separated the TIPC module into a CORE and a STACK.

The CORE is responsible for interfacing with the kernel -
registering AF_TIPC, netlink and handling the creation of new sockets.

The STACK is responsbile for the TIPC specific behaviour:
name_table, bearers, links, etc.

When you load TIPC you specify a role for the module.
Valid roles are:
  - TIPC_FULL  = 0 - default, usual tipc behaviour.
  - TIPC_CORE  = 1 - just register AF_TIPC and dispatch socket creation.
  - TIPC_STACK = 2 - register with CORE and handle all internals
                     of this particular stack_id.

When a STACK is loaded, it is also assigned a stack_id.
The STACK binds with the core to register callback functions for
  - socket creation: tipc_create and
  - netlink commands: cfg_do_cmd
The patch supports up to 8 STACKs but that can be easily changed.

To interact with each of these stacks, tipc-config has been modified
to accept an addtional parameter:
  -s=<stack_id>
The default stack id is zero.


 From a user process, you indicate which TIPC stack a socket is
associated with by specifying the "protocol" parameter in:

   int socket(AF_TIPC, int type, int protocol);

I suggest that: 0 = control network, 1 = data network.

A single process can bind sockets to any TIPC stack that is loaded.
The stacks are completely isolated so if you want to transfer a
message from the control network to the data network, you must
do so via userspace (or using the tipc kernel API).

Compiling
=========

  1. unpack a fresh tipc-1.5.12.tar.gz
  2. cd tipc-1.5.12
  3. patch -p1 < /tmp/tipc-1.5.12-vtipc-01.patch
  4. make - as you would normally.
  5. cp net/tipc/tipc.ko ..
    or cp tipc.ko ..   depending on how you build.
  6. cp tipcstack.ko ..
  7. cd ..

Basics of loading
=================

This is all backwards compatible so:
# insmod tipc.ko and
# tipc-config <any option>
# rmmod tipc
work as you would expect.

New behaviour requires one or two module parameters to be
specified at module load time:
# modinfo tipc.ko
...
parm:   tipc_role:Role of tipc:
        0 = full stack, 1 = core only, 2 = stack only (int)
parm:   tipc_stack_id:TIPC stack id: 0-7 permitted. (int)


# insmod tipc.ko tipc_role=1

would load tipc as TIPC_CORE. It will not initialize a TIPC stack
at all. You might want to do this to keep the core klm as simple as
possible.


# insmod tipcstack.ko tipc_role=2 tipc_stack_id=1

would load tipc (stack variant), initialize stack_id to 1, and
register it with the previously loaded tipc core module.

You can add up to 8 (0-7) separate stacks:
# insmod tipcstack.ko tipc_role=2 tipc_stack_id=0
# insmod tipcstack.ko tipc_role=2 tipc_stack_id=2
# insmod tipcstack.ko tipc_role=2 tipc_stack_id=3
# insmod tipcstack.ko tipc_role=2 tipc_stack_id=4
# insmod tipcstack.ko tipc_role=2 tipc_stack_id=5
# insmod tipcstack.ko tipc_role=2 tipc_stack_id=6
# insmod tipcstack.ko tipc_role=2 tipc_stack_id=7
# insmod tipcstack.ko tipc_role=2 tipc_stack_id=8 <-- error!

Testing
=======

1. sudo insmod /tipc.ko
2. sudo insmod tipcstack.ko tipc_role=2 tipc_stack_id=1
3. configure network:
   nodeA: sudo ./tipc-config -s=0 -addr=1.1.1 -be=eth:eth0
   nodeA: sudo ./tipc-config -s=1 -addr=1.1.1 -be=eth:eth1

   nodeB: sudo ./tipc-config -s=0 -addr=1.1.2 -be=eth:eth0
   nodeB: sudo ./tipc-config -s=1 -addr=1.1.2 -be=eth:eth1

Since the tipc stacks are completely independent, the Z.C.N
addresses don't have to be the same - you could have:

   nodeA: sudo ./tipc-config -s=0 -addr=1.1.1 -be=eth:eth0
   nodeA: sudo ./tipc-config -s=1 -addr=1.1.21 -be=eth:eth1

   nodeB: sudo ./tipc-config -s=0 -addr=1.1.2 -be=eth:eth0
   nodeB: sudo ./tipc-config -s=1 -addr=1.1.22 -be=eth:eth1

4. take a look at the links:
   sudo ./tools/tipc-config -l
   sudo ./tools/tipc-config -s=1 -l

-------------------

Limitations
===========

When you bind each TIPC stack to a bearer you should
ensure that separate LANs are used. Should we enforce this?

There is one issue marked with RKR (Ravi's initials)
that should be fixed - please read the comment and provide feedback.

We should figure out how to make this work with Kconfig.


Questions
=========

1. We'd like to rebase this onto a newer version of TIPC.
What version would you suggest Allan?

2. Are the copyright notices okay?
Shouldn't we add a GPL or dual notice like this:
-------------------------------------
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License version 2 as
  * published by the Free Software Foundation.
---------------------------------------
be included?


That's all.
// Randy and Ravi.