[mpls-linux-devel] [PATCH] Port MII management to kobject/sysfs

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Hi James/all,

This patch has been obtained from the current head as of the time of
writting (which, I think does not yet include yesterday's patch, you may
drop it, this one supersedes it). This is not a trivial patch, and I
wonder if you could also try it on your side. I think that it clarifies a
little MII management, and reuses the kobject api (details follow).

I'll work on the same thing for MOI objects. I plan to port MOI management
to kobject api, get rid of the global moi_list, and clean up flush/delete
of outgoing labels.

Comments Welcome!

Regards,
R.

Remarks:

* Destroy methods for af_mpls.c are still missing. I don't know if you
want to complete those or not, and how we are going to netlink (I have a
mpls_netlink.c somewhere, which I used to test basic netlink
functionality, let me know if you want to take a look at it).

Features
-------------------------

* Added 2004 (c) in files.

* Added EXPORT_SYMBOL for spec_nh. There are known issues (see below)

* mpls_init has more robust error checking.

* Bumped version to #define MPLS_LINUX_VERSION     0x01090002

* Changed
RADIX_TREE(mii_tree, GFP_ATOMIC)
RADIX_TREE(moi_tree, GFP_ATOMIC)

Otherwise, we were getting the sleeping while atomic bug (write_lock for
the Incoming Outgoing Radix Tree + allocating a node in the tree with
GFP_KERNEL which may sleep) e.g.:

Debug: sleeping function called from invalid context at mm/slab.c:1856
in_atomic():1, irqs_disabled():0
Call Trace:
[<c012974c>] __might_sleep+0xac/0xe0
[<c0160e9a>] kmem_cache_alloc+0x1da/0x1e0
[<c022d9a0>] radix_tree_node_alloc+0x20/0x60
[<c022db8b>] radix_tree_insert+0x9b/0xe0
[<c039a7f0>] mpls_insert_mii+0x30/0xc0
[<c039abdc>] mpls_add_in_label+0x17c/0x2d0
[<c039c93b>] mpls_ioctl+0x79b/0x9e0

This happens, for example in:

int mpls_insert_mii (unsigned int key, struct mpls_in_info *mii)
{
...
	write_lock(&mpls_mii_lock);
	retval = radix_tree_insert(&mii_tree,key,mii);
	write_unlock(&mpls_mii_lock);
	return retval;
}

* Added a new kobject/sysfs subsystem, "mpls", that appears under /sys.
This subsystem exports in read only two attributes fullname and version.
For now, it is redundant with proc. In the future, the MPLS procfs may
become obsolete and favor either a mplsfs or do everything in sysfs with
attributes.

gandalf sys# ls
block  bus  cdev  class  devices  firmware  mpls  power
gandalf sys# tree mpls/
mpls/
|-- fullname
|-- mpls_mii
`-- version

* Added a new kset "mpls_mii" to hold all mii objects (still pending for
moi management, tunnel and labelspace management). This means that all
struct mpls_in_info (MII/mii) objects are now full kobjects, and struct
members are exposed (readonly for now) as attributes.

IMPLICATIONS

- Field __refcnt has been removed. refcount is now based on kobjects.

- Field global_entry has been removed. We can now get all the MII objects
from the MPLS MII kset (using the semaphore). See mpls_in_info flush tree
or mpls_proc for details. There is no need to manage a parallel list (although this is done by the kobject API anyway).

- Global mii_list has been deleted.

- mpls_mii_kobject_release (struct kobject *kobj) has been added. This one
replaces __mpls_del_label.... This is the "very latest function", where we
free the object. See documentation/{filesystems/sysfs,kobject}.txt for
details.

- mpls_in_info_hold is something like:
{
 struct kobject *kobj = NULL;
 kobj = kobject_get(&mii->kobj);
 return container_of(kobj,struct mpls_in_info,kobj);
}

- The name of the kobject MII is either

kobject_set_name(&mii->kobj,"mii-gen-%u:%u-%u",
kobject_set_name(&mii->kobj,"mii-atm-%u:%u/%u-%u",
kobject_set_name(&mii->kobj,"mii-fr-%u:%u-%u",

that is labelspace:labelvalue-key (decimal)

Tests:
---------------------------------------

1a.- Add generic 30,31,32,33, 34, 35 Incoming labels to ILM, using
mplsadm.

1b.- Check with procfs
gandalf casellas# cat /proc/net/mpls/*
Debug: 1

Printing Info for MII
Key:0x0001e001 0/0/0
gen 30
MII 0 2
POP PEEK

Printing Info for MII
Key:0x0001f001 0/0/0
gen 31
MII 0 2
POP PEEK

Printing Info for MII
Key:0x00020001 0/0/0
gen 32
MII 0 2
POP PEEK

Printing Info for MII
Key:0x00021001 0/0/0
gen 33
MII 0 2
POP PEEK

Printing Info for MII
Key:0x00022001 0/0/0
gen 34
MII 0 2
POP PEEK

Printing Info for MII
Key:0x00023001 0/0/0
gen 35
MII 0 2
POP PEEK

gandalf sys# ls
block  bus  cdev  class  devices  firmware  mpls  power
gandalf casellas# tree /sys/mpls/
/sys/mpls/
|-- fullname
|-- mpls_mii
|   |-- mii-gen-0:31-126977
|   |   |-- age
|   |   |-- bytes
|   |   |-- drops
|   |   |-- key
|   |   |-- labelspace
|   |   |-- packets
|   |   `-- proto
|   |-- mii-gen-0:32-131073
|   |   |-- age
|   |   |-- bytes
|   |   |-- drops
|   |   |-- key
|   |   |-- labelspace
|   |   |-- packets
|   |   `-- proto
|   |-- mii-gen-0:33-135169
|   |   |-- age
|   |   |-- bytes
|   |   |-- drops
|   |   |-- key
|   |   |-- labelspace
|   |   |-- packets
|   |   `-- proto
|   `-- mii-gen-0:34-139265
|       |-- age
|       |-- bytes
|       |-- drops
|       |-- key
|       |-- labelspace
|       |-- packets
|       `-- proto
`-- version

 gandalf mii-gen-0:31-126977# cat *
 257622
 0
 0
 1f001
 0
 0
 8

Flush trees
gandalf mplsadm# ./mplsadm -F
MPLS DEBUG net/mpls/mpls_ioctls.c:39:mpls_ioctl: IOCTL 35858
MPLS DEBUG net/mpls/mpls_in_info.c:784:mpls_flush_in_tree: enter
MPLS DEBUG net/mpls/mpls_in_info.c:798:mpls_flush_in_tree: # mii-gen-0:31-126977
MPLS DEBUG net/mpls/mpls_in_info.c:621:mpls_mii_kobject_release: enter
MPLS DEBUG net/mpls/mpls_utils.c:482:mpls_instruction_clear: enter
MPLS DEBUG net/mpls/mpls_utils.c:491:mpls_instruction_clear: exit
MPLS DEBUG net/mpls/mpls_in_info.c:637:mpls_mii_kobject_release: exit
MPLS DEBUG net/mpls/mpls_in_info.c:798:mpls_flush_in_tree: # mii-gen-0:32-131073
MPLS DEBUG net/mpls/mpls_in_info.c:621:mpls_mii_kobject_release: enter
MPLS DEBUG net/mpls/mpls_utils.c:482:mpls_instruction_clear: enter
MPLS DEBUG net/mpls/mpls_utils.c:491:mpls_instruction_clear: exit
MPLS DEBUG net/mpls/mpls_in_info.c:637:mpls_mii_kobject_release: exit
MPLS DEBUG net/mpls/mpls_in_info.c:798:mpls_flush_in_tree: # mii-gen-0:33-135169
MPLS DEBUG net/mpls/mpls_in_info.c:621:mpls_mii_kobject_release: enter
MPLS DEBUG net/mpls/mpls_utils.c:482:mpls_instruction_clear: enter
MPLS DEBUG net/mpls/mpls_utils.c:491:mpls_instruction_clear: exit
MPLS DEBUG net/mpls/mpls_in_info.c:637:mpls_mii_kobject_release: exit
MPLS DEBUG net/mpls/mpls_in_info.c:798:mpls_flush_in_tree: # mii-gen-0:34-139265
MPLS DEBUG net/mpls/mpls_in_info.c:621:mpls_mii_kobject_release: enter
MPLS DEBUG net/mpls/mpls_utils.c:482:mpls_instruction_clear: enter
MPLS DEBUG net/mpls/mpls_utils.c:491:mpls_instruction_clear: exit
MPLS DEBUG net/mpls/mpls_in_info.c:637:mpls_mii_kobject_release: exit
MPLS DEBUG net/mpls/mpls_in_info.c:806:mpls_flush_in_tree: exit
MPLS DEBUG net/mpls/mpls_ioctls.c:39:mpls_ioctl: IOCTL 35859
MPLS DEBUG net/mpls/mpls_out_info.c:545:mpls_flush_out_tree: enter
MPLS DEBUG net/mpls/mpls_out_info.c:558:mpls_flush_out_tree: exit

Known Bugs
------------------------
* Spec NH management is buggy.

ipv6: Unknown symbol spec_nh_find
request_module: failed /sbin/modprobe -- net-pf-10. error = 256

* It is impossible to add out operations. dst moi is null. Could you take
a look at that? We should try to release a working test version soon...

MPLS DEBUG net/mpls/mpls_ioctls.c:39:mpls_ioctl: IOCTL 35847
MPLS DEBUG net/mpls/mpls_ioctls.c:39:mpls_ioctl: IOCTL 35864
MPLS DEBUG net/mpls/mpls_out_info.c:255:mpls_set_out_label_instructions: enter
MPLS DEBUG net/mpls/mpls_utils.c:411:mpls_instruction_build: enter
MPLS DEBUG net/mpls/mpls_utils.c:138:mpls_make_dst: enter
Unable to handle kernel NULL pointer dereference at virtual address 00000060
 printing eip:
 c015c822
 *pde = 00000000
 Oops: 0000 [#2]
 CPU:    0
 EIP:    0060:[<c015c822>]    Not tainted
 EFLAGS: 00010246
 EIP is at kmem_flagcheck+0x22/0x40
 eax: 00000000   ebx: eb36b7a8   ecx: 00000001   edx: c039d360
 esi: 00000000   edi: 00000020   ebp: f5b694e4   esp: f5b694e4
 ds: 007b   es: 007b   ss: 0068
 Process mplsadm (pid: 11467, threadinfo=f5b68000 task=e889e9c0)
 Stack: f5b69508 c015d8d5 00000000 00000020 dabaa498 f5b6951c eb36b7a8 c04576c0
 00000020 f5b69528 c03278ff 00000000 00000020 dabaa498 eb36b7a8 00000000
 c1bda074 f5b69554 c039d3d9 c04576c0 c03fb2b2 0000008a c03c5183 f5b69554
 Call Trace:
 [<c015d8d5>] kmem_cache_alloc+0x25/0x1e0
 [<c03278ff>] dst_alloc+0x2f/0xa0
 [<c039d3d9>] mpls_make_dst+0x69/0x2d0
 [<c039b02e>] mpls_build_opcode_set+0x5e/0x1e0
 [<c039db09>] mpls_instruction_build+0x209/0x2f0
 [<c039c189>] mpls_set_out_label_instructions+0x79/0x210

// -------------------------------------------------------------------
// Ramon Casellas - GET/ENST/INFRES/RHD/A508 - cas...@in...