Re: [E1000-devel] Maximum number of multicast groups on e1000e?

[Brandeburg, J. <jes...@in...>]

Wow, thanks Bill, that explanation was spot on, and very well put.  Thanks for taking the time to respond.

Jesse

-----Original Message-----
From: Bill Paul [mailto:wp...@wi...] 
Sent: Monday, October 19, 2009 4:17 PM
To: sco...@gs...
Cc: e10...@li...
Subject: Re: [E1000-devel] Maximum number of multicast groups on e1000e?


> Hello e1000 developers,
> 
> Is there a limit, in either the driver or the hardware, to the number of 
> multicast groups an e1000e adapter may subscribe to?

> We do know this is a tunable in Linux (we use RHEL5u3) via 
> /proc/sys/net/ipv4/igmp_max_memberships, but want to understand where the 
> ceiling is.

> Thanks,
> Scott

There's a couple of issues here. One is that there is a difference between 
IPv4 multicast groups and ethernet groups. For IPv4, there group space is 
defined by IP addresses in the 224.x.x.x range, while for ethernet the group 
space is defined as any address with the least significant bit in the first 
octet set. The IPv4 space is mapped to a portion of the ethernet space.

The multicast filter in the hardware is concerned only with ethernet multicast 
groups. With the PRO/1000 devices, the RX filter typically has a 16 entry 
perfect filter and a 4096 bit multicast hash table filter. The 16 entry 
perfect filter is fairly self-explanatory: the receiver checks the 
destination address in incoming frames against the addresses in the filter 
slots, and if they match, the frame is admitted. Each slot can be enabled or 
disabled separately. Typically, only one slot is used for the station 
address, and you can use the remaining entries for multicast groups if you 
want.

After testing against the perfect filter, the chip will then test multicast 
frames against the hash table filter. The filter is programmed by the host 
calculating a hash of the desired ethernet group address and then using 11 
bits of the hash as a table index. The driver sets the corresponding bit in 
the hash table at the calculated index. When a multicast frame is received, 
the chip will use the same algorithm to calculate a hash index value in 
hardware, then see if the corresponding bit in the table is set. If it is, 
then the frame is admitted, otherwise it's discarded.

The hash table method is used because it's not possible (or at least not 
practical) to implement a perfect filtering method for mutlicast groups. You 
would need a CAM filter with a lot of memory to accomodate all possible group 
addresses. The hash table approach is a compromise. Intel at least uses a 
fairly large table -- many devices use just a 64-bit table.

Lastly, the PRO/1000 hardware also implements an 'all multicast' mode, where 
you can set a single bit in the RCTL register to admit all multicast frames, 
bypassing the hash table. This is often used for systems that will be 
configured as IP multicast routers.

As for the maximum number of groups you can join, technically there shouldn't 
really be a limit. Obviously, if the hash table method is used and you join a 
very large number of groups, you will inevitably begin filling up more and 
more bits in the table, which will lead to hash table collisions. If you take 
it to the extreme, you will end up setting all the bits, thereby allowing all 
mutlicast frames to be admitted. (This is an alternate way of implementing 
the 'all multicast' mode described above, which is sometimes needed on 
devices that don't have an explicit 'all multicast' filtering option.)

I've seen varying approaches to dealing with this in different drivers/OSes. 
Sometimes the driver keeps track of the number of groups and will start 
returning failure once the group count reaches a pre-defined limit. It looks 
like the Linux driver doesn't do this, and just keep adding entries whenever 
it's asked. While it may be sub-optimal to eventually degrade to allmulti 
mode, that approach at least keeps things working. For IPv4 multicast at 
least, additional software address filtering is required anyway.

-Bill

-- 
=============================================================================
-Bill Paul            (510) 749-2329 | Senior Engineer, Master of Unix-Fu
                 wp...@wi... | Wind River Systems
=============================================================================
   "I put a dollar in a change machine. Nothing changed." - George Carlin
=============================================================================


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
E1000-devel mailing list
E10...@li...
https://lists.sourceforge.net/lists/listinfo/e1000-devel