Re: [RTnet-developers] Re: Common rt-buffers

[Jan K. <jan...@we...>]

Yuchen Zhang wrote:
> [...]
> This is because my own idea was already more or less similiar to
> yours, i.e. to move memory management to a base layer, like RTDM. BUT,
> I dont think my idea is exactly the same as what you said, see below:
> 1. I think the buffer size should be an argument to API of allocating
> parent pools, i.e. the kernel_slab_pool.

Did I expressed this differently? Yes, this makes sense most (more
precisely: buffer size and minimum header size).

> 
> 2. I think  We DONT need to have different APIs for creating and
> joining parent pool. The name of the pool is given in the creating API
> anyway, but in case a pool of same name already exists, the pointer to
> the existing pool is returned. BUT this is transparent to
> applications.

Also what I meant. You call some rtdm_prntpool_create(name, buf_size,
min_head_size) or so, and that pool gets either created or the existing
one is returned.

> 
> 
>>>Now, I have some further idea: what if we also bind the packet
>>>capturing service with the memory management. That is if a real-time
>>>application needs real-time memory allocation, it can also ask for
>>>capturing of all those allocated memory (packets). We can use some
>>>user-space tools to analyze the captured memory. The captured memory
>>>can be delivered via fake Ethernet interface, so tools like Ethereal
>>>can be used (only need to extend it based on the application).
>>>
>>>The capturng of a packet can be done when that packet memory is freed,
>>>where the capturing operation is transparent to the application. Well,
>>>the application must claim somewhere  before the packet is freed that
>>>it should be captured, e.g. right after it is allocated.
>>>
>>>Concerning the packet capturing idea, we can explore more, if it is
>>>agreed to be interesting and effort-worth.
>>
>>Actually, I'm not yet convinced that outsourcing /this/ job to RTDM is
>>useful.
>>
> 
> Ok, now time for real debating:)
> Let's first sketch the main frame code to support this service.
> 
> struct Cap_struct {
>          struct net_dev *fakedev;  /*the fake ethernet dev to deliver
> captured packet to user-space*/
>          struct rt-buffer-pool   *cap_pool;  /*where the buffers to
> compensate application memory come from*/
>         int buffer_size;  /* the size of buffer that can be captured here */
> }

So, that structure describes a capturing instance, conceptually
something like rtcap service of RTnet?

> 
> struct rt-buffer{
>           .....
>           struct Cap_struct *aCap;
> }
> 

And this link is needed as there can be multiple capturers, right? But
what about the additional fields required by rtcap to save the initial
buffer layout, the compensation buffer, timestamp, and other stuff? Note
that rtcap does deferred capturing. Does your service in RT-FireWire
work the same way?

> /*function to create a Cap_struct*/
> struct Cap_struct *create_cap( int size, //  the buffer size
>                                              struct kernel_slab_pool
> *parent_pool,
>                                               char *name) // the name
> of fake ethernet dev, should be
>                                                                
> //specific about application
> {
>        if(*name already exists){
>                return (address of existing Cap_struct)
>        }
>       .............
> }
> 
> /*function to remove a Cap_struct*/
> void remove_cap(struct Cap_struct *aCap_struct)
> 

Ok, this makes sense.

> /*function to capture a buffer*/
> struct rt-buffer *cap_rtbuffer(struct rt-buffer *abuffer, int
> buffer_size, struct cap_struct *aCap)
> {
>      if(aCap->buffer_size != buffer_size)
>                    //print error msg and return
>      else{
>           //allocate a compesating buffer from the cap pool
>           // exchange buffers
>           //deliver the captured buffer to fake ethernet dev

I hope you do NOT deliver immediately, just pend the delivery for Linux
to get in the CPU again.

>           //return the address of compensating buffer
>          return &comp_buffer;
>      }
> }

What's the caller supposed to do with that returned buffer?

> 
> /*function to free a rt buffer*/
> void free_rtbuffer(struct rt-buffer *abuffer){
>     if(abuffer->aCap){ //pointer NOT null, so we should capture this one
>          abuffer = cap_rtbuffer(abuffer, sizeof(abuffer), abuffer->aCap);

Is this the only use case for cap_rtbuffer? How do you mark the actual
capturing date? Only on buffer release? Or did you leave out this aspect
in the draft?

>     }
>     //free abuffer
>     ......
> }
> 
> 
>>At least for RTnet, capturing is a debugging feature you don't what
>>compiled in for production systems. It adds overhead to some critical
>>paths that may hurt on low-end boxes.
>>
> 
> I dont think the overhead is noticable. When the base module (say
> RTDM) has capturing option enabled, the only overhead that happens to
> applications that do NOT need capturing   is, in the free_rtbuffer
> function, the extra checking of a rt-buffer's cap_struct pointer.

Well, the current kfree_rtskb in RTnet has to do more. It iterates over
all rtskbs in a (possible) chain and checks for each if it is a captured
one. Anyway, this needless overhead in the capturing-compiled-in-but-
not-used case can be avoided with a simple check before entering any
loop. So, you are likely right, the overhead is acceptable - and could
still be reduced when we add a CONFIG_switch to RTDM.

Apropos chains: does RT-FireWire's rtpk support them? If we drag
capturing into the core service layer, this is required.

> 
> 
>>The question to me is then what functionality of the capturing subsystem
>>could be moved to RTDM while still keeping the control of using it or
>>not (without runtime costs) at the configuration level of the driver
>>using it. I would like to be able to switch capturing off during "make
>>menuconfig" of RTnet, but maybe switching it on for RT-FireWire at the
>>same time.
>>
> 
> The capturing I am proposing is conceptually different from the
> capturing in RTnet (what I understood). It is fully oriented to
> applications, while RTnet's capturing is more or less per networking
> interface. That means in this new capturing service, buffer capturing
> can happen /anywhere/, even something totally away from network
> transaction. This is especially useful when we develop multiple
> applications over /one/ FireWire interface: we may only want to
> capture buffers used by one high-level application. Besides FireWire

I may not know my own code well enough, but I don't see a reason why
setting the capturing mark in RTnet should not be movable, also away
from the general places where it is right now.

> and Ethernet, other applications based on RTDM, like socketCAN, can
> also benefit from this service, as long as they need some analysis
> over the used memory to gain some insight to the system behavior.

I had Socket-CAN in mind as well first, but Sebastian's implementation
is smartly not using discrete CAN-packet buffers. Rather, there are ring
buffers per receiving socket. This is more efficient than throwing
around 8 bytes data + some bytes head per CAN frame in individual
packets. CAN packets are too small to justify the maintenance costs of
skb-like approaches - that's at least my strong believe (but we should
soon be able to do some benchmarks on this as well - there is a Linux
skb-based variant on the way).

Anyway, one may consider if "external" users of the rt-packet capturing
services make sense as well. But that's more kind of a second or third step.

> 
> BTW, are you going to work on these soon, at least the memory
> management part? How can I contribute? My idea is to make RTDM

My plan was to burden much of the development on you, as my time is more
than just limited. ;)

Well, I have no fixed schedule, but I guess things will move quickly as
soon as we have some API and data structures.

> temporarily a separate module that can be compiled over xenomai, and
> put it either on RTnet svn or RT-FireWire svn.

I wouldn't drag the whole RTDM module back in RTnet again (that's too
platform dependent anyway). I would suggest to develop the required
services as an optional subsystem first inside RT-FireWire and RTnet. As
soon as this API is stable for each individual project, we could than
merge it into a private Xenomai tree, test the cooperation between RTnet
and RT-FireWire, and then start submitting patches against Xenomai first
and RTAI later.

And, with the RTDM-packet subsystem still in both projects available, we
could continue to provide support for older RTDM cores without that
feature. Detecting the RTDM API revision is easy and could be used to
decide if the internal rt-packet code should be build.

Jan