Re: [tipc-discussion] [PATCH 5/5] tipc: add ip/udp media type

[Erik H. <eri...@er...>]

On Mon, Jan 19, 2015 at 10:43:56AM +0800, Ying Xue wrote:
> On 01/16/2015 07:49 PM, Erik Hugne wrote:
> >>> > > +/**
> >>> > > + * tipc_send_msg - enqueue a send request
> >>> > > + *
> >>> > > + * The send request need to be deferred since we cannot call kernel
> >>> > > + * socket API functions while holding the node spinlock.
> >>> > > + */
> >> > 
> >> > Yes. Unfortunately this is necessary for now, but we should aim at
> >> > releasing the node spinlock before sending the buffers, on all
> >> > bearers. This will probably have some positive impact on performance.
> >> >
> > What's holding me from sending another version now is the skb reallocation
> > from kernel_sendmsg().
> > On lowmem systems this is causing the kworker thread to spin heavily on memory
> > allocation because it has run out of memory in the kmalloc-2048 pool.
> > It should be possible to reuse the TIPC skb and pass that to the IP stack, but
> > i don't know exactly how to do this.
> 
> 
> If this is another reason why we have to send TIPC buffers in a separate
> thread, maybe we should seriously consider the idea I proposed before.
> If we are really able to make the TIPC sending path non-atomic, this is
> not only of benefit to UDP bearer, but also it gives TIPC an extreme
> flexibility, for example, we can use all kinds of tunnels implemented in
> networking stack as TIPC bearers, such as, ipip, vti, vxlan,
> ipv6_tunnels etc.
> 

Making the send path non-atomic would help in the sense that we'd get rid of the
bearer-level send workqueue and related stuff, but there's still the issue with
each packet going through kernel_sendmsg is reallocated/copied to a new skb.

I'm wondering if this would be feasible:
1. export the udp_send_skb() function
2. skb_push(sizeof udphdr) on the TIPC SKB.
3. Do a routing lookup for the destination ip and pass that, together with the
   TIPC SKB to udp_send_skb()


//E