ocf-linux-users

Re: [Ocf-linux-users] [Openswan Users] IPSec L2tpv3 throughput low using Netkey kernel stack

[Paul W. <pa...@xe...>]

On Mon, 16 May 2011, Vasanth Ragavendran wrote:

> but when the Sec driver is configured as module into the kernel the module doesn;t get inserted
> gives segmentation fault and if the Sec driver is configured as non-module type the kernel doesn't
> bootsup at all!

David might be able to help with this. The ubsec driver is mostly used on linksys/asus machines,
perhaps yours is slightly different?

Note that slow throughput could also be caused by fragmentation as a result of the extra IPsec
header space, causing you to encrypt two packets for every one unencrypted packet. Try setting
your LAN MTU to 1400 and see if the problem remains?

Paul

Re: [Ocf-linux-users] [Openswan Users] IPSec L2tpv3 throughput low using Netkey kernel stack

[Kim P. <kim...@fr...>]

On Tue, 17 May 2011 14:27:59 -0400
Paul Wouters <pa...@xe...> wrote:

> On Mon, 16 May 2011, Vasanth Ragavendran wrote:
> 
> > but when the Sec driver is configured as module into the kernel the module doesn;t get inserted
> > gives segmentation fault and if the Sec driver is configured as non-module type the kernel doesn't
> > bootsup at all!
> 
> David might be able to help with this. The ubsec driver is mostly used on linksys/asus machines,
> perhaps yours is slightly different?

Based on the freescale URL posted to users@openswan, I assume you're
referring to the "SEC23DRVRS" driver?  That is a standalone driver,
and won't be of use unless you're prepared to write code to
integrate it into your IPsec stack.

Known working (to me at least) IPsec offload configuration for the
8315 should be NETKEY with CONFIG_CRYPTO_DEV_TALITOS configured in
a vanilla kernel.  To be able to tell whether h/w crypto offload is
in operation, see 'grep talitos /proc/interrupts' run.

> Note that slow throughput could also be caused by fragmentation as a result of the extra IPsec
> header space, causing you to encrypt two packets for every one unencrypted packet. Try setting
> your LAN MTU to 1400 and see if the problem remains?

or lower?  Whether this is causing the problem can be verified by
comparing no-encryption vs. 'null' algorithm encryption.

Kim

Re: [Ocf-linux-users] [Openswan Users] IPSec L2tpv3 throughput low using Netkey kernel stack

[Paul W. <pa...@xe...>]

On Tue, 17 May 2011, Kim Phillips wrote:

>>> but when the Sec driver is configured as module into the kernel the module doesn;t get inserted
>>> gives segmentation fault and if the Sec driver is configured as non-module type the kernel doesn't
>>> bootsup at all!
>>
>> David might be able to help with this. The ubsec driver is mostly used on linksys/asus machines,
>> perhaps yours is slightly different?
>
> Based on the freescale URL posted to users@openswan, I assume you're
> referring to the "SEC23DRVRS" driver?  That is a standalone driver,
> and won't be of use unless you're prepared to write code to
> integrate it into your IPsec stack.
>
> Known working (to me at least) IPsec offload configuration for the
> 8315 should be NETKEY with CONFIG_CRYPTO_DEV_TALITOS configured in
> a vanilla kernel.  To be able to tell whether h/w crypto offload is
> in operation, see 'grep talitos /proc/interrupts' run.

Ah, i thought he meant the ubsec one. Talitos I believe is also supported with OCF and KLIPS
IPsec, but I'm sure David can confirm that.

Paul

Re: [Ocf-linux-users] [Openswan Users] IPSec L2tpv3 throughput low using Netkey kernel stack

[Paul W. <pa...@xe...>]

On Wed, 18 May 2011, Paul Wouters wrote:

> Subject: Re: [Ocf-linux-users] [Openswan Users] IPSec L2tpv3 throughput low
>     using Netkey kernel stack

Remind me to bring the Xelerance internal wiki pages on openswan / ocf benchmarking
to the public wiki. They're currently on a VM I don't have access to. But ping me
in a few days when I have access to these if I haven't posted them.

We found various tweaks to increase the traffic and got numbers that were comparable
in speed despite the SAref support overhead for L2TP/Transport Mode. This was using
cryptosoft with KLIPS on SMP machines without crypto offload hardware.

A few notes:

- Do not run iperf on the IPsec machines but on machines behind those.
- Run multiple TCP streams to reduce effects of a single stalled/lost packet
- Play a LOT with the MTU sizes
- Different brands of eth cards make a huge difference
- Disable various nic card offloading/checksumming
- Ensure the OCF buffers are high enough.Openswan's _startklips script tries
   to do this for you based on CPUs, but not based on hardware crypto offload.
   Specifically look at:
    /sys/module/ocf/parameters/crypto_q_max
    /sys/module/ipsec/parameters/ipsec_irs_cache_allocated_max
    /sys/module/ipsec/parameters/ipsec_ixs_cache_allocated_max

Paul

Re: [Ocf-linux-users] [Openswan Users] IPSec L2tpv3 throughput low using Netkey kernel stack

[David M. <dav...@mc...>]

Jivin Paul Wouters lays it down ...
> On Tue, 17 May 2011, Kim Phillips wrote:
> 
> >>> but when the Sec driver is configured as module into the kernel the module doesn;t get inserted
> >>> gives segmentation fault and if the Sec driver is configured as non-module type the kernel doesn't
> >>> bootsup at all!
> >>
> >> David might be able to help with this. The ubsec driver is mostly used on linksys/asus machines,
> >> perhaps yours is slightly different?
> >
> > Based on the freescale URL posted to users@openswan, I assume you're
> > referring to the "SEC23DRVRS" driver?  That is a standalone driver,
> > and won't be of use unless you're prepared to write code to
> > integrate it into your IPsec stack.
> >
> > Known working (to me at least) IPsec offload configuration for the
> > 8315 should be NETKEY with CONFIG_CRYPTO_DEV_TALITOS configured in
> > a vanilla kernel.  To be able to tell whether h/w crypto offload is
> > in operation, see 'grep talitos /proc/interrupts' run.
> 
> Ah, i thought he meant the ubsec one. Talitos I believe is also supported with OCF and KLIPS
> IPsec, but I'm sure David can confirm that.

Yeah, Kim wrote the driver,  but the in kernel one is more up to date ;-)

Cheers,
Davidm

-- 
David McCullough,      dav...@mc...,  Ph:+61 734352815
McAfee - SnapGear      http://www.mcafee.com         http://www.uCdot.org

Re: [Ocf-linux-users] [Openswan Users] IPSec L2tpv3 throughput low using Netkey kernel stack

[Vasanth R. <rag...@gm...>]

Thank you all so much for responding.

On Wed, May 18, 2011 at 7:02 PM, David McCullough <
dav...@mc...> wrote:

>
> Jivin Paul Wouters lays it down ...
> > On Tue, 17 May 2011, Kim Phillips wrote:
> >
> > >>> but when the Sec driver is configured as module into the kernel the
> module doesn;t get inserted
> > >>> gives segmentation fault and if the Sec driver is configured as
> non-module type the kernel doesn't
> > >>> bootsup at all!
> > >>
> > >> David might be able to help with this. The ubsec driver is mostly used
> on linksys/asus machines,
> > >> perhaps yours is slightly different?
> > >
> > > Based on the freescale URL posted to users@openswan, I assume you're
> > > referring to the "SEC23DRVRS" driver?  That is a standalone driver,
> > > and won't be of use unless you're prepared to write code to
> > > integrate it into your IPsec stack.
> > >
>
Yeah i was mentioning about SEC23DRVRS apologies for mentioning it in short
form.
and as Paul has asked to test with a lower MTU say 1400, the thruput worsens
i get only
4.71Mbps however with MTU being 1500 i get the thruput as 14.7Mbps.

> > Known working (to me at least) IPsec offload configuration for the
> > > 8315 should be NETKEY with CONFIG_CRYPTO_DEV_TALITOS configured in
> > > a vanilla kernel.  To be able to tell whether h/w crypto offload is
> > > in operation, see 'grep talitos /proc/interrupts' run.
> >
>
As Kim had mentioned I had loaded the CONFIG_CRYPTO_DEV_TALITOS as a module
with the module in the kernel i am getting a lower throughput! I am getting
only 13.4Mbps however without the module inserted i get 14.7Mbps how could
this be possible and the results sounds really ridiculous to me! And when
the CONFIG_CRYPTO_DEV_TALITOS is loaded i am able to view it using grep
talitos /proc/interrupts. so the hardware accelerator is getting used
however resulting in a lower throughput! That's absurd am I missing
something here?

> Ah, i thought he meant the ubsec one. Talitos I believe is also supported
> with OCF and KLIPS
> > IPsec, but I'm sure David can confirm that.
>
> I tried even with KLIPS and ocf,cryptodev and ocf-talitos and even here i
get no better throughput its only 11.7Mbps! Why am I getting such a lower
throughputs?
Thank you so much everybody for helping out. Awaiting your replies to
proceed further.

Yeah, Kim wrote the driver,  but the in kernel one is more up to date ;-)
>
> Cheers,
> Davidm
>
> --
> David McCullough,      dav...@mc...,  Ph:+61 734352815
> McAfee - SnapGear      http://www.mcafee.com         http://www.uCdot.org
>



-- 
Thanks and Regards
R.Vasanth Ragavendran

Re: [Ocf-linux-users] [Openswan Users] IPSec L2tpv3 throughput low using Netkey kernel stack

[Kim P. <kim...@fr...>]

On Thu, 19 May 2011 20:18:40 -0700
Vasanth Ragavendran <rag...@gm...> wrote:

> On Wed, May 18, 2011 at 7:02 PM, David McCullough <
> dav...@mc...> wrote:
> > Jivin Paul Wouters lays it down ...
> > > On Tue, 17 May 2011, Kim Phillips wrote:
> > >
> > > > Known working (to me at least) IPsec offload configuration for the
> > > > 8315 should be NETKEY with CONFIG_CRYPTO_DEV_TALITOS configured in
> > > > a vanilla kernel.  To be able to tell whether h/w crypto offload is
> > > > in operation, see 'grep talitos /proc/interrupts' run.
> >
> As Kim had mentioned I had loaded the CONFIG_CRYPTO_DEV_TALITOS as a module
> with the module in the kernel i am getting a lower throughput! I am getting
> only 13.4Mbps however without the module inserted i get 14.7Mbps how could
> this be possible and the results sounds really ridiculous to me! And when
> the CONFIG_CRYPTO_DEV_TALITOS is loaded i am able to view it using grep
> talitos /proc/interrupts. so the hardware accelerator is getting used
> however resulting in a lower throughput! That's absurd am I missing
> something here?

Is this a *vanilla* kernel CONFIG_CRYPTO_DEV_TALITOS driver, or is
it from the freescale BSP?  If the latter, please forward your
inquiry to the standard freescale BSP support channels.

Otherwise (vanilla kernel), sounds like too little crypto
payload and/or rate - so little that sending it to the accelerator
and waiting for results takes longer than s/w crypto on the core.
Can you benchmark using the 'null' cipher algorithm to make sure
this is the case?

Kim

Re: [Ocf-linux-users] [Openswan Users] IPSec L2tpv3 throughput low using Netkey kernel stack

[Vasanth R. <rag...@gm...>]

On Fri, May 20, 2011 at 12:20 PM, Kim Phillips
<kim...@fr...>wrote:

> On Thu, 19 May 2011 20:18:40 -0700
> Vasanth Ragavendran <rag...@gm...> wrote:
>
> > On Wed, May 18, 2011 at 7:02 PM, David McCullough <
> > dav...@mc...> wrote:
> > > Jivin Paul Wouters lays it down ...
> > > > On Tue, 17 May 2011, Kim Phillips wrote:
> > > >
> > > > > Known working (to me at least) IPsec offload configuration for the
> > > > > 8315 should be NETKEY with CONFIG_CRYPTO_DEV_TALITOS configured in
> > > > > a vanilla kernel.  To be able to tell whether h/w crypto offload is
> > > > > in operation, see 'grep talitos /proc/interrupts' run.
> > >
> > As Kim had mentioned I had loaded the CONFIG_CRYPTO_DEV_TALITOS as a
> module
> > with the module in the kernel i am getting a lower throughput! I am
> getting
> > only 13.4Mbps however without the module inserted i get 14.7Mbps how
> could
> > this be possible and the results sounds really ridiculous to me! And when
> > the CONFIG_CRYPTO_DEV_TALITOS is loaded i am able to view it using grep
> > talitos /proc/interrupts. so the hardware accelerator is getting used
> > however resulting in a lower throughput! That's absurd am I missing
> > something here?
>
> Is this a *vanilla* kernel CONFIG_CRYPTO_DEV_TALITOS driver, or is
> it from the freescale BSP?  If the latter, please forward your
> inquiry to the standard freescale BSP support channels.
>
> No this is the vanilla kernel's and not from the freescale.

> Otherwise (vanilla kernel), sounds like too little crypto
> payload and/or rate - so little that sending it to the accelerator
> and waiting for results takes longer than s/w crypto on the core.
> Can you benchmark using the 'null' cipher algorithm to make sure
> this is the case?
>
Will check on the 'null' cipher part. But why is the thruput lower when
using Klips, ocf and cryptodev and the rest of the settings being the same i
achieve only around 11Mbps. Thank u so much and sorry for the late reply.

>
> Kim
>
>


-- 
Thanks and Regards
R.Vasanth Ragavendran

Re: [Ocf-linux-users] [Openswan Users] IPSec L2tpv3 throughput low using Netkey kernel stack

[Kim P. <kim...@fr...>]

On Mon, 23 May 2011 16:46:34 +0800
Vasanth Ragavendran <rag...@gm...> wrote:

> On Fri, May 20, 2011 at 12:20 PM, Kim Phillips
> <kim...@fr...>wrote:
> 
> > On Thu, 19 May 2011 20:18:40 -0700
> > Vasanth Ragavendran <rag...@gm...> wrote:
> >
> > > On Wed, May 18, 2011 at 7:02 PM, David McCullough <
> > > dav...@mc...> wrote:
> > > > Jivin Paul Wouters lays it down ...
> > > > > On Tue, 17 May 2011, Kim Phillips wrote:
> > > > >
> > > > > > Known working (to me at least) IPsec offload configuration for the
> > > > > > 8315 should be NETKEY with CONFIG_CRYPTO_DEV_TALITOS configured in
> > > > > > a vanilla kernel.  To be able to tell whether h/w crypto offload is
> > > > > > in operation, see 'grep talitos /proc/interrupts' run.
> > > >
> > > As Kim had mentioned I had loaded the CONFIG_CRYPTO_DEV_TALITOS as a
> > module
> > > with the module in the kernel i am getting a lower throughput! I am
> > getting
> > > only 13.4Mbps however without the module inserted i get 14.7Mbps how
> > could
> > > this be possible and the results sounds really ridiculous to me! And when
> > > the CONFIG_CRYPTO_DEV_TALITOS is loaded i am able to view it using grep
> > > talitos /proc/interrupts. so the hardware accelerator is getting used
> > > however resulting in a lower throughput! That's absurd am I missing
> > > something here?
> >
> > Is this a *vanilla* kernel CONFIG_CRYPTO_DEV_TALITOS driver, or is
> > it from the freescale BSP?  If the latter, please forward your
> > inquiry to the standard freescale BSP support channels.
> >
> > No this is the vanilla kernel's and not from the freescale.

ok, so some version of the kernel.org kernel + klips + ocf patches.
What version?

> > Otherwise (vanilla kernel), sounds like too little crypto
> > payload and/or rate - so little that sending it to the accelerator
> > and waiting for results takes longer than s/w crypto on the core.
> > Can you benchmark using the 'null' cipher algorithm to make sure
> > this is the case?
> >
> Will check on the 'null' cipher part. But why is the thruput lower when
> using Klips, ocf and cryptodev and the rest of the settings being the same i

OCF_CRYPTODEV=y?  shouldn't be relevant.

> achieve only around 11Mbps. Thank u so much and sorry for the late reply.

so, given the following configuration settings:

CRYPTO_DEV_TALITOS=y
OCF_CRYPTOSOFT=y
#OCF_TALITOS is not set

and

grep talitos /proc/interrupts

indeed shows numbers incrementing with traffic - i.e., the
algorithms chosen for the tunnel are implemented in the driver -
then there's some performance bottleneck between the klips and/or
ocf code and the kernel's cryptoAPI.  How does a pure vanilla kernel
perform? i.e, without klips & ocf patches.

Kim