Re: [Ocf-linux-users] OCF userland support

[Egor N. M. <eg...@pa...>]

David - thanks for a quick response.  I have comments in line.

David McCullough wrote:

>Jivin Egor N. Martovetsky lays it down ...
>  
>
>>David,
>>
>>I noticed that the throughput I get when using cryptotest or OpenSSL speed
>>is much worse than what I get using ocf-bench.  I also don't get much
>>improvement, if at all, when running mutiple threads of the above programs.
>>    
>>
>
>ocf-bench runs in kernel mode,  the data does not need to be
>copied from user space to kernel space and back.  This make a massive
>difference to performance.
>
>All user apps need to pass their data through to the kernel and back.
>Unfortunately we don't have a zero copy API for OCF (yet ;-)
>
>Basically,  for OCF accelerated user space,  you need to be using larger
>packets to help overcome the overheads of the user-kernel-user copies,
>but it will never be a good as in-kernel crypto with a zero copy
>interface.
>
>  
>
Yes, I was aware of the copying, and it explains some of the performance 
degradation,
that I see with a single thread user space program vs. kernel mode.  As 
you point out, the performance
of user space program gets better relative to kernel mode, as the packet 
size is increased.  However,
in ocf-bench I can keep cpu 100% utilized submitting and processing done 
packets, while a single thread
of cryptotest is unable to do that, so I tried to run a few threads, and 
saw the throughput get worse.

>>It seems that this is a result of using ioctl(vs unlocked_ioctl) to 
>>access /dev/crypto, which
>>would only allow one process doing crypto at a given time.  Is that a 
>>known problem and
>>are there plans to fix it?
>>    
>>
>
>I wasn't aware that ioctl would prevent multiple processes from working
>in parallel.  I have seen performance improvements with multiple threads
>on 2.4 systems.  Haven't checked on 2.6
>
>  
>
In 2.6 kernel the do_ioctl() function in fs/ioctl.c does a kernel lock 
before calling device's ioctl.
Since cryptodev ioctl submits a packet and waits for completion before 
returning, effectively
only one request can be processed at a given time, and I am not able to 
take advantage of multiple
crypto channels executing in parallel.

>>Also, is ocf-bench SMP safe?  I had to set CRYPTO_F_BATCH in the 
>>crp_flags to make it work,
>>otherwise with the CRYPTO_F_CBIMM it would not work in the SMP mode.
>>    
>>
>
>I have never thought nor checked that ocf-bench is SMP safe.  Which OCF
>driver are you using when doing your tests ?  It could explain a few
>things,
>  
>
It's my own driver, for a new PA Semi chip, and since it is still under 
development -
yes, it can explain a few things. :)
But in this case, I don't think so, because in general it works fine, 
and ocf-bench
works fine in nosmp mode, or with CRYPTO_F_BATCH mode, which makes
the completions go through a callback queue that is protected by 
spinlocks, as opposed
to immediate callbacks.

>Cheers,
>Davidm
>
>  
>


-- 
Egor N. Martovetsky