Re: [Ocf-linux-users] PD: OCF's ixp4xx freeze on 2.6.18-rt7

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Hi David,

David McCullough wrote:
> Jivin Tomasz Rostanski lays it down ...
>> David,
>>
>>> Jivin Tomasz Rostanski lays it down ...
>>>> Hi David,
>>>>
>>>>> Jivin Tomasz Rostanski lays it down ...
>>>>>> Hi David,
>>>>>>> Jivin Tomasz Rostanski lays it down ...
>>>>>>>> Hi David,
>>>>>>>>
>>>>>>>> Do you made any progress?
>>>>>>> Yep,  sorry for the delay,  just ran it up on 2.6.23 using CSR2.4 and
>>>>>>> ocf.  Seems to be ok.  Haven't tried ipsec yet,  but openssl speed
>>>>>>> works fine (although throughput seems to be down).
>>>>>>>
>>>>>>> The driver is at least processing requests ok.
>>>>>>>
>>>>>>> Are you installing the correct NPE code with something like:
>>>>>>>
>>>>>>> 	cat /etc/IxNpeMicrocode.dat >/dev/ixNpe
>>>>>> I have NPE code compiled in. I have also made a build without compiling 
>>>>>> it in, the result was the same.
>>>>> We have an updated patch for CSR-2.4 at:
>>>>>
>>>>> 	http://www.snapgear.org/snapgear/downloads.html
>>>>>
>>>>> might be something in there that helps you.  I am effectively running SG
>>>>> Linux 3.5 on my IXP425's with that access lib patch applied,
>>>> I have imported this patch (not everything I need) to our trunk and 
>>>> checked with the OCF 20070727 and... is working!
>>>> Thank you for your help.
>>> No problems,  glad it worked ;-)
>> But now I have a problems with openssh.
>>
>> When I run first "openssl speed -evp des -elapsed -engine cryptodev"
>>
>> I got:
>>
>> # ssh 192.168.0.10
>> [  151.390000] ixp_newsession()
>> [  151.400000] ixp_freesession()
>> [  151.400000] ixp_newsession()
>> [  151.400000] ixp_freesession()
>> [  151.410000] ixp_newsession()
>> [  151.410000] ixp_freesession()
>> [  151.410000] ixp_newsession()
>> [  151.420000] ixp_freesession()
>> [  151.440000] ixp_newsession()
>> [  151.440000] ixp_freesession()
>> [  151.450000] ixp_newsession()
>> [  151.450000] ixp_freesession()
>> [  151.450000] ixp_newsession()
>> [  151.460000] ixp_freesession()
>> [  151.460000] ixp_newsession()
>> [  151.460000] ixp_freesession()
>> [  151.470000] ixp_newsession()
>> [  151.470000] ixp_freesession()
>> [  151.470000] ixp_newsession()
>> [  151.470000] ixp_freesession()
>> [  151.480000] ixp_newsession()
>> [  151.480000] ixp_freesession()
>> [  151.480000] ixp_newsession()
>> [  151.490000] ixp_freesession()
>> The authenticity of host '192.168.0.10 (192.168.0.10)' can't be established.
>> RSA key fingerprint is 4d:4c:97:d9:0b:48:35:f4:5d:e5:3b:1a:55:ff:87:57.
>> Are you sure you want to continue connecting (yes/no)? yes
>> Warning: Permanently added '192.168.0.10' (RSA) to the list of known hosts.
>> [  153.270000] ixp_newsession()
>> [  153.270000] ixp_newsession()
>> [  153.280000] ixp_process()
>> [  153.280000] ixp_register_cb(-1031187288, 00000000, 0)
>> [  153.280000] ixp_process_pending(c3575400)
>> [  153.290000] ixp_process_pending in while
>> [  153.290000] ixp_q_process(c28949e0)
>> evp_crypt: EVP_Cipher failed
>> [  153.300000] ixp_freesession()
>> [  153.300000] ixp_freesession()
>>
>> but when I run ssh without running openssl speed before I got the freeze 
>> in ixp_q_process.
>>
>> Do you use some patch for ssh too? I found some patches which adds some 
>>  openssl engine initialization to ssh but it won't helped.
> 
> Sorry I took so long to get back to you on this.
> 
> I have just gone through the above sequences here and ssh works fine for
> me in both cases.  I am not getting the "evp_crypt: EVP_Cipher failed"
> error either.  I tried both into my ixp and out of the ixp board.
> 
> Which versions of openssl/openssh are you using ?  Not that it should
> really matter.  I am running with  CSR 2.4 and 0.9.8g (I've updated) and
> ssh 4.7p1,  probably also updated,  but no patches are needed for ssh
> IIRC.
> 
> I don't see any ixp_perform_cb,  to me this says the ixp crypto is not
> happy still.  Does the "openssl speed" test give you numbers comparable
> to that on the ocf-linux web page benchmarks section ?  If not it sounds
> like the crypto is just not working,  which comes back to the correct
> NPE code or the call to ixCryptoAccInit failing.  Perhaps add some debug
> to that code to make sure he crypto engine is started correctly.  Look
> for a kernel message:
> 
> 	ixCryptoAccInit failed, assuming already initialised!
> 
> when loading the ixp4xx driver,  if you get that investigate further ;-)
> 
> I should have a new version out today (famous last words).  It might be
> good to try it out and see if there has been any improvements.  Quite a
> lot of potential locking/sleeping and other bus have been fixed and you
> may be just hitting one of those.

I have used the latest version but got the same. However I have finally 
found out what causes the freezes for me. It's something in ixp400_eth 
driver. When I use the driver in version 1.4 with ixp400 access lib 2.4 
the crypto works. When I use driver in version 1.6 or 1.7 it freezes. I 
also made a test and use ixp400_eth 1.6 with ixp400 access lib 2.0 and 
get the freeze (1.4 works perfect).
So it seems that the problem is being caused by something in ixp400_eth 
driver.
Unfortunately the number of changes between 1.4 and 1.6 is huge and I 
don't have the driver in version 1.5 to check it so it will be difficult 
to find out what causes problem :( As soon as I track it down I let you 
know what was wrong.

Regards,
Tomasz