Re: [Ocf-linux-users] OCF's ixp4xx freeze on 2.6.18-rt7

[Tomasz R. <tro...@pr...>]

David,

I have also noticed that the ixp_process_pending have only one element 
on the list. This function is run only once. Is it normal behaviour or not?
I have add a call to crypto_done just before the end of the function but 
I got no change at all - as openssl was freezing still is freezing.
If you have any clue or idea which piece of code should I debug closer 
please let me know.

Thanks,
Tomasz

David McCullough pisze:
> Jivin Tomasz Rostanski lays it down ...
>> Hi,
>>
>> It seems that the crypto_done is never been called, so the loop is 
>> waiting infinitive and the device freezes.
>> So it definitelly could as you have described. However I have checked 
>> the function ixCryptoAccAuthCryptPerform to see if it has changed since 
>> 2.0 version of the library but is exactly the same.
>> I'll try to find what are the differences in crypto part of the library 
>> - maybe there I will find the clue.
> 
> Lets hope.  We have 2.1 running here (not sure if thats in your tree or
> not).  It seems to be behaving so far.  Mostly IXP465 testing though.
> 
>> btw. The ixpCryptoAccCtxRegister failed 6 error I got when I tried to 
>> run this on another board - I have returned to my current one ;)
> 
> Ok,  best I can tell, 6 = ALG unsupported, which doesn't really make
> sense,  but perhaps that helps you.  You can find the enum (grrr enum
> errors) in:
> 
> 	ixp400-1.4/ixp400_xscale_sw/src/include/IxCryptoAcc.h
> 
> Cheers,
> Davidm
> 
>> David McCullough wrote:
>>> Jivin Tomasz Rostanski lays it down ...
>>>> Hi,
>>>>
>>>> I have debug the code a bit and found where the freeze is.
>>>> The waiting code in cryptodev_op is never ending from the while loop:
>>>> do {
>>>> 	wait_event_interruptible(...);
>>>> } while ((crp->crp_flags & CRYPTO_F_DONE) == 0);
>>>>
>>>> When I set the rp->crp_flags to CRYPTO_F_DONE in ixp_q_process before 
>>>> return when status equals to IX_CRYPTO_ACC_STATUS_SUCCESS the freeze is 
>>>> not happening.
>>>> So maybe there is something is wrong with wait_event_interruptible 
>>>> function in my kernel or I don't know.
>>>
>>> We have seen something like this just recently.  A hash of a 0 length
>>> buffer would cause an error and the access lib would not report it,  or
>>> call the callback.
>>>
>>> Any chance you are doing something like this ?
>>>
>>> Setting CRYPTO_F_DONE in ixp_q_process when you get a
>>> IX_CRYPTO_ACC_STATUS_SUCCESS is not good,  you will be returning before
>>> the operation is completed.
>>>
>>> CRYPTO_F_DONE is set by the call to crypto_done,  which happens on error
>>> or when the callback occurs.
>>>
>>> I am guessing you are past this error as well:
>>>
>>> 	"ixpCryptoAccCtxRegister failed 6"
>>>
>>> Cheers,
>>> Davidm
>>>
>>>
>>>> David McCullough napisa?(a):
>>>>> Jivin Tomasz Rostanski lays it down ...
>>>>>> Hi,
>>>>>>
>>>>>>>> I have checked the latest ixp400 access lib (2.4) and ixp_400 
>>>>>>>> ethernet driver (1.7). I didn't use the Intel patches for crypto 
>>>>>>>> initialization in ixp400_eth.
>>>>>>>> The result is still the same - the device freezes when I run openssl 
>>>>>>>> or ssh.
>>>>>>>> I'm having microcode compiled in driver - load it from file to 
>>>>>>>> /dev/ixNpe, so I have recompiled it and load the microcode from file 
>>>>>>>> but this didn't make any difference.
>>>>>>> Are you using the "crypto version" of the access library ?
>>>>>> Yes, I'm using the crypto version of the library.
>>>>>>
>>>>>>>> Could this issue be related with the hardware? I'm using the 
>>>>>>>> AirTegrity box which was designed completely by them and not uses 
>>>>>>>> redboot bootloader but their own one. Could it be the reason of the 
>>>>>>>> problem I'm having?
>>>>>>> No,  we do the same,  it has no bearing on how the NPE's do their 
>>>>>>> thing.
>>>>>>>
>>>>>>> Are you loading the correct NPE code (with crypto support etc).
>>>>>> Normally I have NPE code compiled in the sources. I'm using the version 
>>>>>> with crypto support so it should be ok.
>>>>> ok.
>>>>>
>>>>>>>> I'll give a try on Gateworks GW2347 and see if it will be working as 
>>>>>>>> it should or not.
>>>>>> Ok, I have a problem with initialization crypto on the gw2347 - the 
>>>>>> message:
>>>>>> "ixpCryptoAccCtxRegister failed 6" appears when I run openssl - I have 
>>>>> Thats sounds weird.  It sounds like the crypto isn't working properly
>>>>> for some reason.
>>>>>
>>>>> ixpCryptoAccCtxRegister doesn't need much to work.
>>>>>
>>>>>> checked and the ixp4xx module when loading returns the error that 
>>>>>> cannot initialize crypto.
>>>>> That is ok,  if the network driver init's the crypto then you will get
>>>>> this error since we cannot tell that it has been done,  yet it returns a
>>>>> fail if it is already initted.
>>>>>
>>>>> Might be worth checking the details of the error.
>>>>>
>>>>> Cheers,
>>>>> Davidm
>>>>>
>>>>>>>> David McCullough napisa?(a):
>>>>>>>>> Jivin Tomasz Rostanski lays it down ...
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>>> Which access library version are you using ?
>>>>>>>>>> The full name is ixp400accesslibrarywithcrypto-2.3.1
>>>>>>>>> Ok,  not sure we have tested OCF with version 2.3 of the access lib.
>>>>>>>>> It is part of our tree so I will try it.
>>>>>>>>>
>>>>>>>>> Not sure you mentioned which sources you are using,  but the
>>>>>>>>> uClinux-dist or SnapGear dists have very good IXP425 support
>>>>>>>>> and less patching ;-)
>>>>>>>>>
>>>>>>>>>>> Can you load ocf, cryptodev and ixp4xx all with debug enabled:
>>>>>>>>>>>
>>>>>>>>>>> 	modprobe ocf crypto_debug=1
>>>>>>>>>>> 	modprobe cryptodev cryptodev_debug=1
>>>>>>>>>>> 	modprobe ixp4xx ixp_debug=1
>>>>>>>>>>>
>>>>>>>>>>> The run the test, capture all the console output and send me a copy
>>>>>>>>>>> for reference.
>>>>>>>>>> The output is attached (ocf.log).
>>>>>>>>> Ok,  I will try and get some time on it in the next day or so, 
>>>>>>>>> thanks.
>>>>>>>>>
>>>>>>>>> Actually,  just had a quick look,  I think the access lib is blowing 
>>>>>>>>> the
>>>>>>>>> stack.  It is renowned for doing this.
>>>>>>>>>
>>>>>>>>> If you can try increasing the kernel stack size to 8K if it isn't
>>>>>>>>> already it might help.
>>>>>>>>>
>>>>>>>>> Otherwise,  you may find the info to fix it in our access lib patch:
>>>>>>>>>
>>>>>>>>> 	http://ftp.snapgear.org/pub/snapgear/src/snapgear-modules-20061012.sh
>>>>>>>>>
>>>>>>>>> Or perhaps try the Snapgear/uClinux-dist if you have time.  We have 
>>>>>>>>> all
>>>>>>>>> these sort or problems sorted already ;-)
>>>>>>>>>
>>>>>>>>>>> Have you tried turning off preemption ?  I don't think I have ever
>>>>>>>>>>> tested with that.
>>>>>>>>>> Yes, I have tried but didn't help :(
>>>>>>>>> Oh well,  looks like a real bug then ;-)
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Davidm
>>>>>>>>>
>>>>>>>>>>>> David McCullough napisa?(a):
>>>>>>>>>>>>> Jivin Tomasz Rostanski lays it down ...
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I tried using ixp4xx module for hardware crypto on my ixp425 
>>>>>>>>>>>>>> device with kernel 2.6.18-rt7 (PREEMPT_DESKTOP). I'm using 
>>>>>>>>>>>>>> ixp400 access library with crypto in version 2.3.1 and 
>>>>>>>>>>>>>> ixp400_eth driver in version 1.6 (with Intel's patch for OCF 
>>>>>>>>>>>>>> support - 
>>>>>>>>>>>>>> http://downloadcenter.intel.com/detail_desc.aspx?ProductID=2100&DwnldID=11266&agr=Y).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm loading the modules like described in Intel readme:
>>>>>>>>>>>>>> mknod /dev/crypto c 10 70
>>>>>>>>>>>>>> mknod -m 666 /dev/ixNpe c 241 0
>>>>>>>>>>>>>> modprobe ixp400 >/dev/null 2>/dev/null
>>>>>>>>>>>>>> modprobe ixp400_eth >/dev/null 2>/dev/null
>>>>>>>>>>>>>> modprobe ocf
>>>>>>>>>>>>>> modprobe cryptodev
>>>>>>>>>>>>>> modprobe ixp4xx ixp_init_crypto=0
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Then when I run the: openssl speed -elapsed -evp des-ede3-cbc 
>>>>>>>>>>>>>> -cpu -engine cryptodev the device hangs (I'm using 
>>>>>>>>>>>>>> openssl-0.9.8a patched for OCF). The same happend when I tried 
>>>>>>>>>>>>>> to connect from the device using ssh.
>>>>>>>>>>>>>> I have enabled debugging and saw that the debugging from 
>>>>>>>>>>>>>> ixp_q_process is the last one displayed. So I have started 
>>>>>>>>>>>>>> adding some debug messages to that function and found that the 
>>>>>>>>>>>>>> hand appears after the following code:
>>>>>>>>>>>>>> if (IX_CRYPTO_ACC_STATUS_SUCCESS == status)
>>>>>>>>>>>>>> 	return;
>>>>>>>>>>>>>> So I have changed return to goto done and check what will 
>>>>>>>>>>>>>> happen - this time the openssl didn't hang and did it's work. 
>>>>>>>>>>>>>> But the ssh is not working - displays evp_crypt: EVP_Cipher 
>>>>>>>>>>>>>> failed and exits.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have tried the cryptosoft module instead of ixp4xx and this 
>>>>>>>>>>>>>> one works without any problems, so it seems that some problem 
>>>>>>>>>>>>>> exists in ixp4xx module.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Do you have any clue what could be wrong? I'm almost sure that 
>>>>>>>>>>>>>> someday on older kernel (2.6.12) I got ixp4xx working without 
>>>>>>>>>>>>>> problems.
>>>>>>>>>>>>> You might want to try incorporating the latest code from the 
>>>>>>>>>>>>> sourceforge
>>>>>>>>>>>>> site.  I haven't seen a like up like you describe on the ixp for 
>>>>>>>>>>>>> a long
>>>>>>>>>>>>> time and I don't know exactly everything that is in the Intel 
>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Try out the latest download at:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 	ocf-linux.sourceforge.net
>>>>>>>>>>>>>
>>>>>>>>>>>>> and then it will be much easier for me to help you.  I run IXP 
>>>>>>>>>>>>> boards
>>>>>>>>>>>>> here and can easily try some things,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>> Davidm
>>>>>>>>>>>>>
>