Re: [Ocf-linux-users] OCF's ixp4xx freeze on 2.6.18-rt7

[Tomasz R. <tro...@pr...>]

David,

> The latest I can try with easily is 2.3.  Do you have that ?

We've been using version 2.3 but the OCF wasn't working too. Some time 
ago we have moved to the version 2.4 because we have found it working 
faster that the 2.3 (or maybe I have configured it better).
I know the version 3.0 is released so I would like to try it too. It 
should have the ethernet driver written better, so perhaps someday I 
would need to go to 3.0.

> I believe we have 2.4 going into our tree very soon now (week or so)
> so I can try it then.

Ok, so please let me know how would it work then.

Thanks,
Tomasz

>> David McCullough pisze:
>>> Jivin Tomasz Rostanski lays it down ...
>>>> Hi,
>>>>
>>>> It seems that the crypto_done is never been called, so the loop is 
>>>> waiting infinitive and the device freezes.
>>>> So it definitelly could as you have described. However I have checked 
>>>> the function ixCryptoAccAuthCryptPerform to see if it has changed since 
>>>> 2.0 version of the library but is exactly the same.
>>>> I'll try to find what are the differences in crypto part of the library 
>>>> - maybe there I will find the clue.
>>> Lets hope.  We have 2.1 running here (not sure if thats in your tree or
>>> not).  It seems to be behaving so far.  Mostly IXP465 testing though.
>>>
>>>> btw. The ixpCryptoAccCtxRegister failed 6 error I got when I tried to 
>>>> run this on another board - I have returned to my current one ;)
>>> Ok,  best I can tell, 6 = ALG unsupported, which doesn't really make
>>> sense,  but perhaps that helps you.  You can find the enum (grrr enum
>>> errors) in:
>>>
>>> 	ixp400-1.4/ixp400_xscale_sw/src/include/IxCryptoAcc.h
>>>
>>> Cheers,
>>> Davidm
>>>
>>>> David McCullough wrote:
>>>>> Jivin Tomasz Rostanski lays it down ...
>>>>>> Hi,
>>>>>>
>>>>>> I have debug the code a bit and found where the freeze is.
>>>>>> The waiting code in cryptodev_op is never ending from the while loop:
>>>>>> do {
>>>>>> 	wait_event_interruptible(...);
>>>>>> } while ((crp->crp_flags & CRYPTO_F_DONE) == 0);
>>>>>>
>>>>>> When I set the rp->crp_flags to CRYPTO_F_DONE in ixp_q_process before 
>>>>>> return when status equals to IX_CRYPTO_ACC_STATUS_SUCCESS the freeze is 
>>>>>> not happening.
>>>>>> So maybe there is something is wrong with wait_event_interruptible 
>>>>>> function in my kernel or I don't know.
>>>>> We have seen something like this just recently.  A hash of a 0 length
>>>>> buffer would cause an error and the access lib would not report it,  or
>>>>> call the callback.
>>>>>
>>>>> Any chance you are doing something like this ?
>>>>>
>>>>> Setting CRYPTO_F_DONE in ixp_q_process when you get a
>>>>> IX_CRYPTO_ACC_STATUS_SUCCESS is not good,  you will be returning before
>>>>> the operation is completed.
>>>>>
>>>>> CRYPTO_F_DONE is set by the call to crypto_done,  which happens on error
>>>>> or when the callback occurs.
>>>>>
>>>>> I am guessing you are past this error as well:
>>>>>
>>>>> 	"ixpCryptoAccCtxRegister failed 6"
>>>>>
>>>>> Cheers,
>>>>> Davidm
>>>>>
>>>>>
>>>>>> David McCullough napisa?(a):
>>>>>>> Jivin Tomasz Rostanski lays it down ...
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>>>> I have checked the latest ixp400 access lib (2.4) and ixp_400 
>>>>>>>>>> ethernet driver (1.7). I didn't use the Intel patches for crypto 
>>>>>>>>>> initialization in ixp400_eth.
>>>>>>>>>> The result is still the same - the device freezes when I run 
>>>>>>>>>> openssl or ssh.
>>>>>>>>>> I'm having microcode compiled in driver - load it from file to 
>>>>>>>>>> /dev/ixNpe, so I have recompiled it and load the microcode from 
>>>>>>>>>> file but this didn't make any difference.
>>>>>>>>> Are you using the "crypto version" of the access library ?
>>>>>>>> Yes, I'm using the crypto version of the library.
>>>>>>>>
>>>>>>>>>> Could this issue be related with the hardware? I'm using the 
>>>>>>>>>> AirTegrity box which was designed completely by them and not uses 
>>>>>>>>>> redboot bootloader but their own one. Could it be the reason of the 
>>>>>>>>>> problem I'm having?
>>>>>>>>> No,  we do the same,  it has no bearing on how the NPE's do their 
>>>>>>>>> thing.
>>>>>>>>>
>>>>>>>>> Are you loading the correct NPE code (with crypto support etc).
>>>>>>>> Normally I have NPE code compiled in the sources. I'm using the 
>>>>>>>> version with crypto support so it should be ok.
>>>>>>> ok.
>>>>>>>
>>>>>>>>>> I'll give a try on Gateworks GW2347 and see if it will be working 
>>>>>>>>>> as it should or not.
>>>>>>>> Ok, I have a problem with initialization crypto on the gw2347 - the 
>>>>>>>> message:
>>>>>>>> "ixpCryptoAccCtxRegister failed 6" appears when I run openssl - I 
>>>>>>>> have 
>>>>>>> Thats sounds weird.  It sounds like the crypto isn't working properly
>>>>>>> for some reason.
>>>>>>>
>>>>>>> ixpCryptoAccCtxRegister doesn't need much to work.
>>>>>>>
>>>>>>>> checked and the ixp4xx module when loading returns the error that 
>>>>>>>> cannot initialize crypto.
>>>>>>> That is ok,  if the network driver init's the crypto then you will get
>>>>>>> this error since we cannot tell that it has been done,  yet it returns 
>>>>>>> a
>>>>>>> fail if it is already initted.
>>>>>>>
>>>>>>> Might be worth checking the details of the error.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Davidm
>>>>>>>
>>>>>>>>>> David McCullough napisa?(a):
>>>>>>>>>>> Jivin Tomasz Rostanski lays it down ...
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>>> Which access library version are you using ?
>>>>>>>>>>>> The full name is ixp400accesslibrarywithcrypto-2.3.1
>>>>>>>>>>> Ok,  not sure we have tested OCF with version 2.3 of the access 
>>>>>>>>>>> lib.
>>>>>>>>>>> It is part of our tree so I will try it.
>>>>>>>>>>>
>>>>>>>>>>> Not sure you mentioned which sources you are using,  but the
>>>>>>>>>>> uClinux-dist or SnapGear dists have very good IXP425 support
>>>>>>>>>>> and less patching ;-)
>>>>>>>>>>>
>>>>>>>>>>>>> Can you load ocf, cryptodev and ixp4xx all with debug enabled:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 	modprobe ocf crypto_debug=1
>>>>>>>>>>>>> 	modprobe cryptodev cryptodev_debug=1
>>>>>>>>>>>>> 	modprobe ixp4xx ixp_debug=1
>>>>>>>>>>>>>
>>>>>>>>>>>>> The run the test, capture all the console output and send me a 
>>>>>>>>>>>>> copy
>>>>>>>>>>>>> for reference.
>>>>>>>>>>>> The output is attached (ocf.log).
>>>>>>>>>>> Ok,  I will try and get some time on it in the next day or so, 
>>>>>>>>>>> thanks.
>>>>>>>>>>>
>>>>>>>>>>> Actually,  just had a quick look,  I think the access lib is 
>>>>>>>>>>> blowing the
>>>>>>>>>>> stack.  It is renowned for doing this.
>>>>>>>>>>>
>>>>>>>>>>> If you can try increasing the kernel stack size to 8K if it isn't
>>>>>>>>>>> already it might help.
>>>>>>>>>>>
>>>>>>>>>>> Otherwise,  you may find the info to fix it in our access lib 
>>>>>>>>>>> patch:
>>>>>>>>>>>
>>>>>>>>>>> 	http://ftp.snapgear.org/pub/snapgear/src/snapgear-modules-20061012.sh
>>>>>>>>>>>
>>>>>>>>>>> Or perhaps try the Snapgear/uClinux-dist if you have time.  We 
>>>>>>>>>>> have all
>>>>>>>>>>> these sort or problems sorted already ;-)
>>>>>>>>>>>
>>>>>>>>>>>>> Have you tried turning off preemption ?  I don't think I have 
>>>>>>>>>>>>> ever
>>>>>>>>>>>>> tested with that.
>>>>>>>>>>>> Yes, I have tried but didn't help :(
>>>>>>>>>>> Oh well,  looks like a real bug then ;-)
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>> Davidm
>>>>>>>>>>>
>>>>>>>>>>>>>> David McCullough napisa?(a):
>>>>>>>>>>>>>>> Jivin Tomasz Rostanski lays it down ...
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I tried using ixp4xx module for hardware crypto on my ixp425 
>>>>>>>>>>>>>>>> device with kernel 2.6.18-rt7 (PREEMPT_DESKTOP). I'm using 
>>>>>>>>>>>>>>>> ixp400 access library with crypto in version 2.3.1 and 
>>>>>>>>>>>>>>>> ixp400_eth driver in version 1.6 (with Intel's patch for OCF 
>>>>>>>>>>>>>>>> support - 
>>>>>>>>>>>>>>>> http://downloadcenter.intel.com/detail_desc.aspx?ProductID=2100&DwnldID=11266&agr=Y).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm loading the modules like described in Intel readme:
>>>>>>>>>>>>>>>> mknod /dev/crypto c 10 70
>>>>>>>>>>>>>>>> mknod -m 666 /dev/ixNpe c 241 0
>>>>>>>>>>>>>>>> modprobe ixp400 >/dev/null 2>/dev/null
>>>>>>>>>>>>>>>> modprobe ixp400_eth >/dev/null 2>/dev/null
>>>>>>>>>>>>>>>> modprobe ocf
>>>>>>>>>>>>>>>> modprobe cryptodev
>>>>>>>>>>>>>>>> modprobe ixp4xx ixp_init_crypto=0
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Then when I run the: openssl speed -elapsed -evp des-ede3-cbc 
>>>>>>>>>>>>>>>> -cpu -engine cryptodev the device hangs (I'm using 
>>>>>>>>>>>>>>>> openssl-0.9.8a patched for OCF). The same happend when I 
>>>>>>>>>>>>>>>> tried to connect from the device using ssh.
>>>>>>>>>>>>>>>> I have enabled debugging and saw that the debugging from 
>>>>>>>>>>>>>>>> ixp_q_process is the last one displayed. So I have started 
>>>>>>>>>>>>>>>> adding some debug messages to that function and found that 
>>>>>>>>>>>>>>>> the hand appears after the following code:
>>>>>>>>>>>>>>>> if (IX_CRYPTO_ACC_STATUS_SUCCESS == status)
>>>>>>>>>>>>>>>> 	return;
>>>>>>>>>>>>>>>> So I have changed return to goto done and check what will 
>>>>>>>>>>>>>>>> happen - this time the openssl didn't hang and did it's work. 
>>>>>>>>>>>>>>>> But the ssh is not working - displays evp_crypt: EVP_Cipher 
>>>>>>>>>>>>>>>> failed and exits.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I have tried the cryptosoft module instead of ixp4xx and this 
>>>>>>>>>>>>>>>> one works without any problems, so it seems that some problem 
>>>>>>>>>>>>>>>> exists in ixp4xx module.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Do you have any clue what could be wrong? I'm almost sure 
>>>>>>>>>>>>>>>> that someday on older kernel (2.6.12) I got ixp4xx working 
>>>>>>>>>>>>>>>> without problems.
>>>>>>>>>>>>>>> You might want to try incorporating the latest code from the 
>>>>>>>>>>>>>>> sourceforge
>>>>>>>>>>>>>>> site.  I haven't seen a like up like you describe on the ixp 
>>>>>>>>>>>>>>> for a long
>>>>>>>>>>>>>>> time and I don't know exactly everything that is in the Intel 
>>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Try out the latest download at:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 	ocf-linux.sourceforge.net
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> and then it will be much easier for me to help you.  I run IXP 
>>>>>>>>>>>>>>> boards
>>>>>>>>>>>>>>> here and can easily try some things,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>> Davidm
>>>>>>>>>>>>>>>
>