Re: [Ocf-linux-users] PD: OCF's ixp4xx freeze on 2.6.18-rt7
Brought to you by:
david-m
Menu
▾
▴
Re: [Ocf-linux-users] PD: OCF's ixp4xx freeze on 2.6.18-rt7
|
From: Tomasz R. <tro...@pr...> - 2007-10-19 06:28:24
|
Hi David,
> Jivin Tomasz Rostanski lays it down ...
>> Hi David,
>>
>> Do you made any progress?
>
> Yep, sorry for the delay, just ran it up on 2.6.23 using CSR2.4 and
> ocf. Seems to be ok. Haven't tried ipsec yet, but openssl speed
> works fine (although throughput seems to be down).
>
> The driver is at least processing requests ok.
>
> Are you installing the correct NPE code with something like:
>
> cat /etc/IxNpeMicrocode.dat >/dev/ixNpe
I have NPE code compiled in. I have also made a build without compiling
it in, the result was the same.
Tomasz
>>> -----Wiadomość oryginalna-----
>>>> Od: David McCullough [mailto:Dav...@se...]
>>>> Wysłano: Pt 2007-09-28 15:25
>>>> Do: Rostanski, Tomasz
>>>> DW: ocf...@li...
>>>> Temat: Re: OCF's ixp4xx freeze on 2.6.18-rt7
>>>>
>>>>
>>>> Jivin Tomasz Rostanski lays it down ...
>>>>>> David,
>>>>>>
>>>>>>>> The latest I can try with easily is 2.3. Do you have that ?
>>>>>> We've been using version 2.3 but the OCF wasn't working too.
>> Some time
>>>>>> ago we have moved to the version 2.4 because we have found it
>> working
>>>>>> faster that the 2.3 (or maybe I have configured it better).
>>>> :-)
>>>>
>>>>>> I know the version 3.0 is released so I would like to try it
>> too. It
>>>>>> should have the ethernet driver written better, so perhaps
>> someday I
>>>>>> would need to go to 3.0.
>>>> Check the release notes for 3.0, ixp42x has been dropped, ixp43x and
>>>> ixp46x are still supported. Looks like 2.4 is the last for the
>>>> ixp420/425.
>>>>
>>>>>>>> I believe we have 2.4 going into our tree very soon now (week
>> or so)
>>>>>>>> so I can try it then.
>>>>>> Ok, so please let me know how would it work then.
>>>> Once it's in I'll let you know,
>>>>
>>>> Cheers,
>>>> Davidm
>>>>
>>>>>>>>>> David McCullough pisze:
>>>>>>>>>>>> Jivin Tomasz Rostanski lays it down ...
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It seems that the crypto_done is never been called, so
>> the loop is
>>>>>>>>>>>>>> waiting infinitive and the device freezes.
>>>>>>>>>>>>>> So it definitelly could as you have described. However I
>> have checked
>>>>>>>>>>>>>> the function ixCryptoAccAuthCryptPerform to see if it
>> has changed
>>>> since
>>>>>>>>>>>>>> 2.0 version of the library but is exactly the same.
>>>>>>>>>>>>>> I'll try to find what are the differences in crypto part
>> of the
>>>> library
>>>>>>>>>>>>>> - maybe there I will find the clue.
>>>>>>>>>>>> Lets hope. We have 2.1 running here (not sure if thats in
>> your tree or
>>>>>>>>>>>> not). It seems to be behaving so far. Mostly IXP465
>> testing though.
>>>>>>>>>>>>>> btw. The ixpCryptoAccCtxRegister failed 6 error I got
>> when I tried to
>>>>>>>>>>>>>> run this on another board - I have returned to my
>> current one ;)
>>>>>>>>>>>> Ok, best I can tell, 6 = ALG unsupported, which doesn't
>> really make
>>>>>>>>>>>> sense, but perhaps that helps you. You can find the enum
>> (grrr enum
>>>>>>>>>>>> errors) in:
>>>>>>>>>>>>
>>>>>>>>>>>> ixp400-1.4/ixp400_xscale_sw/src/include/IxCryptoAcc.h
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>> Davidm
>>>>>>>>>>>>
>>>>>>>>>>>>>> David McCullough wrote:
>>>>>>>>>>>>>>>> Jivin Tomasz Rostanski lays it down ...
>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I have debug the code a bit and found where the
>> freeze is.
>>>>>>>>>>>>>>>>>> The waiting code in cryptodev_op is never ending
>> from the while
>>>> loop:
>>>>>>>>>>>>>>>>>> do {
>>>>>>>>>>>>>>>>>> wait_event_interruptible(...);
>>>>>>>>>>>>>>>>>> } while ((crp->crp_flags & CRYPTO_F_DONE) == 0);
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> When I set the rp->crp_flags to CRYPTO_F_DONE in
>> ixp_q_process
>>>> before
>>>>>>>>>>>>>>>>>> return when status equals to
>> IX_CRYPTO_ACC_STATUS_SUCCESS the freeze
>>>>>>>>>>>>>>>>>> is not happening.
>>>>>>>>>>>>>>>>>> So maybe there is something is wrong with
>> wait_event_interruptible
>>>>>>>>>>>>>>>>>> function in my kernel or I don't know.
>>>>>>>>>>>>>>>> We have seen something like this just recently. A
>> hash of a 0 length
>>>>>>>>>>>>>>>> buffer would cause an error and the access lib would
>> not report it,
>>>> or
>>>>>>>>>>>>>>>> call the callback.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Any chance you are doing something like this ?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Setting CRYPTO_F_DONE in ixp_q_process when you get a
>>>>>>>>>>>>>>>> IX_CRYPTO_ACC_STATUS_SUCCESS is not good, you will be
>> returning
>>>> before
>>>>>>>>>>>>>>>> the operation is completed.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> CRYPTO_F_DONE is set by the call to crypto_done,
>> which happens on
>>>>>>>>>>>>>>>> error
>>>>>>>>>>>>>>>> or when the callback occurs.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I am guessing you are past this error as well:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> "ixpCryptoAccCtxRegister failed 6"
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>> Davidm
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> David McCullough napisa?(a):
>>>>>>>>>>>>>>>>>>>> Jivin Tomasz Rostanski lays it down ...
>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I have checked the latest ixp400 access lib
>> (2.4) and ixp_400
>>>>>>>>>>>>>>>>>>>>>>>>>> ethernet driver (1.7). I didn't use the
>> Intel patches for crypto
>>>>>>>>>>>>>>>>>>>>>>>>>> initialization in ixp400_eth.
>>>>>>>>>>>>>>>>>>>>>>>>>> The result is still the same - the device
>> freezes when I run
>>>>>>>>>>>>>>>>>>>>>>>>>> openssl or ssh.
>>>>>>>>>>>>>>>>>>>>>>>>>> I'm having microcode compiled in driver -
>> load it from file to
>>>>>>>>>>>>>>>>>>>>>>>>>> /dev/ixNpe, so I have recompiled it and load
>> the microcode from
>>>>>>>>>>>>>>>>>>>>>>>>>> file but this didn't make any difference.
>>>>>>>>>>>>>>>>>>>>>>>> Are you using the "crypto version" of the
>> access library ?
>>>>>>>>>>>>>>>>>>>>>> Yes, I'm using the crypto version of the library.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Could this issue be related with the
>> hardware? I'm using the
>>>>>>>>>>>>>>>>>>>>>>>>>> AirTegrity box which was designed completely
>> by them and not
>>>> uses
>>>>>>>>>>>>>>>>>>>>>>>>>> redboot bootloader but their own one. Could
>> it be the reason of
>>>>>>>>>>>>>>>>>>>>>>>>>> the problem I'm having?
>>>>>>>>>>>>>>>>>>>>>>>> No, we do the same, it has no bearing on how
>> the NPE's do their
>>>>>>>>>>>>>>>>>>>>>>>> thing.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Are you loading the correct NPE code (with
>> crypto support etc).
>>>>>>>>>>>>>>>>>>>>>> Normally I have NPE code compiled in the
>> sources. I'm using the
>>>>>>>>>>>>>>>>>>>>>> version with crypto support so it should be ok.
>>>>>>>>>>>>>>>>>>>> ok.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I'll give a try on Gateworks GW2347 and see
>> if it will be
>>>> working
>>>>>>>>>>>>>>>>>>>>>>>>>> as it should or not.
>>>>>>>>>>>>>>>>>>>>>> Ok, I have a problem with initialization crypto
>> on the gw2347 -
>>>> the
>>>>>>>>>>>>>>>>>>>>>> message:
>>>>>>>>>>>>>>>>>>>>>> "ixpCryptoAccCtxRegister failed 6" appears when
>> I run openssl - I
>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>> Thats sounds weird. It sounds like the crypto
>> isn't working
>>>> properly
>>>>>>>>>>>>>>>>>>>> for some reason.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> ixpCryptoAccCtxRegister doesn't need much to work.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> checked and the ixp4xx module when loading
>> returns the error that
>>>>>>>>>>>>>>>>>>>>>> cannot initialize crypto.
>>>>>>>>>>>>>>>>>>>> That is ok, if the network driver init's the
>> crypto then you will
>>>>>>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>>> this error since we cannot tell that it has been
>> done, yet it
>>>>>>>>>>>>>>>>>>>> returns a
>>>>>>>>>>>>>>>>>>>> fail if it is already initted.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Might be worth checking the details of the error.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>>>>> Davidm
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> David McCullough napisa?(a):
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jivin Tomasz Rostanski lays it down ...
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Which access library version are you
>> using ?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The full name is
>> ixp400accesslibrarywithcrypto-2.3.1
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Ok, not sure we have tested OCF with
>> version 2.3 of the access
>>>>>>>>>>>>>>>>>>>>>>>>>>>> lib.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is part of our tree so I will try it.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Not sure you mentioned which sources you
>> are using, but the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> uClinux-dist or SnapGear dists have very
>> good IXP425 support
>>>>>>>>>>>>>>>>>>>>>>>>>>>> and less patching ;-)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you load ocf, cryptodev and ixp4xx
>> all with debug
>>>> enabled:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> modprobe ocf crypto_debug=1
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> modprobe cryptodev cryptodev_debug=1
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> modprobe ixp4xx ixp_debug=1
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The run the test, capture all the
>> console output and send me
>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> copy
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for reference.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The output is attached (ocf.log).
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Ok, I will try and get some time on it in
>> the next day or so,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> thanks.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Actually, just had a quick look, I think
>> the access lib is
>>>>>>>>>>>>>>>>>>>>>>>>>>>> blowing the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> stack. It is renowned for doing this.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> If you can try increasing the kernel stack
>> size to 8K if it
>>>> isn't
>>>>>>>>>>>>>>>>>>>>>>>>>>>> already it might help.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Otherwise, you may find the info to fix
>> it in our access lib
>>>>>>>>>>>>>>>>>>>>>>>>>>>> patch:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> http://ftp.snapgear.org/pub/snapgear/src/snapgear-modules-20061012.sh
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Or perhaps try the Snapgear/uClinux-dist
>> if you have time. We
>>>>>>>>>>>>>>>>>>>>>>>>>>>> have all
>>>>>>>>>>>>>>>>>>>>>>>>>>>> these sort or problems sorted already ;-)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Have you tried turning off preemption
>> ? I don't think I have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ever
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tested with that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, I have tried but didn't help :(
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oh well, looks like a real bug then ;-)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Davidm
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David McCullough napisa?(a):
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jivin Tomasz Rostanski lays it
>> down ...
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I tried using ixp4xx module for
>> hardware crypto on my
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ixp425 device with kernel
>> 2.6.18-rt7 (PREEMPT_DESKTOP).
>>>> I'm
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> using ixp400 access library with
>> crypto in version 2.3.1
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and ixp400_eth driver in version
>> 1.6 (with Intel's patch
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for OCF support -
>>>>>>>>>>>>>>>>>> http://downloadcenter.intel.com/detail_desc.aspx?ProductID=2100&DwnldID=11266&agr=Y).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm loading the modules like
>> described in Intel readme:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mknod /dev/crypto c 10 70
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mknod -m 666 /dev/ixNpe c 241 0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> modprobe ixp400 >/dev/null
>> 2>/dev/null
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> modprobe ixp400_eth >/dev/null
>> 2>/dev/null
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> modprobe ocf
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> modprobe cryptodev
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> modprobe ixp4xx ixp_init_crypto=0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Then when I run the: openssl
>> speed -elapsed -evp
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> des-ede3-cbc -cpu -engine
>> cryptodev the device hangs (I'm
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> using openssl-0.9.8a patched for
>> OCF). The same happend
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when I tried to connect from the
>> device using ssh.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I have enabled debugging and saw
>> that the debugging from
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ixp_q_process is the last one
>> displayed. So I have started
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> adding some debug messages to
>> that function and found that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hand appears after the
>> following code:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if (IX_CRYPTO_ACC_STATUS_SUCCESS
>> == status)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> return;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So I have changed return to goto
>> done and check what will
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> happen - this time the openssl
>> didn't hang and did it's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work. But the ssh is not working
>> - displays evp_crypt:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> EVP_Cipher failed and exits.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I have tried the cryptosoft
>> module instead of ixp4xx and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this one works without any
>> problems, so it seems that some
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> problem exists in ixp4xx module.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Do you have any clue what could
>> be wrong? I'm almost sure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that someday on older kernel
>> (2.6.12) I got ixp4xx working
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> without problems.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You might want to try
>> incorporating the latest code from
>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sourceforge
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> site. I haven't seen a like up
>> like you describe on the
>>>> ixp
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for a long
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time and I don't know exactly
>> everything that is in the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Intel patches.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Try out the latest download at:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ocf-linux.sourceforge.net
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and then it will be much easier
>> for me to help you. I run
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> IXP boards
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> here and can easily try some things,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Davidm
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> --
>>>> David McCullough, dav...@se..., Ph:+61
>> 734352815
>>>> Secure Computing - SnapGear http://www.uCdot.org
>> http://www.cyberguard.com
>>>>
>>>>
>
|
