Thread: Re: [Ocf-linux-users] URGENT bug.

Brought to you by: david-m

ocf-linux-users

Re: [Ocf-linux-users] URGENT bug.

From: David M. <Dav...@se...> - 2007-12-10 10:45:06

Jivin Nawang Chhetan lays it down ...
> Hi David,
>              I browsed through the swcr_newsession code. I realized there is
> something seriously wrong.
> I am attaching dmesg dump, which clearly indicates where the following bug
> message is generated
>             BUG: sleeping function called from invalid context at
> kernel/rwsem.c:20

Is this with pre-emption turned on still ?

Thanks,
David

PS
	I wil get to your other mail,  just very busy at the moment :-(

> I am using OCF with cryptosoft. Basically this result because we take a lock
> in cryto_newsession before calling the swcr_newsession. swcr_newsession
> further calls a linux crypto library function which is sleeping.
> 
> ******************
> dmesg
> [  524.873107] crypto/ocf/crypto.c,354: DRIVER_LOCK()
> [  524.873120] swcr_newsession()
> [  524.873127] swcr_newsession crypto_alloc_blkcipher(cbc(aes), 0x0)
> [  524.873136] BUG: sleeping function called from invalid context at
> kernel/rwsem.c:20
> [  524.880781] in_atomic():0, irqs_disabled():1
> [  524.880825]  [<c013fa32>] down_read+0x12/0x20
> [  524.880845]  [<c01e8bce>] crypto_alg_lookup+0x1e/0x50
> [  524.880865]  [<c01e8c30>] crypto_alg_mod_lookup+0x30/0x260
> [  524.880891]  [<c01e8ea6>] crypto_alloc_base+0x26/0x80
> [  524.880914]  [<e0ae2f6d>] swcr_newsession+0x1cd/0x770 [cryptosoft]
> [  524.880964]  [<e09f8351>] crypto_newsession+0x1d1/0x240 [ocf]
> ******************
> 
> Please comment.
> 
> -- 
> Nawang Chhetan
> Software Engineer
> SafeNet India.

-- 
David McCullough,  dav...@se...,   Ph:+61 734352815
Secure Computing - SnapGear  http://www.uCdot.org http://www.cyberguard.com

Re: [Ocf-linux-users] URGENT bug.

From: David M. <Dav...@se...> - 2007-12-10 10:45:09

Jivin Nawang Chhetan lays it down ...
> No.

Ok,  what sort of load/testing  produced it ?

I ran OCF + cryptosoft through our benchmark suite on an IXP425 (arm-be)
just a couple of days ago without problems.

I can send you the diffs against that last release in case there is
something we have fixed in there.  The trick is the openswan diffs,
as I have moved up to 2.4.11 now,

Cheers,
Davidm

> 
> On Dec 10, 2007 4:07 PM, David McCullough <
> Dav...@se...> wrote:
> 
> >
> > Jivin Nawang Chhetan lays it down ...
> > > Hi David,
> > >              I browsed through the swcr_newsession code. I realized
> > there is
> > > something seriously wrong.
> > > I am attaching dmesg dump, which clearly indicates where the following
> > bug
> > > message is generated
> > >             BUG: sleeping function called from invalid context at
> > > kernel/rwsem.c:20
> >
> > Is this with pre-emption turned on still ?
> >
> > Thanks,
> > David
> >
> > PS
> >        I wil get to your other mail,  just very busy at the moment :-(
> >
> > > I am using OCF with cryptosoft. Basically this result because we take a
> > lock
> > > in cryto_newsession before calling the swcr_newsession. swcr_newsession
> > > further calls a linux crypto library function which is sleeping.
> > >
> > > ******************
> > > dmesg
> > > [  524.873107] crypto/ocf/crypto.c,354: DRIVER_LOCK()
> > > [  524.873120] swcr_newsession()
> > > [  524.873127] swcr_newsession crypto_alloc_blkcipher(cbc(aes), 0x0)
> > > [  524.873136] BUG: sleeping function called from invalid context at
> > > kernel/rwsem.c:20
> > > [  524.880781] in_atomic():0, irqs_disabled():1
> > > [  524.880825]  [<c013fa32>] down_read+0x12/0x20
> > > [  524.880845]  [<c01e8bce>] crypto_alg_lookup+0x1e/0x50
> > > [  524.880865]  [<c01e8c30>] crypto_alg_mod_lookup+0x30/0x260
> > > [  524.880891]  [<c01e8ea6>] crypto_alloc_base+0x26/0x80
> > > [  524.880914]  [<e0ae2f6d>] swcr_newsession+0x1cd/0x770 [cryptosoft]
> > > [  524.880964]  [<e09f8351>] crypto_newsession+0x1d1/0x240 [ocf]
> > > ******************
> > >
> > > Please comment.
> > >
> > > --
> > > Nawang Chhetan
> > > Software Engineer
> > > SafeNet India.
> >
> > --
> > David McCullough,  dav...@se...,   Ph:+61
> > 734352815
> > Secure Computing - SnapGear  http://www.uCdot.org
> > http://www.cyberguard.com
> >
> 
> 
> 
> -- 
> Nawang Chhetan
> Software Engineer
> SafeNet India.

-- 
David McCullough,  dav...@se...,   Ph:+61 734352815
Secure Computing - SnapGear  http://www.uCdot.org http://www.cyberguard.com

Re: [Ocf-linux-users] OCF CBIMM=1 in SMP breaks

From: David M. <Dav...@se...> - 2007-12-13 01:28:31

Jivin Kabir Ahsan lays it down ...
> Hi David
> Sorry for writing directly. I tried using the openswan/ocf mailing list
> but I guess my posting is not showing up.
> Any how, I wanted to let you know that when I run OCF-based IPsec with
> CBIMM=1 in SMP mode for kernel 2.6.23 then I run spinlock recursion
> issue in the ipsec_rsm routine. This problem could be easily manifested
> using the OCF-null driver. 
> Is this a known bug? If yes, is there any recent patch to fix the
> problem?

That makes sense I guess,  as CBIMM means we call back into ipsec
before the call to do the request returns.  Makes the coding much
trickier to get right.

On an SMP kernel I would disable CBIMM in fact I would almost
always do that,  but I know that small low- cache systems can go
quite a bit faster using CBIMM,  so it stays.

If anyone want to suggest fixes for these it would be appreciated :-)
Otherwise I will try and review the locking when I get some time and
see if we can reduce the locking windows and cleanup CBIMM,

Cheers,
Davidm


-- 
David McCullough,  dav...@se...,   Ph:+61 734352815
Secure Computing - SnapGear  http://www.uCdot.org http://www.cyberguard.com

Re: [Ocf-linux-users] URGENT bug.

From: David M. <Dav...@se...> - 2007-12-13 01:45:10

Attachments: spinlock.diff

Jivin Nawang Chhetan lays it down ...
> HI David,
>              Please send me the diff for the changes, you made since last
> release. I am using a driver which calls crpto_newsession function from
> soft_irq context. When I do a simple ping the sessions are created fine with
> only the warning. But if I flood heavy traffic my machine hangs.
> 
> This is how I use.
> But in general don't you think it is not advisable, to take a spin lock and
> then call a sleeping fucntion, which what we do when we call
> crypto_newsession with cryptosoft as provider.

Ok,  thats was fairly easy to fix.  I do not run my kernels with
spinlock debug enabled,  adding that highlighted a couple of locking
issues and I have put some fixes in place.

See attached patch.

I am trying to get a release out this week (fixes, new openswan, new
drivers etc).

Cheers,
Davidm


> On Dec 10, 2007 4:18 PM, David McCullough <
> Dav...@se...> wrote:
> 
> >
> > Jivin Nawang Chhetan lays it down ...
> > > No.
> >
> > Ok,  what sort of load/testing  produced it ?
> >
> > I ran OCF + cryptosoft through our benchmark suite on an IXP425 (arm-be)
> > just a couple of days ago without problems.
> >
> > I can send you the diffs against that last release in case there is
> > something we have fixed in there.  The trick is the openswan diffs,
> > as I have moved up to 2.4.11 now,
> >
> > Cheers,
> > Davidm
> >
> > >
> > > On Dec 10, 2007 4:07 PM, David McCullough <
> > > Dav...@se...> wrote:
> > >
> > > >
> > > > Jivin Nawang Chhetan lays it down ...
> > > > > Hi David,
> > > > >              I browsed through the swcr_newsession code. I realized
> > > > there is
> > > > > something seriously wrong.
> > > > > I am attaching dmesg dump, which clearly indicates where the
> > following
> > > > bug
> > > > > message is generated
> > > > >             BUG: sleeping function called from invalid context at
> > > > > kernel/rwsem.c:20
> > > >
> > > > Is this with pre-emption turned on still ?
> > > >
> > > > Thanks,
> > > > David
> > > >
> > > > PS
> > > >        I wil get to your other mail,  just very busy at the moment :-(
> > > >
> > > > > I am using OCF with cryptosoft. Basically this result because we
> > take a
> > > > lock
> > > > > in cryto_newsession before calling the swcr_newsession.
> > swcr_newsession
> > > > > further calls a linux crypto library function which is sleeping.
> > > > >
> > > > > ******************
> > > > > dmesg
> > > > > [  524.873107] crypto/ocf/crypto.c,354: DRIVER_LOCK()
> > > > > [  524.873120] swcr_newsession()
> > > > > [  524.873127] swcr_newsession crypto_alloc_blkcipher(cbc(aes), 0x0)
> > > > > [  524.873136] BUG: sleeping function called from invalid context at
> > > > > kernel/rwsem.c:20
> > > > > [  524.880781] in_atomic():0, irqs_disabled():1
> > > > > [  524.880825]  [<c013fa32>] down_read+0x12/0x20
> > > > > [  524.880845]  [<c01e8bce>] crypto_alg_lookup+0x1e/0x50
> > > > > [  524.880865]  [<c01e8c30>] crypto_alg_mod_lookup+0x30/0x260
> > > > > [  524.880891]  [<c01e8ea6>] crypto_alloc_base+0x26/0x80
> > > > > [  524.880914]  [<e0ae2f6d>] swcr_newsession+0x1cd/0x770
> > [cryptosoft]
> > > > > [  524.880964]  [<e09f8351>] crypto_newsession+0x1d1/0x240 [ocf]
> > > > > ******************
> > > > >
> > > > > Please comment.
> > > > >
> > > > > --
> > > > > Nawang Chhetan
> > > > > Software Engineer
> > > > > SafeNet India.
> > > >
> > > > --
> > > > David McCullough,  dav...@se...,   Ph:+61
> > > > 734352815
> > > > Secure Computing - SnapGear  http://www.uCdot.org
> > > > http://www.cyberguard.com
> > > >
> > >
> > >
> > >
> > > --
> > > Nawang Chhetan
> > > Software Engineer
> > > SafeNet India.
> >
> > --
> > David McCullough,  dav...@se...,   Ph:+61
> > 734352815
> > Secure Computing - SnapGear  http://www.uCdot.org
> > http://www.cyberguard.com
> >
> 
> 
> 
> -- 
> Nawang Chhetan
> Software Engineer
> SafeNet India.

-- 
David McCullough,  dav...@se...,   Ph:+61 734352815
Secure Computing - SnapGear  http://www.uCdot.org http://www.cyberguard.com