linuxsh-dev

2.6.20 instability

[Manuel L. <ma...@ro...>]

Hello,

I wanted to ask if I'm the only one experiencing this:
With 2.6.20-rc, most apps I try to launch are killed
due to random segfaults. I tried to build udev
on the target itself; make V=1 would fail with
gcc simply segfaulting, sometimes /bin/sh can no longer
be found, sometimes parts of gcc's cmdline arguments
are missing 2-3 chars at the beginning and gcc 
aborts with for example 
"udev_sysdeps.c:73:1: error: ointer-arith: No such file ..."

2.6.19 compiles xfce-4 fine so I assume this is some
subtle sh-kernel specific fault.

I appreciate any hints!

Thanks,

-- 
 Manuel Lauss

Re: 2.6.20 instability

[Paul M. <le...@li...>]

On Mon, Jan 29, 2007 at 10:27:19AM +0100, Manuel Lauss wrote:
> I wanted to ask if I'm the only one experiencing this:
> With 2.6.20-rc, most apps I try to launch are killed
> due to random segfaults. I tried to build udev
> on the target itself; make V=1 would fail with
> gcc simply segfaulting, sometimes /bin/sh can no longer
> be found, sometimes parts of gcc's cmdline arguments
> are missing 2-3 chars at the beginning and gcc 
> aborts with for example 
> "udev_sysdeps.c:73:1: error: ointer-arith: No such file ..."
> 
> 2.6.19 compiles xfce-4 fine so I assume this is some
> subtle sh-kernel specific fault.
> 
> I appreciate any hints!
> 
We switched the lazy dcache writeback model to PG_dcache_dirty, which
results in measurably less flushing. Unfortunately it seems like there
are still some explicit flushes required that aren't being handled.

My boards do not frequently exhibit this behaviour however, so it's slow
debugging. If you want a quick fix, you can either revert to PG_mapped
behaviour, or insert a dummy flush for the faulting page's dcache lines
in update_mmu_cache().

Re: 2.6.20 instability

[Manuel L. <ma...@ro...>]

On Wed, Feb 07, 2007 at 07:40:02PM +0900, Paul Mundt wrote:
> On Fri, Feb 02, 2007 at 11:48:43AM +0100, Manuel Lauss wrote:
> > I don't run -git kernels so patch 1 is not applied anyway and
> > without patch 2 the situation is much worse: 
> > Kernel oopses every time a pcmcia ethernet card is inserted
> > (cf cards are not affected or much less frequently).
> > The difference is that for the ethernet card io ports are allocated
> > starting from 0x300, while the cf card io ports start at 0x0.
> > Also, the oops does not occur if the ethernet card is already in the
> > socket when kernel boots. I can then also reapeatedly eject/insert
> > it without problems.
> > 
> Can you post the log of the oops?

Please grab it here:

http://mlau.at/sh/pcnet_cs-oops-1.txt

For reference the debug output with a cf card (ide-cs)
http://mlau.at/sh/pcmcia_cfcard.txt

Also, the socket driver source:
http://mlau.at/sh/exm7760_ss.c


Does not seem to be irq related; pcnet_cs does not get that
far, and the CD irq is debounced in a CPLD.

Thanks!

-- 
 ml.

Re: 2.6.20 instability

[Manuel L. <ma...@ro...>]

On Fri, Feb 09, 2007 at 08:46:53PM +0900, Paul Mundt wrote:
> On Fri, Feb 09, 2007 at 12:38:41PM +0100, Manuel Lauss wrote:
> > On Fri, Feb 09, 2007 at 07:23:16PM +0900, Paul Mundt wrote:
> > > Once you've pre-faulted the page, please try to dump the whole page
> > > and see if it manages to blow up at that same location. You can setup the
> > > translation with:
> > > 
> > > 	pte_t entry = pfn_pte(0x18000000 >> PAGE_SHIFT, pgprot);
> > > 	update_mmu_cache(NULL, 0xc0600000, entry);
> > 
> > Inserted this after the p3_ioremap() call; still oopsing at the same
> > address.
> > 
> > The oops is triggered at drivers/net/pcmcia/pcnet_cs::get_hwinfo(),
> > by the first readb().
> >  
> That makes me suspect that we're doing something evil to PTEA. Can you
> try reverting the fast-path TLB miss handler (may need a bit of
> munging..)?

Sorry for the late reply;
Vanilla 2.6.20 with the TLB miss handler fast path reverted
looks ok. Kernel survives 100+ insert/ejects of the pcnet_cs
card just fine, also with the (default) 4MB memory windows.

Thanks!

-- 
 ml.

Re: 2.6.20 instability

[Paul M. <le...@li...>]

On Tue, Feb 13, 2007 at 10:46:42AM +0100, Manuel Lauss wrote:
> On Fri, Feb 09, 2007 at 08:46:53PM +0900, Paul Mundt wrote:
> > On Fri, Feb 09, 2007 at 12:38:41PM +0100, Manuel Lauss wrote:
> > > On Fri, Feb 09, 2007 at 07:23:16PM +0900, Paul Mundt wrote:
> > > > Once you've pre-faulted the page, please try to dump the whole page
> > > > and see if it manages to blow up at that same location. You can setup the
> > > > translation with:
> > > > 
> > > > 	pte_t entry = pfn_pte(0x18000000 >> PAGE_SHIFT, pgprot);
> > > > 	update_mmu_cache(NULL, 0xc0600000, entry);
> > > 
> > > Inserted this after the p3_ioremap() call; still oopsing at the same
> > > address.
> > > 
> > > The oops is triggered at drivers/net/pcmcia/pcnet_cs::get_hwinfo(),
> > > by the first readb().
> > >  
> > That makes me suspect that we're doing something evil to PTEA. Can you
> > try reverting the fast-path TLB miss handler (may need a bit of
> > munging..)?
> 
> Sorry for the late reply;
> Vanilla 2.6.20 with the TLB miss handler fast path reverted
> looks ok. Kernel survives 100+ insert/ejects of the pcnet_cs
> card just fine, also with the (default) 4MB memory windows.
> 
> Thanks!
> 
That's what I was afraid of. I've hit a few other problems with it as
well, so I think the right thing to do for now is to simply revert it. We
can toss it back in once it's recieved some more testing, since the
fast-path bits are quite small on their own.

It might also be worth checking the __do_page_fault() ->
update_mmu_cache() path for PG_dcache_dirty tests that might be causing
PTEA problems, but this is obviously something that needs to be looked
at more closely, especially for the legacy parts.

Thanks for the exhaustive testing!

Re: 2.6.20 instability

[Manuel L. <ma...@ro...>]

Hi Paul,

On Mon, Jan 29, 2007 at 06:40:24PM +0900, Paul Mundt wrote:
> We switched the lazy dcache writeback model to PG_dcache_dirty, which
> results in measurably less flushing. Unfortunately it seems like there
> are still some explicit flushes required that aren't being handled.
> 
> My boards do not frequently exhibit this behaviour however, so it's slow
> debugging. If you want a quick fix, you can either revert to PG_mapped
> behaviour, or insert a dummy flush for the faulting page's dcache lines
> in update_mmu_cache().

With commit 0ca6a188.. ("Lazy dcache writeback optimizations") applied
it seems pretty stable. I'll let it run overnight and see where
it fails.

Thanks,

-- 
 ml.

Re: 2.6.20 instability

[Manuel L. <ma...@ro...>]

On Wed, Feb 07, 2007 at 01:31:50PM +0100, Manuel Lauss wrote:
> On Wed, Feb 07, 2007 at 07:40:02PM +0900, Paul Mundt wrote:
> > On Fri, Feb 02, 2007 at 11:48:43AM +0100, Manuel Lauss wrote:
> > > I don't run -git kernels so patch 1 is not applied anyway and
> > > without patch 2 the situation is much worse: 
> > > Kernel oopses every time a pcmcia ethernet card is inserted
> > > (cf cards are not affected or much less frequently).
> > > The difference is that for the ethernet card io ports are allocated
> > > starting from 0x300, while the cf card io ports start at 0x0.
> > > Also, the oops does not occur if the ethernet card is already in the
> > > socket when kernel boots. I can then also reapeatedly eject/insert
> > > it without problems.
> > > 
> > Can you post the log of the oops?
> 
> Please grab it here:
> 
> http://mlau.at/sh/pcnet_cs-oops-1.txt
> 
> For reference the debug output with a cf card (ide-cs)
> http://mlau.at/sh/pcmcia_cfcard.txt
> 
> Also, the socket driver source:
> http://mlau.at/sh/exm7760_ss.c

I have a little more info:

It seems the error condition is depeding on the size of the
window getting p3_ioremap()ed in the socket driver.
For example, a minimal kernel with SCIF/pcmcia/net/ide support
will NOT oops when the window size is reduced to 2MB
(the MAP_SIZE constant in exm7760_ss.c).

For a larger kernel with more built-in hardware I need to
reduce the window size to 1MB.

Oopses occur when something tries to access beyond 0xc0400000.
(just a guess though)

more logs here:
http://mlau.at/sh/exm7760-2.6.20-pcmcia_net-oopses-3.txt

-- 
 ml.

Re: 2.6.20 instability

[Manuel L. <ma...@ro...>]

Hi Paul,
 
> On Mon, Jan 29, 2007 at 06:40:24PM +0900, Paul Mundt wrote:
> > We switched the lazy dcache writeback model to PG_dcache_dirty, which
> > results in measurably less flushing. Unfortunately it seems like there
> > are still some explicit flushes required that aren't being handled.
> > 
> > My boards do not frequently exhibit this behaviour however, so it's slow
> > debugging. If you want a quick fix, you can either revert to PG_mapped
> > behaviour, or insert a dummy flush for the faulting page's dcache lines
> > in update_mmu_cache().
> 
> With commit 0ca6a188.. ("Lazy dcache writeback optimizations") applied
> it seems pretty stable. I'll let it run overnight and see where
> it fails.

Compiled a few large packages overnight, and none failed.
If there is still a chance, can the aforementioned patch go into
final 2.6.20? Otherwise SH is almost unusable.

Thanks,

-- 
 ml.

Re: 2.6.20 instability

[Paul M. <le...@li...>]

On Thu, Feb 08, 2007 at 08:43:39AM +0100, Manuel Lauss wrote:
> It seems the error condition is depeding on the size of the
> window getting p3_ioremap()ed in the socket driver.
> For example, a minimal kernel with SCIF/pcmcia/net/ide support
> will NOT oops when the window size is reduced to 2MB
> (the MAP_SIZE constant in exm7760_ss.c).
> 
> For a larger kernel with more built-in hardware I need to
> reduce the window size to 1MB.
> 
> Oopses occur when something tries to access beyond 0xc0400000.
> (just a guess though)
> 
This looks quite strange indeed. First, are you doing anything strange
with VMALLOC_START? If not, can you set it to P3 + 4MB and see if you hit
the same problem elsewhere outside of the p3_ioremap() context (or
optionally just remap an empty hole to bump it up)?

Assuming no luck and you hit the same problem again, but at a different
offset, and it's repeatable, can you walk the page tables and print out
the pgprot encoding as well as the corresponding pfn << PAGE_SHIFT for
each remapped PTE?

Going by the oops, it looks like you have a bogus PGD. Can you print out
the value of get_TTB() (and also get_TTB() + pgd_index(addr)) in
do_page_fault() under the oops path before the die()?

If that still doesn't provide any hints, can you trap the faulting
address in the TLB miss handler and see whether you're coming out alive
or not? Any branches in to handle_exception() from tlb_miss() you might
be able to spot with a JTAG on that faulting range would be interesting.

Re: 2.6.20 instability

[Paul M. <le...@li...>]

On Tue, Jan 30, 2007 at 09:18:54AM +0100, Manuel Lauss wrote:
> > On Mon, Jan 29, 2007 at 06:40:24PM +0900, Paul Mundt wrote:
> > > We switched the lazy dcache writeback model to PG_dcache_dirty, which
> > > results in measurably less flushing. Unfortunately it seems like there
> > > are still some explicit flushes required that aren't being handled.
> > > 
> > > My boards do not frequently exhibit this behaviour however, so it's slow
> > > debugging. If you want a quick fix, you can either revert to PG_mapped
> > > behaviour, or insert a dummy flush for the faulting page's dcache lines
> > > in update_mmu_cache().
> > 
> > With commit 0ca6a188.. ("Lazy dcache writeback optimizations") applied
> > it seems pretty stable. I'll let it run overnight and see where
> > it fails.
> 
> Compiled a few large packages overnight, and none failed.
> If there is still a chance, can the aforementioned patch go into
> final 2.6.20? Otherwise SH is almost unusable.
> 
My hesitation with this so far has been that some folks in the office
have hit troubles booting from CF with that applied, which I didn't see
on SATA when I was debugging it initially. I would prefer not to break
one set of boards in exchange for another, though it looks like it's
going to be 2.6.21 before I can sort out the kmap API rework for the page
colouring, which I was hoping would fix the remaining issues.

I'll have Linus pull it for 2.6.20 if there are enough people that are
experiencing troubles with Linus's current tree as it is.

Re: 2.6.20 instability

[Manuel L. <ma...@ro...>]

On Thu, Feb 08, 2007 at 05:26:23PM +0900, Paul Mundt wrote:
> On Thu, Feb 08, 2007 at 08:43:39AM +0100, Manuel Lauss wrote:
> > It seems the error condition is depeding on the size of the
> > window getting p3_ioremap()ed in the socket driver.
> > For example, a minimal kernel with SCIF/pcmcia/net/ide support
> > will NOT oops when the window size is reduced to 2MB
> > (the MAP_SIZE constant in exm7760_ss.c).
> > 
> > For a larger kernel with more built-in hardware I need to
> > reduce the window size to 1MB.
> > 
> > Oopses occur when something tries to access beyond 0xc0400000.
> > (just a guess though)
> > 
> This looks quite strange indeed. First, are you doing anything strange
> with VMALLOC_START? If not, can you set it to P3 + 4MB and see if you hit
> the same problem elsewhere outside of the p3_ioremap() context (or
> optionally just remap an empty hole to bump it up)?

The faulting address increases by the amount of added to VMALLOC_START.
System is usable though until the card is removed, then it blows up
completely. I can even insert/remove a CF card multiple times without
incident.

 
> Assuming no luck and you hit the same problem again, but at a different
> offset, and it's repeatable, can you walk the page tables and print out
> the pgprot encoding as well as the corresponding pfn << PAGE_SHIFT for
> each remapped PTE?

That may take a (long) while, I don't know yet where to start :)

> Going by the oops, it looks like you have a bogus PGD. Can you print out
> the value of get_TTB() (and also get_TTB() + pgd_index(addr)) in
> do_page_fault() under the oops path before the die()?

output below

> If that still doesn't provide any hints, can you trap the faulting
> address in the TLB miss handler and see whether you're coming out alive
> or not? Any branches in to handle_exception() from tlb_miss() you might
> be able to spot with a JTAG on that faulting range would be interesting.

okay, I'll try

Thanks!

exm7760_ss:  \-> p3_ioremap(18000000, 400000, 60000534) = C0600000
exm7760_ss: set_memmap:(1) map=1 flags=0x0021 ss=0xc0600000 cs=0x00000000 [ ACT ATTRIB]
Unable to handle kernel paging request at virtual address c0600ff0
pc = 8c006374
get_TTB() = 8f512000  TTB()+pgd_index() = 8f512301
*pde = 00000000
Oops: 0000 [#1]
Modules linked in:

Pid : 206, Comm:              pccardd
PC is at generic_readb+0x0/0x6
PC  : 8c006374 SP  : 8fd2fb6c SR  : 40008101 TEA : c0600ff0    Not tainted
R0  : 8c006374 R1  : 003fffff R2  : 00000000 R3  : 00000001
R4  : c0600ff0 R5  : 0000242a R6  : 60000534 R7  : 000000f0
R8  : 8c2ce94c R9  : c0600ff0 R10 : 8c2ce948 R11 : c0600000
R12 : 8f992000 R13 : 00000000 R14 : 8c2b9860
MACH: 000003da MACL: 00000000 GBR : 00000000 PR  : 8c12629c

Call trace:
[<8c0101ca>] __set_personality+0x6e/0x14c
[<8c0058e2>] handle_timer_tick+0x2a/0xe4
[<8c01824c>] run_timer_softirq+0x54/0x15c
[<8c0180b4>] cascade+0x0/0x5c
[<8c00a224>] tmu_timer_interrupt+0x1c/0x3c
[<8c014e7c>] __do_softirq+0x3c/0x98
[<8c014f02>] do_softirq+0x2a/0x58
[<8c014f02>] do_softirq+0x2a/0x58
[<8c005120>] do_IRQ+0x50/0x78
[<8c0070cc>] ret_from_exception+0x0/0x14
[<8c0050d0>] do_IRQ+0x0/0x78
[<8c2b001c>] __kstrtab_pcmcia_unregister_driver+0x0/0x1c
[<8c0109a4>] __call_console_drivers+0x3c/0x58
[<8c010d18>] release_console_sem+0x138/0x226
[<8c011060>] vprintk+0x244/0x2a0
[<8c26bf48>] __func__.0+0x53584/0x89a44
[<8c014f02>] do_softirq+0x2a/0x58
[<8c014f02>] do_softirq+0x2a/0x58
[<8c0f0bc8>] number+0x108/0x26c
[<8c0110cc>] printk+0x10/0x1c
[<8c0110cc>] printk+0x10/0x1c
[<8c0110bc>] printk+0x0/0x1c
[<8c15b64a>] exm_set_memmap+0xaa/0x230
[<8c1562c8>] set_cis_map+0x78/0xb4
[<8c1566ac>] read_cis_cache+0x8c/0xe8
[<8c1566bc>] read_cis_cache+0x9c/0xe8
[<8c156d86>] pccard_get_tuple_data+0x4e/0x5c
[<8c1579c6>] pccard_read_tuple+0x6e/0x9c
[<8c1588a6>] pcmcia_device_probe+0xc6/0x178
[<8c076b23>] sysfs_create_link+0xab/0x120
[<8c11be14>] really_probe+0x78/0x140
[<8c11bf72>] driver_probe_device+0x7e/0xb0
[<8c11bd9c>] really_probe+0x0/0x140
[<8c11b47c>] bus_for_each_drv+0x34/0x70
[<8c11bfa4>] __device_attach+0x0/0xc
[<8c11c012>] device_attach+0x62/0xc0
[<8c11b620>] bus_attach_device+0x24/0x58
[<8c119fee>] device_add+0x2e6/0x494
[<8c158e6e>] pcmcia_device_add+0x17e/0x284
[<8c159018>] pcmcia_card_add+0xa4/0xe0
[<8c155bb8>] pccardd+0x0/0x270
[<8c28f778>] __func__.0+0x76db4/0x89a44
[<8c155bb8>] pccardd+0x0/0x270
[<8c1fed16>] schedule_timeout+0x66/0xa8
[<8c018958>] process_timeout+0x0/0xc
[<8c0ee29c>] kobject_get+0x10/0x1c
[<8c159d4e>] ds_event+0x7a/0xb4
[<8c1553e2>] send_event+0x9a/0xd0
[<8c15598c>] socket_insert+0xd0/0x130
[<8c154e88>] cs_debug_level+0x0/0x10
[<8c0110bc>] printk+0x0/0x1c
[<8c155d12>] pccardd+0x15a/0x270
[<8c00d264>] default_wake_function+0x0/0x18
[<8c00d264>] default_wake_function+0x0/0x18
[<8c021384>] kthread+0xd4/0x124
[<8c00d39c>] complete+0x0/0x58
[<8c02129c>] kthread_should_stop+0x0/0x14
[<8c003004>] kernel_thread_helper+0x4/0x10

Process: pccardd (pid: 206, stack limit = 8fd2e001)
Stack: (0x8fd2fb6c to 0x8fd30000)
fb60:                            00000000 0000001b 00000100 00000000 00000061
fb80: 8000141b 00000014 8fd2fd00 00110020 63ce4801 0000003f 0007a120 0006ddd0
fba0: 00086470 00030d40 00030d40 0003d090 8c2bbb04 8c2bba00 00000001 00000000
fbc0: 00000000 00000000 00000001 00000000 00000000 8c2bbb00 00000000 00000000
fbe0: 00000000 8fd2fbe4 8fd2fbe4 ffffffff 00000000 000002bc 00000001 00000000
fc00: 8fc12040 00000000 00000024 8c0101ca 00000300 00000020 8c0058e2 00000007
fc20: 8fc1bf8c 8c2bc064 8c01824c 00000000 8c319108 8c0180b4 8c00a224 8fd2fc3c
fc40: 8fd2fc3c 8c014e7c 8c323a24 00000021 8c318ec0 0000000a 8c318f04 00000001
fc60: 8c014f02 8c014f02 8c2f89ef 00000000 000000f0 8c005120 8c31d2d0 8c0070cc
fc80: 8c0050d0 ffffffff 00000000 00000000 00000030 0000ffff 8c2b001c 00000042
fca0: 00000007 000000f0 ffffffff 8c0109a4 000000f0 00002072 00002070 8c2cb190
fcc0: 8c010d18 8c2ba76c 00002072 00002072 8c011060 00000021 8c323a25 8c2f89ea
fce0: 8c26bf48 ffffffff 8c318f04 00000001 8c014f02 8c014f02 8c2f8a36 00000000
fd00: 3f1d81e0 255d4d55 72fc2d25 030060ca ffff301f 8c0f0bc8 44434234 35343433
fd20: 40008100 00000000 8c0110cc 00000000 8c0110cc 8c0110bc 8c323a24 00000021
fd40: 8c323a25 00000000 8c323a0c 8c323bc8 8c15b64a 00000042 00000007 000000f0
fd60: 8c1562c8 8c2d1428 8c323a0c 00000021 00000000 00000000 8c323a24 8c323a0c
fd80: 00000006 c0600000 00400000 00000000 000000b4 00000001 8fc200c5 00000002
fda0: 00000021 8c1566ac 8c2d1428 00000001 8c323a0c 8fc200c0 0000005a 00000005
fdc0: 8c1566bc 00000000 8c156d86 8f992000 00000000 8c320000 8fd2fe24 8c1579c6
fde0: 00000000 8fb5d400 8fb5d700 00000002 00000000 8f992040 8fb5d400 0000005a
fe00: ff00051a 8c1588a6 8c2d1428 8c322e8c 8c3236c0 8f992084 8f992000 8c2cbfec
fe20: 8f992040 8c076b23 000003f8 00000003 00000000 00000000 00000000 8f992000
fe40: 8c11be14 8fea08c0 8c2cc004 00000000 8f992084 8f992140 8c11bf72 00000000
fe60: 00000000 8c11bd9c 8f992084 8c2cc080 8fea08c0 8c11b47c 00000000 00000000
fe80: 8f992084 8c11bfa4 00000000 8c2d1284 8c2d1284 8c2cc064 8c11c012 8c2d1174
fea0: 8f992084 8f992100 8c11b620 8f992084 8f992100 8c119fee 8f992140 8f992084
fec0: 8f992180 8c158e6e 8c3236c0 8f992009 00000000 8c323b48 8c323a0c 8f992000
fee0: 8f992010 8c159018 00000000 fffffffc 8c155bb8 00000000 8c323a0c 00000001
ff00: 00000001 8c28f778 ffffffff 00000000 fffffffc 8c155bb8 8c1fed16 ffff99ef
ff20: 8c2bb560 ffff99ef 00000000 00200200 ffff99ef 8c018958 8fc12040 8c319108
ff40: 8c0ee29c 8c323b70 0000000f 8c159d4e 8c323a0c 8c323a0c 00000004 8c1553e2
ff60: 00000000 00000004 8c323a0c 8c323b48 8c15598c 8c154e88 00000000 8c0110bc
ff80: 8c323a0c 8c155d12 8c323b08 8c323a0c 8fd2e000 00000080 00000000 8fc12040
ffa0: 8c00d264 8c323b3c 8c323b3c 00000000 8fc12040 8c00d264 00000000 00000000
ffc0: 000010c0 8c021384 8c323a0c 8c3d9f38 8c00d39c 8c02129c ffffffff ffffffff
ffe0: 00000001 8c003004 00000000 00000000 00000000 00000000 00000000 00000000

Re: 2.6.20 instability

[Paul M. <le...@li...>]

On Thu, Feb 08, 2007 at 12:02:47PM +0100, Manuel Lauss wrote:
> On Thu, Feb 08, 2007 at 05:26:23PM +0900, Paul Mundt wrote:
> > This looks quite strange indeed. First, are you doing anything strange
> > with VMALLOC_START? If not, can you set it to P3 + 4MB and see if you hit
> > the same problem elsewhere outside of the p3_ioremap() context (or
> > optionally just remap an empty hole to bump it up)?
> 
> The faulting address increases by the amount of added to VMALLOC_START.
> System is usable though until the card is removed, then it blows up
> completely. I can even insert/remove a CF card multiple times without
> incident.
> 
So you're saying this improves the situation? Or that there's no change?
The oops seems to confirm the latter. These addresses will be faulted
regardless, it's only the unhandled faults we care about.

> > Assuming no luck and you hit the same problem again, but at a different
> > offset, and it's repeatable, can you walk the page tables and print out
> > the pgprot encoding as well as the corresponding pfn << PAGE_SHIFT for
> > each remapped PTE?
> 
> That may take a (long) while, I don't know yet where to start :)
> 
I'll hack something together quickly then and send it to you off-list.

> exm7760_ss:  \-> p3_ioremap(18000000, 400000, 60000534) = C0600000
> exm7760_ss: set_memmap:(1) map=1 flags=0x0021 ss=0xc0600000 cs=0x00000000 [ ACT ATTRIB]
> Unable to handle kernel paging request at virtual address c0600ff0
> pc = 8c006374
> get_TTB() = 8f512000  TTB()+pgd_index() = 8f512301

That looks ok.

> *pde = 00000000

That looks questionable.

> [<8c155d12>] pccardd+0x15a/0x270
> [<8c00d264>] default_wake_function+0x0/0x18
> [<8c00d264>] default_wake_function+0x0/0x18
> [<8c021384>] kthread+0xd4/0x124
> [<8c00d39c>] complete+0x0/0x58
> [<8c02129c>] kthread_should_stop+0x0/0x14
> [<8c003004>] kernel_thread_helper+0x4/0x10

There have been some reports of the child stack pointer getting corrupted
in relation to some kernel thread handling, however, this is quite early
on in your backtrace, and the stacktrace itself looks reasonably sane, so
I don't imagine it's related. We should know more from the PTEs at least..

Re: 2.6.20 instability

[Manuel L. <ma...@ro...>]

On Tue, Jan 30, 2007 at 05:24:42PM +0900, Paul Mundt wrote:
> On Tue, Jan 30, 2007 at 09:18:54AM +0100, Manuel Lauss wrote:
> > > On Mon, Jan 29, 2007 at 06:40:24PM +0900, Paul Mundt wrote:
> > > > We switched the lazy dcache writeback model to PG_dcache_dirty, which
> > > > results in measurably less flushing. Unfortunately it seems like there
> > > > are still some explicit flushes required that aren't being handled.
> > > > 
> > > > My boards do not frequently exhibit this behaviour however, so it's slow
> > > > debugging. If you want a quick fix, you can either revert to PG_mapped
> > > > behaviour, or insert a dummy flush for the faulting page's dcache lines
> > > > in update_mmu_cache().
> > > 
> > > With commit 0ca6a188.. ("Lazy dcache writeback optimizations") applied
> > > it seems pretty stable. I'll let it run overnight and see where
> > > it fails.
> > 
> > Compiled a few large packages overnight, and none failed.
> > If there is still a chance, can the aforementioned patch go into
> > final 2.6.20? Otherwise SH is almost unusable.
> > 
> My hesitation with this so far has been that some folks in the office
> have hit troubles booting from CF with that applied, which I didn't see
> on SATA when I was debugging it initially. I would prefer not to break
> one set of boards in exchange for another, though it looks like it's

Oh, okay then. I'll apply to my local tree and remove it once it works
for all boards.

Thanks,

-- 
 ml.

Re: 2.6.20 instability

[Paul M. <pau...@re...>]

On Thu, Feb 08, 2007 at 12:02:47PM +0100, Manuel Lauss wrote:
> On Thu, Feb 08, 2007 at 05:26:23PM +0900, Paul Mundt wrote:
> > Assuming no luck and you hit the same problem again, but at a different
> > offset, and it's repeatable, can you walk the page tables and print out
> > the pgprot encoding as well as the corresponding pfn << PAGE_SHIFT for
> > each remapped PTE?
> 
> That may take a (long) while, I don't know yet where to start :)
> 
Try this..

diff --git a/arch/sh/mm/ioremap.c b/arch/sh/mm/ioremap.c
index be03d74..643b9f7 100644
--- a/arch/sh/mm/ioremap.c
+++ b/arch/sh/mm/ioremap.c
@@ -23,6 +23,85 @@
 #include <asm/cacheflush.h>
 #include <asm/tlbflush.h>
 
+#define DEBUG
+
+#ifdef DEBUG
+static inline void
+show_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end)
+{
+	pte_t *ptep;
+	spinlock_t *ptl;
+
+	printk_ratelimit();
+
+	ptep = pte_offset_map_lock(&init_mm, pmd, addr, &ptl);
+	do {
+		pte_t pte = *ptep;
+		unsigned long pteval;
+
+		if (!pte_present(pte))
+			continue;
+
+		pteval = pte_val(pte);
+
+		printk("ADDR -> %08lx, PHYS -> %08lx, FLAGS -> %08lx\n",
+		       addr, pteval & PTE_PHYS_MASK,
+		       pteval & ~(PTE_PHYS_MASK | _PAGE_DUMP));
+	} while (ptep++, addr += PAGE_SIZE, addr != end);
+
+	pte_unmap_unlock(ptep - 1, ptl);
+	cond_resched();
+}
+
+static inline void
+show_pmd_range(pud_t *pud, unsigned long addr, unsigned long end)
+{
+	unsigned long next;
+	pmd_t *pmd;
+
+	pmd = pmd_offset(pud, addr);
+	do {
+		next = pmd_addr_end(addr, end);
+		if (pmd_none_or_clear_bad(pmd))
+			break;
+		show_pte_range(pmd, addr, next);
+	} while (pmd++, addr = next, addr != end);
+}
+
+static inline void
+show_pud_range(pgd_t *pgd, unsigned long addr, unsigned long end)
+{
+	unsigned long next;
+	pud_t *pud;
+
+	pud = pud_offset(pgd, addr);
+	do {
+		next = pud_addr_end(addr, end);
+		if (pud_none_or_clear_bad(pud))
+			break;
+		show_pmd_range(pud, addr, next);
+	} while (pud++, addr = next, addr != end);
+}
+
+static void show_pgtables(unsigned long addr, unsigned long end)
+{
+	unsigned long next;
+	pgd_t *pgd;
+
+	pgd = pgd_offset_k(addr);
+
+	printk("Dumping page tables for %08lx -> %08lx\n", addr, end);
+	do {
+		next = pgd_addr_end(addr, end);
+		if (pgd_none_or_clear_bad(pgd))
+			break;
+		show_pud_range(pgd, addr, next);
+	} while (pgd++, addr = next, addr != end);
+}
+#else
+#define show_pgtables(addr, end)	do { } while (0)
+#endif
+
 /*
  * Remap an arbitrary physical address space into the kernel virtual
  * address space. Needed when the kernel wants to access high addresses
@@ -108,6 +187,9 @@ void __iomem *__ioremap(unsigned long phys_addr, unsigned long size,
 			return NULL;
 		}
 
+	if (unlikely(flags & _PAGE_DUMP))
+		show_pgtables(addr, addr + size);
+
 	return (void __iomem *)(offset + (char *)orig_addr);
 }
 EXPORT_SYMBOL(__ioremap);
diff --git a/include/asm-sh/pgtable.h b/include/asm-sh/pgtable.h
index 3721a44..a10dec0 100644
--- a/include/asm-sh/pgtable.h
+++ b/include/asm-sh/pgtable.h
@@ -81,7 +81,7 @@ extern unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)];
  * - Bit 9 is reserved by everyone and used by _PAGE_PROTNONE.
  *
  * - Bits 10 and 11 are low bits of the PPN that are reserved on >= 4K pages.
- *   Bit 10 is used for _PAGE_ACCESSED, bit 11 remains unused.
+ *   Bit 10 is used for _PAGE_ACCESSED, bit 11 is used for _PAGE_DUMP.
  *
  * - Bits 31, 30, and 29 remain unused by everyone and can be used for future
  *   software flags, although care must be taken to update _PAGE_CLEAR_FLAGS.
@@ -115,6 +115,8 @@ extern unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)];
 #define _PAGE_ACCESSED	0x400		/* software: page referenced */
 #define _PAGE_FILE	_PAGE_WT	/* software: pagecache or swap? */
 
+#define _PAGE_DUMP	0x800		/* software: debug flag, pgtable dump */
+
 /* Extended mode bits */
 #define _PAGE_EXT_ESZ0		0x0010	/* ESZ0-bit: Size of page */
 #define _PAGE_EXT_ESZ1		0x0020	/* ESZ1-bit: Size of page */
@@ -153,9 +155,10 @@ extern unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)];
 #ifdef CONFIG_CPU_SH3
 #define _PAGE_CLEAR_FLAGS	(_PAGE_PROTNONE | _PAGE_ACCESSED| \
 				 _PAGE_FILE	| _PAGE_SZ1	| \
-				 _PAGE_HW_SHARED)
+				 _PAGE_HW_SHARED| _PAGE_DUMP)
 #else
-#define _PAGE_CLEAR_FLAGS	(_PAGE_PROTNONE | _PAGE_ACCESSED | _PAGE_FILE)
+#define _PAGE_CLEAR_FLAGS	(_PAGE_PROTNONE | _PAGE_ACCESSED | \
+				 _PAGE_FILE | _PAGE_DUMP)
 #endif
 
 #define _PAGE_FLAGS_HARDWARE_MASK	(0x1fffffff & ~(_PAGE_CLEAR_FLAGS))

Re: 2.6.20 instability

[Manuel L. <ma...@ro...>]

On Tue, Jan 30, 2007 at 09:36:38AM +0100, Manuel Lauss wrote:
> > My hesitation with this so far has been that some folks in the office
> > have hit troubles booting from CF with that applied, which I didn't see
> > on SATA when I was debugging it initially. I would prefer not to break
> > one set of boards in exchange for another, though it looks like it's

I see now what you mean: kernel occasionally oopses when I
insert a PCMCIA card, like this one:

pccard: PCMCIA card inserted into slot 1
pcmcia: registering new device pcmcia1.0
pcmcia: request for exclusive IRQ could not be fulfilled.
pcmcia: the driver needs updating to supported shared IRQ lines.
Unable to handle kernel paging request at virtual address c0a80ff0
pc = 8c0063b0
*pde = 00000000
Oops: 0000 [#1]
Modules linked in:

Pid : 201, Comm:              pccardd
PC is at generic_readb+0x0/0x6
PC  : 8c0063b0 SP  : 8fdb5b68 SR  : 40008101 TEA : c0a80ff0    Not tainted
R0  : 8c0063b0 R1  : 003fffff R2  : 00000000 R3  : 8c319f1c
R4  : c0a80ff0 R5  : 8c319f28 R6  : 60000534 R7  : 40008101
R8  : 8c2c2b1c R9  : c0a80ff0 R10 : 8c2c2b18 R11 : c0a80000
R12 : 8fb3dc00 R13 : 00000000 R14 : 8c2ad874
MACH: 000003da MACL: 00000000 GBR : 00000000 PR  : 8c1289dc

the TEA is in range of the address returned by  p3_ioremap() of
area 5/6.

It's very hard to reproduce reliably, it sometimes just happens :(

-- 
 ml.

Re: 2.6.20 instability

[Manuel L. <ma...@ro...>]

On Fri, Feb 09, 2007 at 05:34:24PM +0900, Paul Mundt wrote:
> On Thu, Feb 08, 2007 at 12:02:47PM +0100, Manuel Lauss wrote:
> > On Thu, Feb 08, 2007 at 05:26:23PM +0900, Paul Mundt wrote:
> > > Assuming no luck and you hit the same problem again, but at a different
> > > offset, and it's repeatable, can you walk the page tables and print out
> > > the pgprot encoding as well as the corresponding pfn << PAGE_SHIFT for
> > > each remapped PTE?
> > 
> > That may take a (long) while, I don't know yet where to start :)
> > 
> Try this..

Thank you.

Courisouly, with the dumping enabled the oops did not occur
the first 3 times, strange, but after a another reset it is
as oops-happy as ever.

This is the log of the first few *working* cases:
http://mlau.at/sh/sh4-ptedumper.txt

And here from the oopsing ones:
http://mlau.at/sh/sh4-ptedumper-2.txt
http://mlau.at/sh/sh4-ptedumper-3.txt

I'll try to find something with the JTAG probe next.

-- 
 ml.

Re: 2.6.20 instability

[Paul M. <pau...@re...>]

On Fri, Feb 02, 2007 at 08:31:26AM +0100, Manuel Lauss wrote:
> On Tue, Jan 30, 2007 at 09:36:38AM +0100, Manuel Lauss wrote:
> > > My hesitation with this so far has been that some folks in the office
> > > have hit troubles booting from CF with that applied, which I didn't see
> > > on SATA when I was debugging it initially. I would prefer not to break
> > > one set of boards in exchange for another, though it looks like it's
> 
> I see now what you mean: kernel occasionally oopses when I
> insert a PCMCIA card, like this one:
> 
How is the situation if you have both

    sh: Don't set reserved _PAGE_WT bit on SH-3.

and

    sh: Lazy dcache writeback optimizations.

reverted? (git show <hash> | patch -p1 -R in that order works fine)

We're still not quite all the way there with the kmap_coherent() stuff
(and the move to copy_user_highpage()), so that's still going to take a
bit of work. While I think most of the page colouring problems are now
taken care of, it seems in the PG_dcache_dirty route we're _still_
missing some flushing that seems to break in certain paths regardless of
the kmap state.

If you have issues with those two reverted, it would be interesting to
see the logs. My SH7760 test board is much more stable with PG_mapped, at
the cost of some performance.

Re: 2.6.20 instability

[Paul M. <pau...@re...>]

On Fri, Feb 09, 2007 at 10:32:49AM +0100, Manuel Lauss wrote:
> On Fri, Feb 09, 2007 at 05:34:24PM +0900, Paul Mundt wrote:
> > On Thu, Feb 08, 2007 at 12:02:47PM +0100, Manuel Lauss wrote:
> > > On Thu, Feb 08, 2007 at 05:26:23PM +0900, Paul Mundt wrote:
> > > > Assuming no luck and you hit the same problem again, but at a different
> > > > offset, and it's repeatable, can you walk the page tables and print out
> > > > the pgprot encoding as well as the corresponding pfn << PAGE_SHIFT for
> > > > each remapped PTE?
> > > 
> > > That may take a (long) while, I don't know yet where to start :)
> > > 
> > Try this..
> 
> Thank you.
> 
> Courisouly, with the dumping enabled the oops did not occur
> the first 3 times, strange, but after a another reset it is
> as oops-happy as ever.
> 
> This is the log of the first few *working* cases:
> http://mlau.at/sh/sh4-ptedumper.txt
> 
> And here from the oopsing ones:
> http://mlau.at/sh/sh4-ptedumper-2.txt
> http://mlau.at/sh/sh4-ptedumper-3.txt
> 
> I'll try to find something with the JTAG probe next.
> 
The page tables look perfectly sane, so that's quite interesting. At
least we know it's not a problem of a bogus page table mapping, or pgprot
oddities.

The next thing to try would be to pre-fault the translation and make sure
it's in the TLB so we don't take a TLB miss for the dying page to see if
the issue persists for that page, or if it's bumped to the next one.
You're clearly on a system that has the old-style PTEA, that might be
something else to look at (and that's also set in the update_mmu_cache()
path). The fact that this fault happens at a fixed location suggestions
that it's not actually having a problem faulting in the translation, so I
would imagine you're not going to find much in the TLB miss case.

Once you've pre-faulted the page, please try to dump the whole page
and see if it manages to blow up at that same location. You can setup the
translation with:

	pte_t entry = pfn_pte(0x18000000 >> PAGE_SHIFT, pgprot);
	update_mmu_cache(NULL, 0xc0600000, entry);

or you can of course just hack something stupid in to the dump code to
pre-fault and bail out early (ie, a dummy read for every PTE).

The next question would be what register you have at 0x18000ff0, whether
you can use 16 or 32-bit reads, and if so, whether it's the same location
that blows up. The fact it worked the first few times and it's an
uncached mapping almost suggests a timing problem, oddly.

Re: 2.6.20 instability

[Manuel L. <ma...@ro...>]

On Fri, Feb 02, 2007 at 04:45:56PM +0900, Paul Mundt wrote:
> On Fri, Feb 02, 2007 at 08:31:26AM +0100, Manuel Lauss wrote:
> > On Tue, Jan 30, 2007 at 09:36:38AM +0100, Manuel Lauss wrote:
> > > > My hesitation with this so far has been that some folks in the office
> > > > have hit troubles booting from CF with that applied, which I didn't see
> > > > on SATA when I was debugging it initially. I would prefer not to break
> > > > one set of boards in exchange for another, though it looks like it's
> > 
> > I see now what you mean: kernel occasionally oopses when I
> > insert a PCMCIA card, like this one:
> > 
> How is the situation if you have both
> 
>     sh: Don't set reserved _PAGE_WT bit on SH-3.
> 
> and
> 
>     sh: Lazy dcache writeback optimizations.
> 
> reverted? (git show <hash> | patch -p1 -R in that order works fine)

I don't run -git kernels so patch 1 is not applied anyway and
without patch 2 the situation is much worse: 
Kernel oopses every time a pcmcia ethernet card is inserted
(cf cards are not affected or much less frequently).
The difference is that for the ethernet card io ports are allocated
starting from 0x300, while the cf card io ports start at 0x0.
Also, the oops does not occur if the ethernet card is already in the
socket when kernel boots. I can then also reapeatedly eject/insert
it without problems.

> We're still not quite all the way there with the kmap_coherent() stuff
> (and the move to copy_user_highpage()), so that's still going to take a
> bit of work. While I think most of the page colouring problems are now
> taken care of, it seems in the PG_dcache_dirty route we're _still_
> missing some flushing that seems to break in certain paths regardless of
> the kmap state.

I guess I should study the mm code more closely to understand all that :)

> If you have issues with those two reverted, it would be interesting to
> see the logs. My SH7760 test board is much more stable with PG_mapped, at
> the cost of some performance.

What logs do you need?

Thanks,

-- 
  ml.

Re: 2.6.20 instability

[Manuel L. <ma...@ro...>]

On Fri, Feb 09, 2007 at 07:23:16PM +0900, Paul Mundt wrote:
> On Fri, Feb 09, 2007 at 10:32:49AM +0100, Manuel Lauss wrote:
> > On Fri, Feb 09, 2007 at 05:34:24PM +0900, Paul Mundt wrote:
> > > On Thu, Feb 08, 2007 at 12:02:47PM +0100, Manuel Lauss wrote:
> > > > On Thu, Feb 08, 2007 at 05:26:23PM +0900, Paul Mundt wrote:
> > > > > Assuming no luck and you hit the same problem again, but at a different
> > > > > offset, and it's repeatable, can you walk the page tables and print out
> > > > > the pgprot encoding as well as the corresponding pfn << PAGE_SHIFT for
> > > > > each remapped PTE?

> The page tables look perfectly sane, so that's quite interesting. At
> least we know it's not a problem of a bogus page table mapping, or pgprot
> oddities.
> 
> The next thing to try would be to pre-fault the translation and make sure
> it's in the TLB so we don't take a TLB miss for the dying page to see if
> the issue persists for that page, or if it's bumped to the next one.
> You're clearly on a system that has the old-style PTEA, that might be

a (old?) SH7760 rev. AD

> something else to look at (and that's also set in the update_mmu_cache()
> path). The fact that this fault happens at a fixed location suggestions
> that it's not actually having a problem faulting in the translation, so I
> would imagine you're not going to find much in the TLB miss case.
> 
> Once you've pre-faulted the page, please try to dump the whole page
> and see if it manages to blow up at that same location. You can setup the
> translation with:
> 
> 	pte_t entry = pfn_pte(0x18000000 >> PAGE_SHIFT, pgprot);
> 	update_mmu_cache(NULL, 0xc0600000, entry);

Inserted this after the p3_ioremap() call; still oopsing at the same
address.

The oops is triggered at drivers/net/pcmcia/pcnet_cs::get_hwinfo(),
by the first readb().
 
> or you can of course just hack something stupid in to the dump code to
> pre-fault and bail out early (ie, a dummy read for every PTE).
> 
> The next question would be what register you have at 0x18000ff0, whether
> you can use 16 or 32-bit reads, and if so, whether it's the same location
> that blows up. The fact it worked the first few times and it's an
> uncached mapping almost suggests a timing problem, oddly.

Yes, if I power the board off for a few minutes and try again,
sometimes the card is recognized without a hitch 1-2 times, then
it starts acting up again. However, it works perfectly with 2.6.19,
and I can insert/remove the card >100 times.

-- 
 ml.

Re: 2.6.20 instability

[Paul M. <pau...@re...>]

On Fri, Feb 02, 2007 at 11:48:43AM +0100, Manuel Lauss wrote:
> I don't run -git kernels so patch 1 is not applied anyway and
> without patch 2 the situation is much worse: 
> Kernel oopses every time a pcmcia ethernet card is inserted
> (cf cards are not affected or much less frequently).
> The difference is that for the ethernet card io ports are allocated
> starting from 0x300, while the cf card io ports start at 0x0.
> Also, the oops does not occur if the ethernet card is already in the
> socket when kernel boots. I can then also reapeatedly eject/insert
> it without problems.
> 
Can you post the log of the oops?

> > We're still not quite all the way there with the kmap_coherent() stuff
> > (and the move to copy_user_highpage()), so that's still going to take a
> > bit of work. While I think most of the page colouring problems are now
> > taken care of, it seems in the PG_dcache_dirty route we're _still_
> > missing some flushing that seems to break in certain paths regardless of
> > the kmap state.
> 
> I guess I should study the mm code more closely to understand all that :)
> 
The PG_dcache_dirty problem ended up being related to the fact that we
were just doing less flushing, which was causing troubles for IDE PIO in
the outsw()/insw() cases, adding some explicit dcache handling there like
we had for CONFIG_IDE fixed up those issues, so it's likely not related
to the problem you're seeing. Though that does seem to be the last
outstanding issue for PG_dcache_dirty..

Now the kmap mess is next on the list..

> What logs do you need?
> 
It would be interesting to know for starters whether your oops is IRQ
related or not? Any sort of debug output you can provide would be useful.
Oopsing if the card isn't available at boot time is certainly something
we need to fix.

Re: 2.6.20 instability

[Paul M. <pau...@re...>]

On Fri, Feb 09, 2007 at 12:38:41PM +0100, Manuel Lauss wrote:
> On Fri, Feb 09, 2007 at 07:23:16PM +0900, Paul Mundt wrote:
> > Once you've pre-faulted the page, please try to dump the whole page
> > and see if it manages to blow up at that same location. You can setup the
> > translation with:
> > 
> > 	pte_t entry = pfn_pte(0x18000000 >> PAGE_SHIFT, pgprot);
> > 	update_mmu_cache(NULL, 0xc0600000, entry);
> 
> Inserted this after the p3_ioremap() call; still oopsing at the same
> address.
> 
> The oops is triggered at drivers/net/pcmcia/pcnet_cs::get_hwinfo(),
> by the first readb().
>  
That makes me suspect that we're doing something evil to PTEA. Can you
try reverting the fast-path TLB miss handler (may need a bit of
munging..)?

commit 9b3a53ab76771e3669e50086c131e1574fe25847
Author: Stuart Menefy <stu...@st...>
Date:   Fri Nov 24 11:42:24 2006 +0900

    sh: TLB miss fast-path optimizations.
    
    Handle simple TLB miss faults which can be resolved completely
    from the page table in assembler.
    
    Signed-off-by: Stuart Menefy <stu...@st...>
    Signed-off-by: Paul Mundt <le...@li...>

diff --git a/arch/sh/Kconfig b/arch/sh/Kconfig
index a03f155..48308dc 100644
--- a/arch/sh/Kconfig
+++ b/arch/sh/Kconfig
@@ -379,6 +379,9 @@ config CPU_HAS_SR_RB
 	  See <file:Documentation/sh/register-banks.txt> for further
 	  information on SR.RB and register banking in the kernel in general.
 
+config CPU_HAS_PTEA
+	bool
+
 endmenu
 
 menu "Timer support"
diff --git a/arch/sh/kernel/cpu/sh3/entry.S b/arch/sh/kernel/cpu/sh3/entry.S
index 869d56f..5de99b4 100644
--- a/arch/sh/kernel/cpu/sh3/entry.S
+++ b/arch/sh/kernel/cpu/sh3/entry.S
@@ -13,8 +13,10 @@
 #include <linux/linkage.h>
 #include <asm/asm-offsets.h>
 #include <asm/thread_info.h>
-#include <asm/cpu/mmu_context.h>
 #include <asm/unistd.h>
+#include <asm/cpu/mmu_context.h>
+#include <asm/pgtable.h>
+#include <asm/page.h>
 
 ! NOTE:
 ! GNU as (as of 2.9.1) changes bf/s into bt/s and bra, when the address
@@ -136,29 +138,14 @@ ENTRY(tlb_protection_violation_store)
 
 call_dpf:
 	mov.l	1f, r0
-	mov	r5, r8
-	mov.l	@r0, r6
-	mov	r6, r9
-	mov.l	2f, r0
-	sts	pr, r10
-	jsr	@r0
-	 mov	r15, r4
-	!
-	tst	r0, r0
-	bf/s	0f
-	 lds	r10, pr
-	rts
-	 nop
-0:	sti
+ 	mov.l	@r0, r6		! address
 	mov.l	3f, r0
-	mov	r9, r6
-	mov	r8, r5
+	sti
 	jmp	@r0
-	 mov	r15, r4
+ 	 mov	r15, r4		! regs
 
 	.align 2
 1:	.long	MMU_TEA
-2:	.long	__do_page_fault
 3:	.long	do_page_fault
 
 	.align	2
@@ -344,9 +331,176 @@ general_exception:
 2:	.long	ret_from_exception
 !
 !
+
+/* This code makes some assumptions to improve performance.
+ * Make sure they are stil true. */
+#if PTRS_PER_PGD != PTRS_PER_PTE
+#error PDG and PTE sizes don't match
+#endif
+
+/* gas doesn't flag impossible values for mov #immediate as an error */
+#if (_PAGE_PRESENT >> 2) > 0x7f
+#error cannot load PAGE_PRESENT as an immediate
+#endif
+#if _PAGE_DIRTY > 0x7f
+#error cannot load PAGE_DIRTY as an immediate
+#endif
+#if (_PAGE_PRESENT << 2) != _PAGE_ACCESSED
+#error cannot derive PAGE_ACCESSED from PAGE_PRESENT
+#endif
+
+#if defined(CONFIG_CPU_SH4)
+#define ldmmupteh(r)	mov.l	8f, r
+#else
+#define ldmmupteh(r)	mov	#MMU_PTEH, r
+#endif
+
 	.balign 	1024,0,1024
 tlb_miss:
-	mov.l	1f, k2
+#ifdef COUNT_EXCEPTIONS
+	! Increment the counts
+	mov.l	9f, k1
+	mov.l	@k1, k2
+	add	#1, k2
+	mov.l	k2, @k1
+#endif
+
+	! k0 scratch
+	! k1 pgd and pte pointers
+	! k2 faulting address
+	! k3 pgd and pte index masks
+	! k4 shift
+
+	! Load up the pgd entry (k1)
+
+	ldmmupteh(k0)			!  9 LS (latency=2)	MMU_PTEH
+
+	mov.w	4f, k3			!  8 LS (latency=2)	(PTRS_PER_PGD-1) << 2
+	mov	#-(PGDIR_SHIFT-2), k4	!  6 EX
+
+	mov.l	@(MMU_TEA-MMU_PTEH,k0), k2	! 18 LS (latency=2)
+
+	mov.l	@(MMU_TTB-MMU_PTEH,k0), k1	! 18 LS (latency=2)
+
+	mov	k2, k0			!   5 MT (latency=0)
+	shld	k4, k0			!  99 EX
+
+	and	k3, k0			!  78 EX
+
+	mov.l	@(k0, k1), k1		!  21 LS (latency=2)
+	mov	#-(PAGE_SHIFT-2), k4	!   6 EX
+
+	! Load up the pte entry (k2)
+
+	mov	k2, k0			!   5 MT (latency=0)
+	shld	k4, k0			!  99 EX
+
+	tst	k1, k1			!  86 MT
+
+	bt	20f			! 110 BR
+
+	and	k3, k0			!  78 EX
+	mov.w	5f, k4			!   8 LS (latency=2)	_PAGE_PRESENT
+
+	mov.l	@(k0, k1), k2		!  21 LS (latency=2)
+	add	k0, k1			!  49 EX
+
+#ifdef CONFIG_CPU_HAS_PTEA
+	! Test the entry for present and _PAGE_ACCESSED
+
+	mov	#-28, k3		!   6 EX
+	mov	k2, k0			!   5 MT (latency=0)
+
+	tst	k4, k2			!  68 MT
+	shld	k3, k0			!  99 EX
+
+	bt	20f			! 110 BR
+
+	! Set PTEA register
+	! MMU_PTEA = ((pteval >> 28) & 0xe) | (pteval & 0x1)
+	!
+	! k0=pte>>28, k1=pte*, k2=pte, k3=<unused>, k4=_PAGE_PRESENT
+
+	and	#0xe, k0		!  79 EX
+
+	mov	k0, k3			!   5 MT (latency=0)
+	mov	k2, k0			!   5 MT (latency=0)
+
+	and	#1, k0			!  79 EX
+
+	or	k0, k3			!  82 EX
+
+	ldmmupteh(k0)			!   9 LS (latency=2)
+	shll2	k4			! 101 EX		_PAGE_ACCESSED
+
+	tst	k4, k2			!  68 MT
+
+	mov.l	k3, @(MMU_PTEA-MMU_PTEH,k0)	! 27 LS
+
+	mov.l	7f, k3			!   9 LS (latency=2)	_PAGE_FLAGS_HARDWARE_MASK
+
+	! k0=MMU_PTEH, k1=pte*, k2=pte, k3=_PAGE_FLAGS_HARDWARE, k4=_PAGE_ACCESSED
+#else
+
+	! Test the entry for present and _PAGE_ACCESSED
+
+	mov.l	7f, k3			!   9 LS (latency=2)	_PAGE_FLAGS_HARDWARE_MASK
+	tst	k4, k2			!  68 MT
+
+	shll2	k4			! 101 EX		_PAGE_ACCESSED
+	ldmmupteh(k0)			!   9 LS (latency=2)
+
+	bt	20f			! 110 BR
+	tst	k4, k2			!  68 MT
+
+	! k0=MMU_PTEH, k1=pte*, k2=pte, k3=_PAGE_FLAGS_HARDWARE, k4=_PAGE_ACCESSED
+
+#endif
+
+	! Set up the entry
+
+	and	k2, k3			!  78 EX
+	bt/s	10f			! 108 BR
+
+	 mov.l	k3, @(MMU_PTEL-MMU_PTEH,k0)	! 27 LS
+
+	ldtlb				! 128 CO
+
+	! At least one instruction between ldtlb and rte
+	nop				! 119 NOP
+
+	rte				! 126 CO
+
+	 nop				! 119 NOP
+
+
+10:	or	k4, k2			!  82 EX
+
+	ldtlb				! 128 CO
+
+	! At least one instruction between ldtlb and rte
+	mov.l	k2, @k1			!  27 LS
+
+	rte				! 126 CO
+
+	! Note we cannot execute mov here, because it is executed after
+	! restoring SSR, so would be executed in user space.
+	 nop				! 119 NOP
+
+
+	.align 5
+	! Once cache line if possible...
+1:	.long	swapper_pg_dir
+4:	.short	(PTRS_PER_PGD-1) << 2
+5:	.short	_PAGE_PRESENT
+7:	.long	_PAGE_FLAGS_HARDWARE_MASK
+8:	.long	MMU_PTEH
+#ifdef COUNT_EXCEPTIONS
+9:	.long	exception_count_miss
+#endif
+
+	! Either pgd or pte not present
+20:	mov.l	1f, k2
 	mov.l	4f, k3
 	bra	handle_exception
 	 mov.l	@k2, k2
@@ -496,6 +650,15 @@ skip_save:
 	bf	interrupt_exception
 	shlr2	r8
 	shlr	r8
+
+#ifdef COUNT_EXCEPTIONS
+	mov.l	5f, r9
+	add	r8, r9
+	mov.l	@r9, r10
+	add	#1, r10
+	mov.l	r10, @r9
+#endif
+
 	mov.l	4f, r9
 	add	r8, r9
 	mov.l	@r9, r9
@@ -509,6 +672,9 @@ skip_save:
 2:	.long	0x000080f0	! FD=1, IMASK=15
 3:	.long	0xcfffffff	! RB=0, BL=0
 4:	.long	exception_handling_table
+#ifdef COUNT_EXCEPTIONS
+5:	.long	exception_count_table
+#endif
 
 interrupt_exception:
 	mov.l	1f, r9
diff --git a/arch/sh/kernel/cpu/sh4/probe.c b/arch/sh/kernel/cpu/sh4/probe.c
index c294de1..afe0f1b 100644
--- a/arch/sh/kernel/cpu/sh4/probe.c
+++ b/arch/sh/kernel/cpu/sh4/probe.c
@@ -79,16 +79,16 @@ int __init detect_cpu_and_cache_system(void)
 	case 0x205:
 		cpu_data->type = CPU_SH7750;
 		cpu_data->flags |= CPU_HAS_P2_FLUSH_BUG | CPU_HAS_FPU |
-				   CPU_HAS_PERF_COUNTER | CPU_HAS_PTEA;
+				   CPU_HAS_PERF_COUNTER;
 		break;
 	case 0x206:
 		cpu_data->type = CPU_SH7750S;
 		cpu_data->flags |= CPU_HAS_P2_FLUSH_BUG | CPU_HAS_FPU |
-				   CPU_HAS_PERF_COUNTER | CPU_HAS_PTEA;
+				   CPU_HAS_PERF_COUNTER;
 		break;
 	case 0x1100:
 		cpu_data->type = CPU_SH7751;
-		cpu_data->flags |= CPU_HAS_FPU | CPU_HAS_PTEA;
+		cpu_data->flags |= CPU_HAS_FPU;
 		break;
 	case 0x2000:
 		cpu_data->type = CPU_SH73180;
@@ -126,23 +126,22 @@ int __init detect_cpu_and_cache_system(void)
 		break;
 	case 0x8000:
 		cpu_data->type = CPU_ST40RA;
-		cpu_data->flags |= CPU_HAS_FPU | CPU_HAS_PTEA;
+		cpu_data->flags |= CPU_HAS_FPU;
 		break;
 	case 0x8100:
 		cpu_data->type = CPU_ST40GX1;
-		cpu_data->flags |= CPU_HAS_FPU | CPU_HAS_PTEA;
+		cpu_data->flags |= CPU_HAS_FPU;
 		break;
 	case 0x700:
 		cpu_data->type = CPU_SH4_501;
 		cpu_data->icache.ways = 2;
 		cpu_data->dcache.ways = 2;
-		cpu_data->flags |= CPU_HAS_PTEA;
 		break;
 	case 0x600:
 		cpu_data->type = CPU_SH4_202;
 		cpu_data->icache.ways = 2;
 		cpu_data->dcache.ways = 2;
-		cpu_data->flags |= CPU_HAS_FPU | CPU_HAS_PTEA;
+		cpu_data->flags |= CPU_HAS_FPU;
 		break;
 	case 0x500 ... 0x501:
 		switch (prr) {
@@ -160,7 +159,7 @@ int __init detect_cpu_and_cache_system(void)
 		cpu_data->icache.ways = 2;
 		cpu_data->dcache.ways = 2;
 
-		cpu_data->flags |= CPU_HAS_FPU | CPU_HAS_PTEA;
+		cpu_data->flags |= CPU_HAS_FPU;
 
 		break;
 	default:
@@ -173,6 +172,10 @@ int __init detect_cpu_and_cache_system(void)
 	cpu_data->dcache.ways = 1;
 #endif
 
+#ifdef CONFIG_CPU_HAS_PTEA
+	cpu_data->flags |= CPU_HAS_PTEA;
+#endif
+
 	/*
 	 * On anything that's not a direct-mapped cache, look to the CVR
 	 * for I/D-cache specifics.
diff --git a/arch/sh/mm/Kconfig b/arch/sh/mm/Kconfig
index 88e9663..6cd6d00 100644
--- a/arch/sh/mm/Kconfig
+++ b/arch/sh/mm/Kconfig
@@ -20,6 +20,7 @@ config CPU_SH4
 	bool
 	select CPU_HAS_INTEVT
 	select CPU_HAS_SR_RB
+	select CPU_HAS_PTEA if !CPU_SUBTYPE_ST40
 
 config CPU_SH4A
 	bool
diff --git a/arch/sh/mm/fault.c b/arch/sh/mm/fault.c
index 128907e..123fb80 100644
--- a/arch/sh/mm/fault.c
+++ b/arch/sh/mm/fault.c
@@ -223,89 +223,3 @@ do_sigbus:
 	if (!user_mode(regs))
 		goto no_context;
 }
-
-#ifdef CONFIG_SH_STORE_QUEUES
-/*
- * This is a special case for the SH-4 store queues, as pages for this
- * space still need to be faulted in before it's possible to flush the
- * store queue cache for writeout to the remapped region.
- */
-#define P3_ADDR_MAX		(P4SEG_STORE_QUE + 0x04000000)
-#else
-#define P3_ADDR_MAX		P4SEG
-#endif
-
-/*
- * Called with interrupts disabled.
- */
-asmlinkage int __kprobes __do_page_fault(struct pt_regs *regs,
-					 unsigned long writeaccess,
-					 unsigned long address)
-{
-	pgd_t *pgd;
-	pud_t *pud;
-	pmd_t *pmd;
-	pte_t *pte;
-	pte_t entry;
-	struct mm_struct *mm = current->mm;
-	spinlock_t *ptl;
-	int ret = 1;
-
-#ifdef CONFIG_SH_KGDB
-	if (kgdb_nofault && kgdb_bus_err_hook)
-		kgdb_bus_err_hook();
-#endif
-
-	/*
-	 * We don't take page faults for P1, P2, and parts of P4, these
-	 * are always mapped, whether it be due to legacy behaviour in
-	 * 29-bit mode, or due to PMB configuration in 32-bit mode.
-	 */
-	if (address >= P3SEG && address < P3_ADDR_MAX) {
-		pgd = pgd_offset_k(address);
-		mm = NULL;
-	} else {
-		if (unlikely(address >= TASK_SIZE || !mm))
-			return 1;
-
-		pgd = pgd_offset(mm, address);
-	}
-
-	pud = pud_offset(pgd, address);
-	if (pud_none_or_clear_bad(pud))
-		return 1;
-	pmd = pmd_offset(pud, address);
-	if (pmd_none_or_clear_bad(pmd))
-		return 1;
-
-	if (mm)
-		pte = pte_offset_map_lock(mm, pmd, address, &ptl);
-	else
-		pte = pte_offset_kernel(pmd, address);
-
-	entry = *pte;
-	if (unlikely(pte_none(entry) || pte_not_present(entry)))
-		goto unlock;
-	if (unlikely(writeaccess && !pte_write(entry)))
-		goto unlock;
-
-	if (writeaccess)
-		entry = pte_mkdirty(entry);
-	entry = pte_mkyoung(entry);
-
-#ifdef CONFIG_CPU_SH4
-	/*
-	 * ITLB is not affected by "ldtlb" instruction.
-	 * So, we need to flush the entry by ourselves.
-	 */
-	__flush_tlb_page(get_asid(), address & PAGE_MASK);
-#endif
-
-	set_pte(pte, entry);
-	update_mmu_cache(NULL, address, entry);
-	ret = 0;
-unlock:
-	if (mm)
-		pte_unmap_unlock(pte, ptl);
-	return ret;
-}
diff --git a/include/asm-sh/pgtable.h b/include/asm-sh/pgtable.h
index b1f21e7..fa62524 100644
--- a/include/asm-sh/pgtable.h
+++ b/include/asm-sh/pgtable.h
@@ -43,12 +43,12 @@ extern unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)];
 /* PGD bits */
 #define PGDIR_SHIFT	(PTE_SHIFT + PTE_BITS)
 #define PGDIR_BITS	(32 - PGDIR_SHIFT)
-#define PGDIR_SIZE	(1UL << PGDIR_SHIFT)
+#define PGDIR_SIZE	(1 << PGDIR_SHIFT)
 #define PGDIR_MASK	(~(PGDIR_SIZE-1))
 
 /* Entries per level */
-#define PTRS_PER_PTE	(1UL << PTE_BITS)
-#define PTRS_PER_PGD	(1UL << PGDIR_BITS)
+#define PTRS_PER_PTE	(1 << PTE_BITS)
+#define PTRS_PER_PGD	(1 << PGDIR_BITS)
 
 #define USER_PTRS_PER_PGD	(TASK_SIZE/PGDIR_SIZE)
 #define FIRST_USER_ADDRESS	0