[lkcd-devel] Re: Update to LKCD

[Suparna B. <su...@in...>]

On Wed, Dec 04, 2002 at 04:45:57PM -0800, Stephen Hemminger wrote:
> Here is an update to LKCD for 2.5 that tries to:
> * Use Linux style
> * Cleanup typo's, const handling stuff like that
> * Try and change as few base kernel files as possible
>   - Change to the SMP cpu handling (get out of sched.c) into dump_i386
>   - Only pluck as few symbols for a LKCD module as required.
>   - Make all dumps non-disruptive, kernel will reboot anyway on
> panic/nmi/...
>   - Use existing panic, sysrq integration hooks
> 
> * Integrate hooks necessary for netdump without touching dump_base
> 
> * Only re-enable interrupts if target requires it.
>    - enable APIC if it got disabled
>    - eventually want to have polling disk dump as well.

I have only looked at it briefly so far, and not the netdump bits
as yet, but I like the general direction of the changes. The less
intrusive this is and the simpler we can make the dependencies the 
better. Some of the reasons for the extra things that had been in for 
for handling quiescing were less of a problem in 2.5 together with
the more async oriented approach for i/o we have now, and it was
one of the goals to minimize those hooks, as also to integrate with
nmi handler callback infrastructure.

It is however worth recapping some of the reasons why things were as
they were to understand any caveats after your changes. BTW, its
OK to take an incremental path of having many of the 
changes checked into cvs first and then tackle any problems with
a fresh look. Most of the issues had to do with using the
standard block device drivers for i/o vs a polled/dedicated 
dump driver.  

- Forcing the other CPUs to spin inside the NMI ipi (rather than
  a more gentle ipi later or by causing a spin in schedule after 
  the return from interrupt) means that another cpu could be holding
  a spin lock irq at that moment, and in case this was a lock required
  in the dump i/o path, or any other code (e.g interrupts and softirqs),
  that could run on the dumping cpu, there could be a deadlock. The 
  chances of this are reduced with the global io_request_lock gone 
  for example, but there are some other possibilities (especially 
  with all interrupts and bottom halves been enabled on the dumping cpu)
  This could happen for non-disruptive dumps for example.
  (In 2.4 we had trouble with wait_on_irq but that's gone in 2.5)
- The second reason for the hook in the scheduler was to avoid
  schedules in case of waits in the i/o path (in 2.4 dump used
  brw_kiovec). Now that we use an async low level i/o submission
  through submit_bio and poll for completion, its not a likely
  situation assuming we have slots in the request queue and no
  allocs / waits should typically happen. But would be good to
  be fully sure (that's something we may look at for aio as well).

Duplicating code outside of the kernel to avoid exporting routines 
is something I'm not so sure of is the best way to go eventually, 
so we should get some opinion on lkml on that. 

The panic notifier call happens after smp_send_stop() in panic.
I noticed that you have code to handle the apic, but how
do we get the register state on other CPUs if they are stopped
before we get to dump ?  

Regarding nmi related general infrastructure, is there a value 
in having a send_nmi_ipi() sort of interface in the kernel, rather
than special branch for DUMP_VECTOR/KDB_VECTOR etc ?

Should we cc lkml on these discussions ?

Regards
Suparna



-- 
Suparna Bhattacharya (su...@in...)
Linux Technology Center
IBM Software Labs, India