From: Jeff D. <jd...@ka...> - 2003-01-24 18:53:02
|
jon...@ya... said: > I'm not familar with the __initcall/_exitcall code. Can someone > suggest the best way to get UML to call __exitcalls for compiled in > modules. How can I access the list of driver __exitcalls from the UML > code so that the functions can be called? I'm not even sure the kernel > builds a list of these for compiled in modules. IIRC, UML used to do the module exitcalls, and that was iffy for reasons I don't recall. UML was the only that did it. Is an exitcall the only mechanism you have for cleaning up? I take it the kernel doesn't unload modules when it halts? > I looked at include/linux/init.h and arch/um/include/init.h, these > files as very similar. Why does uml have it's own initcall system? Because there are things that UML needs to do at init/exit time that can't be done in kernel context. Jeff |
From: Jeff D. <jd...@ka...> - 2003-01-27 17:45:53
|
jon...@ya... said: > Now you kill -9 UML. The USB driver is now gone but the USB hardware > has DMA enabled. Now the host closes the helper driver handle. I can't > wait for the USB DMA to finish since the host driver has no code for > telling when it is going to finish and if is going to finish. All the > helper driver knows is that a DMA buffer was allocated. Don't you have to handle this case anyway? And if you do, doesn't that mean that UML exitcalls don't need to be working? If UML abandons a device in an unknown state, can you just reset it to get it back to a known state? Jeff |
From: Jon S. <jon...@ya...> - 2003-01-27 18:36:39
|
--- Jeff Dike <jd...@ka...> wrote: > Don't you have to handle this case anyway? And if > you do, doesn't that mean > that UML exitcalls don't need to be working? > > If UML abandons a device in an unknown state, can > you just reset it to get > it back to a known state? 1) If UML is shutdown unexpectedly I leave DMA buffers allocated in the host. You will have to take a manual action to free them when you are sure the hardware is reset. 2) If UML is shutdown and runs exitcalls. The hardware drivers receive their shutdown messages. This causes them to shutdown their DMA, free resources, etc. They will call dma_free_coherent() which I map out to the host driver to free the buffers. 3) My PCI support is generic. It works for all possible PCI devices. There is no way for me to know how to reset all possible PCI devices. This is what each drivers' exitcall routine does and why I need to call them. I could leave exitscall out, but then I would never be able to free any DMA buffers in the host. The only time I should free buffers in the host is when I get a dma_free_coherent() call in UML since the drivers won't call this unless the hardware is done with the buffer. The only way to get the drivers to shutdown their hardware is to call exitcalls. ===== Jon Smirl jon...@ya... __________________________________________________ Do you Yahoo!? Yahoo! Mail Plus - Powerful. Affordable. Sign up now. http://mailplus.yahoo.com |
From: Adam H. <ad...@la...> - 2003-01-28 03:21:26
|
On Mon, 27 Jan 2003, Jon Smirl wrote: > --- Jeff Dike <jd...@ka...> wrote: > > Don't you have to handle this case anyway? And if > > you do, doesn't that mean > > that UML exitcalls don't need to be working? > > > > If UML abandons a device in an unknown state, can > > you just reset it to get > > it back to a known state? > > 1) If UML is shutdown unexpectedly I leave DMA buffers > allocated in the host. You will have to take a manual > action to free them when you are sure the hardware is > reset. > 2) If UML is shutdown and runs exitcalls. The hardware > drivers receive their shutdown messages. This causes > them to shutdown their DMA, free resources, etc. They > will call dma_free_coherent() which I map out to the > host driver to free the buffers. > 3) My PCI support is generic. It works for all > possible PCI devices. There is no way for me to know > how to reset all possible PCI devices. This is what > each drivers' exitcall routine does and why I need to > call them. Have your stub driver maintain more state. Ie, when some program dies unexpectedly, it will function as you currently explain. When the program next starts, the program tells the stub driver that "I've reset everything, you can release anything that was previously assigned/allocated to this device." |
From: Jon S. <jon...@ya...> - 2003-01-28 04:40:18
|
--- Adam Heath <ad...@la...> wrote: > Have your stub driver maintain more state. > Ie, when some program dies unexpectedly, it will > function as you currently > explain. When the program next starts, the program > tells the stub driver that > "I've reset everything, you can release anything > that was previously > assigned/allocated to this device." > There is no simple way to track when the devices get reset. For example you may have manually insmod'd device drivers that do DMA while inside of UML. The next time you start UML you may chose not to do this. Or the driver ids used to register the DMA will change if you recompile between runs. My helper is a single, small (8K) driver that tracks alloc/free of DMA memory. It is not feasible for it to track the DMA state of 1,000s of different PCI DMA devices each of which implements their DMA control differently. Remember UML is not a normal app, it is the whole Linux OS so every PCI card that Linux supports can potentially be used in UML. Allowing the buffers to accumulate is not that bad of an option. They are small. They only accumulate when UML exits abnormally. You can 'cat /proc/uml/dma' in the host to see how many and how big. When you are Positive! that all of the DMA has been reset you can rmmod the driver in the host which will free the buffers. Leaving the buffers allocated in the host is the only safe thing to do. If you free these and DMA is still active it is guaranteed that you will be using the reset button in the near future. ===== Jon Smirl jon...@ya... __________________________________________________ Do you Yahoo!? Yahoo! Mail Plus - Powerful. Affordable. Sign up now. http://mailplus.yahoo.com |
From: Adam H. <ad...@la...> - 2003-01-28 05:05:18
|
On Mon, 27 Jan 2003, Jon Smirl wrote: > There is no simple way to track when the devices get > reset. For example you may have manually insmod'd > device drivers that do DMA while inside of UML. The > next time you start UML you may chose not to do this. > Or the driver ids used to register the DMA will change > if you recompile between runs. > > My helper is a single, small (8K) driver that tracks > alloc/free of DMA memory. It is not feasible for it > to track the DMA state of 1,000s of different PCI DMA > devices each of which implements their DMA control > differently. Remember UML is not a normal app, it is > the whole Linux OS so every PCI card that Linux > supports can potentially be used in UML. The physical device on the host has a non-changing id. Your stub driver only needs to be concerned with the detected host pci devices, and what resources have been allocated for them. I sense you are resistant to extend your driver. I suggest you remove that resistance. A user-level program opens a fd. does an ioctl to assign some host pci device to the fd. Enables irqs as RT signals. Does io(not sure how) to reset the device. IOCTL to tell the stub driver than any previous resources can be freed. Does mmap() on the fd, which the stub driver turns into host-locked real memory. If the app dies, the stub driver remembers that the mmap'd buffer was allocated for the real pci device, and doesn't free it. When the app(or any app) next loads, and does the reset, the buffer will be freed. The mmap can error if the reset ioctl has not been sent. |
From: Jon S. <jon...@ya...> - 2003-01-28 05:55:04
|
The goal of this is to run all of the standard Linux device drivers inside of UML without changing them. This might even include binary only drivers. How my code currently works: UML starts and inits my code. I open a fd to the helper in the host. UML starts loading stock Linux device drivers. A driver calls pci_alloc_coherent() I capture this, IOCTL the host, alloc the memory, and mmap it I don't know which piece of hardware in the host is being associated with the call to pci_alloc_coherent(). A driver calls irq_request() I capture this, IOCTL the host and alloc the IRQ Again I don't know what piece of hardware this is for. IRQ's aren't an issue. It is always safe to release them even when UML fails. 1) I might be able to track driver registrations in order to track which hardware. But this is a new feature only implemented by hotplug capable drivers which leaves out about 50% of the existing drivers. I might also be able to play with the initcalls code in UML. 2) Some drivers don't initialize the hardware when they start, they assume the BIOS has done it. My ATI video card is an example, I have to run a special reset program in between UML sessions. Almost all video cards have this problem. If I have forgotten to run this program it is unsafe to free the DMA buffers. --- Adam Heath <ad...@la...> wrote: >does an ioctl to assign some host pci device to the fd. This requires modifying the driver. >IOCTL to tell the stub driver than any previous resources > can be freed. Of course I could free the old buffers if such a callout existed but none of the existing Linux device drivers implement this callout. > I sense you are resistant to extend your driver. I > suggest you remove that resistance. I posted GPL source to the list. I welcome any improvements that you would like to contribute. ===== Jon Smirl jon...@ya... __________________________________________________ Do you Yahoo!? Yahoo! Mail Plus - Powerful. Affordable. Sign up now. http://mailplus.yahoo.com |
From: Jon S. <jon...@ya...> - 2003-01-24 21:20:20
|
After playing around with this for a day, it looks like adding exitcall support back into UML is the only reasonable solution. I checked a bunch of drivers and they aren't implementing device->shutdown or a reboot notifier. I don't see any other solution than exitcalls for shutting down things like DMA. I added code to do_uml_exitcalls() to also call the system __exitcalls. My UML map has 37 exit call entries. When I call them one call traps and the usb one hangs. So now I guess I get to debug the various drivers. How does UML handle the UML process exiting or a Ctrl-C of the main window, does it call do_uml_exitcalls()? I am very close to having the USB driver working. I can now mount the usb bus in /proc/bus/usb and see my devices. I just need to sort out some interrupt problems and exiting issues. If anyone is interested I can post my current code. ===== Jon Smirl jon...@ya... __________________________________________________ Do you Yahoo!? Yahoo! Mail Plus - Powerful. Affordable. Sign up now. http://mailplus.yahoo.com |
From: Henrik N. <hn...@ma...> - 2003-01-25 01:03:23
|
On Fri, 24 Jan 2003, Jon Smirl wrote: > After playing around with this for a day, it looks > like adding exitcall support back into UML is the only > reasonable solution. I checked a bunch of drivers and > they aren't implementing device->shutdown or a reboot > notifier. I don't see any other solution than > exitcalls for shutting down things like DMA. Normally there isn't much need to, as a) The host will reset any resources used when the linux process dies b) Real life drivers need to reinitialize the "hardware" on each boot as the status may be unknown. However, if you are writing programs which play tricks on the host hardware, bypassing the host kernel, then care obviously needs to be taken. If the sole purpose of this is to test hardware drivers meant to run in "the real" kernel under UML then having exitcalls or reboot/reset handlers to clean up may be a good idea, but this is somewhat of a corner case I think. If the code is meant to run in UML or other userspace programs (in the view of the host) then you'd better fix the host to handle the situation gracefully. You are not guaranteed exitcalls will be called at all times, there may be other events of shutdown which does not allow the exitcalls to proceed such as a catastropic UML kernel panic or a "kill -9" from the host.. Regards Henrik |
From: Jon S. <jon...@ya...> - 2003-01-25 01:42:33
|
--- Henrik Nordstrom <hn...@ma...> wrote: > there may be other events of shutdown which does not > allow the exitcalls > to proceed such as a catastropic UML kernel panic or > a "kill -9" from the > host.. My implementation uses a helper driver in the host. This driver alloc/free's DMA memory and implements IRQ forwarding. To get DMA memory I intercept dma_alloc_coherent calls inside of UML. I then open the driver and IOCTL it to alloc the DMA memory on my behalf. I need to get the USB exitcall working. Once it works it will call dma_free_coherent. I can intercept this and free the buffers in the host. I was freeing the DMA memory when the driver file was closed. I'm changing this around now to only free the memory via explicit dma_free_coherent calls. I'm still releasing the IRQ lines when the driver closes. Now if UML chokes or gets a kill -9 the DMA buffers will stay allocated. This will cause some unowned DMA buffers to accumulate in the host but they are small <40K and this only occurs when UML does not exit cleanly. Hopefully this will give still active DMA somewhere to go. If you are sure that DMA isn't still active you can free the unowned buffers by unloading the host driver. ===== Jon Smirl jon...@ya... __________________________________________________ Do you Yahoo!? Yahoo! Mail Plus - Powerful. Affordable. Sign up now. http://mailplus.yahoo.com |
From: Henrik N. <hn...@ma...> - 2003-01-25 01:57:38
|
In my opinion you should stick to the "on file close" approach unless you have a very good reason why not to, and patch it at that end. As for the DMA still active case, the helper driver file close operation should not succed until the host state can be reset safely. If it needs to wait for the DMA operation to complete then it has to wait.. and this before it frees the DMA buffers.. but this obviously requires some knowledge of the DMA operations in the host helper driver.. Regards Henrik On Fri, 24 Jan 2003, Jon Smirl wrote: > --- Henrik Nordstrom <hn...@ma...> wrote: > > there may be other events of shutdown which does not > > allow the exitcalls > > to proceed such as a catastropic UML kernel panic or > > a "kill -9" from the > > host.. > > My implementation uses a helper driver in the host. > This driver alloc/free's DMA memory and implements IRQ > forwarding. > > To get DMA memory I intercept dma_alloc_coherent calls > inside of UML. I then open the driver and IOCTL it to > alloc the DMA memory on my behalf. > > I need to get the USB exitcall working. Once it works > it will call dma_free_coherent. I can intercept this > and free the buffers in the host. > > I was freeing the DMA memory when the driver file was > closed. I'm changing this around now to only free the > memory via explicit dma_free_coherent calls. I'm still > releasing the IRQ lines when the driver closes. > > Now if UML chokes or gets a kill -9 the DMA buffers > will stay allocated. This will cause some unowned DMA > buffers to accumulate in the host but they are small > <40K and this only occurs when UML does not exit > cleanly. Hopefully this will give still active DMA > somewhere to go. > > If you are sure that DMA isn't still active you can > free the unowned buffers by unloading the host driver. > > > > > > ===== > Jon Smirl > jon...@ya... > > __________________________________________________ > Do you Yahoo!? > Yahoo! Mail Plus - Powerful. Affordable. Sign up now. > http://mailplus.yahoo.com > |
From: Jon S. <jon...@ya...> - 2003-01-25 02:40:03
|
The helper driver in the host is unaware of what the drivers inside of UML have been up. Every driver starts and stop DMA in a different manner. There is no universal function for doing this. For example the USB subsystem starts and stops DMA by diddling an IO port. Take this case, UML is running with a USB driver inside of UML. This USB driver is accessing hardware in the host that the host is not using. The USB driver has diddled the IO port to allow DMA. Now you kill -9 UML. The USB driver is now gone but the USB hardware has DMA enabled. Now the host closes the helper driver handle. I can't wait for the USB DMA to finish since the host driver has no code for telling when it is going to finish and if is going to finish. All the helper driver knows is that a DMA buffer was allocated. The only safe thing to do in this case is to leave the buffer allocated. If on the other hand UML had exited cleanly, exitcalls would have called usb_exit. usb_exit would have terminated DMA and called dma_free_coherent to safely release the buffer. Now all I have to do is fix usb_exit to not hang. ===== Jon Smirl jon...@ya... __________________________________________________ Do you Yahoo!? Yahoo! Mail Plus - Powerful. Affordable. Sign up now. http://mailplus.yahoo.com |