From: Jeff D. <jd...@ad...> - 2005-09-17 23:07:17
|
I've spent some time working on the ubd and AIO problems that have cropped up recently. Patches are attached - a critical look at them would be appreciated. I'm going to start with a problem that hasn't exactly cropped up, and move on to the other ones from there. That is that the ubd driver must issue I/O requests to the host even if there is no memory available. It must make progress, because the request for which it can't get memory might be the swapping that will free some up. The driver allocates memory for several purposes - a header-type buffer per request which holds the bitmap data one aio request per chunk - a request may be broken into chunks because of COWing - some pieces from the backing file, some from the COW file To allow the driver to always make progress, even when no memory is available, I added a static instance of each buffer and made the kmalloc calls atomic. When kmalloc fails, then the static buffer is used. When it is in use, the user acquires a semaphore, which is released in the interrupt handler when the buffer is freed. This implies that future I/O submissions will sleep on the semaphore, so we have changed the reason for sleeping rather than eliminating it. Any I/O submissions which need memory will sleep on the semaphore until the buffer is released, resulting in synchronous, one at a time, I/O submission until the memory shortage has cleared up. Now that it is established that we must sleep, the next problem is making sure the requests get submitted to the host in the order that they were queued to the driver. Because we must sleep, we must drop the queue spinlock, which opens the queue to any other processes that want to do I/O. One of these can queue to the allocation semaphore (or just kmalloc memory now that some is available) and sneak ahead of a process handling an earlier request. To solve this, I add two atomic variables and a wait queue. One of the variables, started, counts the number of requests that have been submitted to the driver and the other, submitted, counts the number that have been submitted to the host. started is incremented before any submissions, submitted is incremented after all submissions for the request. The driver gets the value of started and waits for submitted to catch up to it. When that happens, it is now that requests's turn, and it wakes up from the sequence wake queue. I believe these three fixes - dropping the spinlock around host I/O submission, adding the static buffers, and sequencing requests - cover the scheduling while atomic problem and associated problems. Next is a deadlock that I found during a kernel build loop. The AIO thread seems to be able to fill the pipe back to UML with I/O completions. When this happens, it sleeps in a write and stops calling io_getevents. If UML keeps submitting events, the AIO ring buffer will fill, and io_submit will start returning -EAGAIN. At this point, UML will try to write the error to itself through the same pipe that the AIO thread has filled. It will now sleep. Since interrupts are off, that pipe can never be emptied since it is handled by the driver's interrupt routine, and we are deadlocked. Enabling interrups during the submission seems to help, but doesn't eliminate the deadlock. To fix it for real, I now have the -EAGAIN returned to the driver, which sleeps on a wait queue which is woken up by the interrupt routine. Interrupts are enabled, as they must be during sleep, so the handler can wake the queue after it has processed some completions, and the driver can retry. As for the interactions between the wait queues and the semaphores - the sequence wait queue controls access to everything else. Once a request is past that, it can only deadlock with itself. The allocation semaphore is used by a finishing request to wake up the next one. The eagain wait queue is taken in a loop with no sleeping on anything else. The patches are attached. They are: ubd-drop-lock - drops the ubd_io_lock around the call to do_io and takes it before dequeuing the next request. It also removes the call of do_ubd_request from the interrupt handler since the request handler now processes the entire queue in one call. ubd-atomic - makes the kmalloc calls atomic and adds the static buffers, associated semaphores and related synchronization. Unfinished - it doesn't have a static version of the bitmap, since that is variable length, so it's not so easy to have a static copy of it. ubd-sequence - makes the requests go out in the right order. Adds the sequence wait queue which controls access to the actual submission code. ubd-eagain - the -EAGAIN handling - waits in the eagain wait queue to be woken up by the interrupt handler after it has processed some completions. These are all against 2.6.14-rc1-mm1. They should apply to 2.6.14-rc1, but I haven't checked that. Jeff P.S. exmh insisted on a crazy attachment order - the order is the one listed above, not the order of attachment. |
From: Blaisorblade <bla...@ya...> - 2005-09-20 18:05:46
|
On Sunday 18 September 2005 00:45, Jeff Dike wrote: > I've spent some time working on the ubd and AIO problems that have cropped > up recently. Patches are attached - a critical look at them would be > appreciated. > > I'm going to start with a problem that hasn't exactly cropped up, and move > on to the other ones from there. > That is that the ubd driver must issue I/O requests to the host even if > there is no memory available. It must make progress, because the request > for which it can't get memory might be the swapping that will free some up. > The driver allocates memory for several purposes - > a header-type buffer per request which holds the bitmap data > one aio request per chunk - a request may be broken into chunks because > of COWing - some pieces from the backing file, some from the COW file > To allow the driver to always make progress, even when no memory is > available, I added a static instance of each buffer and made the kmalloc > calls atomic. When kmalloc fails, then the static buffer is used. Hmm, this kind of thing is exactly the one for which mempool's were created - have you looked at whether using them (which can be used for atomic purposes) would be better? > When it > is in use, the user acquires a semaphore, which is released in the > interrupt handler when the buffer is freed. This implies that future I/O > submissions will sleep on the semaphore, so we have changed the reason for > sleeping rather than eliminating it. I've not looked at the code, but have you tested that with sleep-inside-spinlock checking enabled? However, ok, you do release spinlocks so you should be safe. However, in your custom allocation routines, you're going to sleep possibly, so why do you use GFP_ATOMIC? There's absolutely no need. If there's a need, you can't take the semaphore afterwards. Also, GFP_KERNEL|GFP_ATOMIC is bogus - GFP_ATOMIC must be used alone, when needed. Grep kernel/*.c to make sure, or look at the definition. > Any I/O submissions which need memory will sleep on the semaphore until the > buffer is released, resulting in synchronous, one at a time, I/O submission > until the memory shortage has cleared up. About AIO, I've read on http://lse.sourceforge.net/io/aio.html that, indeed, the host AIO code isn't really Asynchronous for buffered I/O, but only for O_DIRECT I/O (which we don't seem to use). We're safe, sure, and the new code is anyhow better in using the new blockdevice model. > Now that it is established that we must sleep, the next problem is making > sure the requests get submitted to the host in the order that they were > queued to the driver. Because we must sleep, we must drop the queue > spinlock, which opens the queue to any other processes that want to do I/O. > One of these can queue to the allocation semaphore (or just kmalloc memory > now that some is available) and sneak ahead of a process handling an > earlier request. > To solve this, I add two atomic variables and a wait queue. One of the > variables, started, counts the number of requests that have been submitted > to the driver and the other, submitted, counts the number that have been > submitted to the host. started is incremented before any submissions, > submitted is incremented after all submissions for the request. > The driver gets the value of started and waits for submitted to catch up to > it. When that happens, it is now that requests's turn, and it wakes up > from the sequence wake queue. These two atomics + one wait queue are very similar to a semaphore, even if not identical. The semaphore value would be submitted - started. The change is that the driver sleep at the first increment of "started" rather than at the last one, but it should be ok. And much less error-prone. If you keep your custom design, you should at least unify the two vars with the difference. I'm not confident about the races between the two different variables, even if it may be perfectly safe (Tanenbaum uses something very similar, with even two vars, for the producer-consumer example). But it must be examined. And yes, on a semaphore you can call up() as many times as you want (which is non-obvious maybe - I didn't realize well until one week ago). In fact, provisions are made to initialize a semaphore with initial count bigger than 1. However, I would like to note that you're not always forced to sequence requests - write barriers were recently implemented, so the filesystem explicitly serializes requests when needed, rather than ask for strictly sequential processing. It's not especially hard to do, when you use your custom semaphore code - you can say "down() but don't sleep". > I believe these three fixes - dropping the spinlock around host I/O > submission, adding the static buffers, and sequencing requests - cover the > scheduling while atomic problem and associated problems. > Next is a deadlock that I found during a kernel build loop. The AIO thread > seems to be able to fill the pipe back to UML with I/O completions. When > this happens, it sleeps in a write and stops calling io_getevents. If UML > keeps submitting events, the AIO ring buffer will fill, and io_submit will > start returning -EAGAIN. At this point, UML will try to write the error to > itself through the same pipe that the AIO thread has filled. It will now > sleep. Since interrupts are off, that pipe can never be emptied since it is > handled by the driver's interrupt routine, and we are deadlocked. -- Inform me of my mistakes, so I can keep imitating Homer Simpson's "Doh!". Paolo Giarrusso, aka Blaisorblade (Skype ID "PaoloGiarrusso", ICQ 215621894) http://www.user-mode-linux.org/~blaisorblade ___________________________________ Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB http://mail.yahoo.it |
From: Jeff D. <jd...@ad...> - 2005-09-20 20:10:57
|
On Tue, Sep 20, 2005 at 02:01:11PM +0200, Blaisorblade wrote: > Hmm, this kind of thing is exactly the one for which mempool's were created - > have you looked at whether using them (which can be used for atomic purposes) > would be better? Yeah, that would be worth looking into. > I've not looked at the code, but have you tested that with > sleep-inside-spinlock checking enabled? > > However, ok, you do release spinlocks so you should be safe. However, in your > custom allocation routines, you're going to sleep possibly, so why do you use > GFP_ATOMIC? There's absolutely no need. If there's a need, you can't take the > semaphore afterwards. ? GFP_ATOMIC doesn't always mean that you're in an interrupt. Generally, it means not to sleep in kmalloc. And here, if it fails, I'll use the static buffers. > Also, GFP_KERNEL|GFP_ATOMIC is bogus - GFP_ATOMIC must be used alone, when > needed. OK. > About AIO, I've read on http://lse.sourceforge.net/io/aio.html that, indeed, > the host AIO code isn't really Asynchronous for buffered I/O, but only for > O_DIRECT I/O (which we don't seem to use). There's another patch, called o_direct, which I didn't send out, which fixes this. > These two atomics + one wait queue are very similar to a semaphore, even if > not identical. The semaphore value would be submitted - started. The change > is that the driver sleep at the first increment of "started" rather than at > the last one, but it should be ok. And much less error-prone. If you keep > your custom design, you should at least unify the two vars with the > difference. The wait queue allows the correct thread to be woken up. If I used a semaphore, its value would be the same for all threads, and they would all be woken up when that value goes to 0. With a wait queue, each thread has a different value of started, and they wait for submitted to catch up to it. Meanwhile, any other sleeping threads stay sleeping because submitted hasn't caught up to their values of started. > However, I would like to note that you're not always forced to sequence > requests - write barriers were recently implemented, so the filesystem > explicitly serializes requests when needed, rather than ask for strictly > sequential processing. It's not especially hard to do, when you use your > custom semaphore code - you can say "down() but don't sleep". I'm concerned about the COW bitmap. That's something that the upper layers know nothing about, so requests could overlap without there being barriers between that. However, this is an artifact of the implementation, where the section of bitmap that will be written out is copied into the aio request when it is started. If I grabbed the bitmap section and set the bits in it just before it is written out, then the sequencing stuff might be able to just go away. We rely on the block layer to put write barriers between requests with overlapping data. Jeff |
From: Blaisorblade <bla...@ya...> - 2005-09-21 15:51:30
|
On Tuesday 20 September 2005 21:06, Jeff Dike wrote: > On Tue, Sep 20, 2005 at 02:01:11PM +0200, Blaisorblade wrote: > > Hmm, this kind of thing is exactly the one for which mempool's were > > created - have you looked at whether using them (which can be used for > > atomic purposes) would be better? > Yeah, that would be worth looking into. > > I've not looked at the code, but have you tested that with > > sleep-inside-spinlock checking enabled? > > However, ok, you do release spinlocks so you should be safe. However, in > > your custom allocation routines, you're going to sleep possibly, so why > > do you use GFP_ATOMIC? There's absolutely no need. If there's a need, you > > can't take the semaphore afterwards. > ? > GFP_ATOMIC doesn't always mean that you're in an interrupt. > Generally, it means not to sleep in kmalloc. Yes, but GFP_ATOMIC was needed in first place because we where in an atomic section (not for interrupts, but because we were under a spinlock). > And here, if it fails, > I'll use the static buffers. Since you're going to possibly sleep, it's better to allow kmalloc to sleep in first place! > > These two atomics + one wait queue are very similar to a semaphore, even > > if not identical. The semaphore value would be submitted - started. The > > change is that the driver sleep at the first increment of "started" > > rather than at the last one, but it should be ok. And much less > > error-prone. If you keep your custom design, you should at least unify > > the two vars with the difference. > The wait queue allows the correct thread to be woken up. If I used a > semaphore, its value would be the same for all threads, and they would > all be woken up when that value goes to 0. With a wait queue, each > thread has a different value of started, and they wait for submitted > to catch up to it. Ok, filtered wait queues. And I also see why you used two atomics rather than one. I'll have to think deep about this, though. > Meanwhile, any other sleeping threads stay > sleeping because submitted hasn't caught up to their values of started. [...] > I'm concerned about the COW bitmap. That's something that the upper > layers know nothing about, so requests could overlap without there > being barriers between that. > However, this is an artifact of the implementation, where the section > of bitmap that will be written out is copied into the aio request when > it is started. If I grabbed the bitmap section and set the bits in it > just before it is written out, then the sequencing stuff might be able > to just go away. Is the COW bitmap mmap'ed? Because otherwise we'd need to queue up an additional write request, which creates problems. Btw, msync() allows sync'ing only a specific region, which fsync() doesn't allow. Don't know whether we need this really, however (we probably don't do it, but we should). Also, I kept forgetting to mention one thing: Device mapper has the support for COW volumes, like we do. E.g. when you create a snapshot, it is not a static immutable copy of the original, it's writable too! Sure, less granular than COW devices, it doesn't copy 512 bytes at a time (which may help with performance though), but works on the host too! I was waiting to let lists know of this after writing a conversion program, which I never finished. > We rely on the block layer to put write barriers > between requests with overlapping data. Don't think it's reasonable to expect this. I think that filesystems like ext2 will never produce write barriers - they are needed for journaled FS's. > Jeff -- Inform me of my mistakes, so I can keep imitating Homer Simpson's "Doh!". Paolo Giarrusso, aka Blaisorblade (Skype ID "PaoloGiarrusso", ICQ 215621894) http://www.user-mode-linux.org/~blaisorblade ___________________________________ Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB http://mail.yahoo.it |
From: Jeff D. <jd...@ad...> - 2005-09-21 18:28:52
|
On Wed, Sep 21, 2005 at 05:49:43PM +0200, Blaisorblade wrote: > Since you're going to possibly sleep, it's better to allow kmalloc to sleep in > first place! WRONG! The first request in line can't sleep, that's the whole point. If it does, the system can deadlock. Subsequent requests can sleep, waiting for previous ones to finish, but there absolutely can't be any sleeping in kmalloc. > Is the COW bitmap mmap'ed? Because otherwise we'd need to queue up an > additional write request, which creates problems. No. That creates ordering problems which I haven't thought about yet, It may be worth doing, but explicit writes make ordering easier to think about. > > Btw, msync() allows sync'ing only a specific region, which fsync() doesn't > allow. Don't know whether we need this really, however (we probably don't do > it, but we should). Or, the host may just flush out the mmapped stuff on its own. This is my concern about ordering. > Also, I kept forgetting to mention one thing: Device mapper has the support > for COW volumes, like we do. E.g. when you create a snapshot, it is not a > static immutable copy of the original, it's writable too! Yeah, that's something to think about. > Don't think it's reasonable to expect this. I think that filesystems like ext2 > will never produce write barriers - they are needed for journaled FS's. I thought your point was to rely on the block layer to order overlapping writes for us. It sounded reasonable to me :-) Jeff |
From: Blaisorblade <bla...@ya...> - 2005-09-21 19:08:04
|
On Wednesday 21 September 2005 20:04, Jeff Dike wrote: > On Wed, Sep 21, 2005 at 05:49:43PM +0200, Blaisorblade wrote: [...] > > Don't think it's reasonable to expect this. I think that filesystems like > > ext2 will never produce write barriers - they are needed for journaled > > FS's. > I thought your point was to rely on the block layer to order overlapping > writes for us. The block layer is supposed to merge as far as possible overlapping writes, that's reasonable, but not dependable. Say it gets the overlapping request *after* it sent the first one - we'll see both. Also, this is trivially true for output done through page cache, but for the rest I don't know if explicit merging is implemented. For the buffer cache (i.e. fs metadata) it may still hold, but I'm not sure at all. Not that it matters for us - we can't depend on it anyway. No, I was thinking to you enforcing ordering to comply to what journaled filesystems expect. Btw: even when we aren't using COW, we're supposed to do writes in the order the fs passed them to us - at least when write barriers are explicitly sent (don't know if there's something to care normally - I know this as a LWN.net reader, not as an hacker in this area). > It sounded reasonable to me :-) > Jeff -- Inform me of my mistakes, so I can keep imitating Homer Simpson's "Doh!". Paolo Giarrusso, aka Blaisorblade (Skype ID "PaoloGiarrusso", ICQ 215621894) http://www.user-mode-linux.org/~blaisorblade ___________________________________ Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB http://mail.yahoo.it |
From: Jeff D. <jd...@ad...> - 2005-09-21 21:28:42
|
On Wed, Sep 21, 2005 at 09:06:38PM +0200, Blaisorblade wrote: > The block layer is supposed to merge as far as possible overlapping writes, > that's reasonable, but not dependable. Say it gets the overlapping request > *after* it sent the first one - we'll see both. Also, this is trivially true > for output done through page cache, but for the rest I don't know if explicit > merging is implemented. Yeah, I'm concerned about O_DIRECT. Also, what happens when we write a page to disk, and immediately afterwards, it gets dirty again and written again? Is there a barrier between the two? > No, I was thinking to you enforcing ordering to comply to what journaled > filesystems expect. Yeah, but there's more ordering than that. > Btw: even when we aren't using COW, we're supposed to do writes in the order > the fs passed them to us - at least when write barriers are explicitly sent > (don't know if there's something to care normally - I know this as a LWN.net > reader, not as an hacker in this area). We get requests from the block layer, not the fs. And physical disks are allowed to reorder requests, so I think there is some flexibility there. Jeff |
From: Blaisorblade <bla...@ya...> - 2005-09-22 20:52:48
|
On Wednesday 21 September 2005 22:45, Jeff Dike wrote: > On Wed, Sep 21, 2005 at 09:06:38PM +0200, Blaisorblade wrote: > > The block layer is supposed to merge as far as possible overlapping > > writes, that's reasonable, but not dependable. Say it gets the > > overlapping request *after* it sent the first one - we'll see both. Also, > > this is trivially true for output done through page cache, but for the > > rest I don't know if explicit merging is implemented. > > Yeah, I'm concerned about O_DIRECT. Also, what happens when we write a > page to disk, and immediately afterwards, it gets dirty again and written > again? Is there a barrier between the two? > > > No, I was thinking to you enforcing ordering to comply to what journaled > > filesystems expect. > > Yeah, but there's more ordering than that. > > > Btw: even when we aren't using COW, we're supposed to do writes in the > > order the fs passed them to us - at least when write barriers are > > explicitly sent (don't know if there's something to care normally - I > > know this as a LWN.net reader, not as an hacker in this area). > > We get requests from the block layer, not the fs. And physical disks are > allowed to reorder requests, so I think there is some flexibility there. I expect the FS to send the barriers - they don't make sense otherwise. In fact, grep -i barrier fs/jbd/*.c gives some results, which point to BH_Ordered. And after grepping it in reiserfs, I found that fs/buffer.c:submit_bh() cares about it (i.e. looks at the buffer head and sees if it is ordered), and sends it down to the elevator. -- Inform me of my mistakes, so I can keep imitating Homer Simpson's "Doh!". Paolo Giarrusso, aka Blaisorblade (Skype ID "PaoloGiarrusso", ICQ 215621894) http://www.user-mode-linux.org/~blaisorblade ___________________________________ Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB http://mail.yahoo.it |
From: Jeff D. <jd...@ad...> - 2005-09-27 18:22:40
|
Attached is my current set of I/O patches. It's significantly different, and simpler, than my previous set. Now, there is no sleeping while running the queue, so the spinlock is never dropped, and all of the extra synchronization is gone. do_ubd_handler runs the queue until it's empty, or the host refuses to take any more AIO requests. If the queue not empty, then the current state of the in-process request (the request itself and which sg entries have not yet been sent to the host) is saved in the device structure. When the interrupt handler empties out the host some, it will take the queue lock and call the request handler to push some more requests to the host. I also made the queues and locks per-device rather than having one for all devices. This means that when a device gets -EAGAIN from the host, the interrupt handler needs to know which queues got stalled. This is handled by having the request handler stick the device on a list when this happems, and the interrupt handler walks that list when rerunning request handlers. The patch name is the same as before, and now quite misleading because the queue lock is now not dropped. ubd-atomic is much the same as before, except simple flags is used to indicate whether the static buffers are available. This is OK since any reading or writing of the flags happens under the queue spinlock. init_aio_err is a simple error path cleanup patch. aio-batching causes the AIO thread not to process completions until the current batch of I/O is submitted. I added this because I was seeing a context switch between UML and the AIO thread on every AIO submission. This keeps the AIO thread asleep until the current I/O is fully submitted and it may be able to process a bunch of completions at once. o_direct adds O_DIRECT support to UML and makes the ubd driver use it. aio-errors makes submit_aio_24 return -errno instead of -1. I've given this a bunch of testing and it has survived overnight kernel build loops on both x86 and x86_64. The attached patches are against 2.6.14-rc2. Jeff |
From: Blaisorblade <bla...@ya...> - 2005-09-28 12:15:17
|
On Tuesday 27 September 2005 20:12, Jeff Dike wrote: > Attached is my current set of I/O patches. It's significantly different, > and simpler, than my previous set. > Now, there is no sleeping while running the queue, so the spinlock is never > dropped, and all of the extra synchronization is gone. Very, very nice. What about the early removal of the request from the queue, before being sure we can complete it (sorry for not reading carefully everything below, I'm going to have lunch and afterwards back to study). > o_direct adds O_DIRECT support to UML and makes the ubd driver use it. > > aio-errors makes submit_aio_24 return -errno instead of -1. Reorder o_direct at the end - no time to read them yet, but guess that more intrusive goes at the end. And I'm not sure whether merging o_direct so late in 2.6.14 is nice (even if you've tested the full patch set, and without o_direct the thing makes less sense). Let's hope for the better, but next time the debug should be done *before* merging, ok? Even because, for instance, if (say) the patch was ready for 2.6.14 and merged into 2.6.15, I could have released a test tree against 2.6.14 (maybe I could do this now). > I've given this a bunch of testing and it has survived overnight kernel > build loops on both x86 and x86_64. > The attached patches are against 2.6.14-rc2. Please, send them to Andrew for -mm, and say "2.6.14 can't go without them, but they might need further review". And ask Jens Axboe if it can give a look to them (this time it's less necessary, since there are less dirty tricks). > Jeff -- Inform me of my mistakes, so I can keep imitating Homer Simpson's "Doh!". Paolo Giarrusso, aka Blaisorblade (Skype ID "PaoloGiarrusso", ICQ 215621894) http://www.user-mode-linux.org/~blaisorblade ___________________________________ Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB http://mail.yahoo.it |
From: Jeff D. <jd...@ad...> - 2005-09-28 16:18:53
|
On Wed, Sep 28, 2005 at 02:14:32PM +0200, Blaisorblade wrote: > Very, very nice. > > What about the early removal of the request from the queue, before being sure > we can complete it (sorry for not reading carefully everything below, I'm > going to have lunch and afterwards back to study). I think that doesn't matter too much. I have to get the sg information out of it, and store that, plus the current position within the sg array if we couldn't send it all to the host. So, I don't refer to the request after I get the sg information out of it, so I might as well remove it from the queue. > Reorder o_direct at the end - no time to read them yet, but guess that more > intrusive goes at the end. And I'm not sure whether merging o_direct so late > in 2.6.14 is nice (even if you've tested the full patch set, and without > o_direct the thing makes less sense). Yeah, I wasn't planning on sending that for 2.6.14. It was just part of that set of patches. > Let's hope for the better, but next time the debug should be done *before* > merging, ok? Even because, for instance, if (say) the patch was ready for > 2.6.14 and merged into 2.6.15, I could have released a test tree against > 2.6.14 (maybe I could do this now). I thought I did. AIO had been in my tree forever with no apparent problems. I recently started running 24 hour/day stress tests on UML, and that turned up some problems. People turning on spinlock debugging turned up the sleeping while atomic problem. And me staring at the code as a result of that turned up the possible deadlock. > Please, send them to Andrew for -mm, and say "2.6.14 can't go without them, > but they might need further review". > > And ask Jens Axboe if it can give a look to them (this time it's less > necessary, since there are less dirty tricks). Yeah. This still needs some work. I need to deal with the bitmap array properly. Jeff |
From: Blaisorblade <bla...@ya...> - 2005-09-28 16:48:13
|
On Wednesday 28 September 2005 17:54, Jeff Dike wrote: > On Wed, Sep 28, 2005 at 02:14:32PM +0200, Blaisorblade wrote: > > Very, very nice. > > Let's hope for the better, but next time the debug should be done > > *before* merging, ok? Even because, for instance, if (say) the patch was > > ready for 2.6.14 and merged into 2.6.15, I could have released a test > > tree against 2.6.14 (maybe I could do this now). > I thought I did. AIO had been in my tree forever with no apparent > problems. We're not currently getting a lot of testing on that. Apart the really current issue of the page not uptodate, users mostly ran that because of x86_64 support, which is now reliable in mainline. In fact, the SKAS0 compiler-dependant bug (the asm code was popping a fixed number of words, which was wrong with different compiler) only came out on the tree I released against 2.6.12... > I recently started running 24 hour/day stress tests on UML, and > that turned up some problems. People turning on spinlock debugging turned > up the sleeping while atomic problem. And me staring at the code as a > result of that turned up the possible deadlock. > > And ask Jens Axboe if it can give a look to them (this time it's less > > necessary, since there are less dirty tricks). > Yeah. This still needs some work. I need to deal with the bitmap array > properly. > Jeff -- Inform me of my mistakes, so I can keep imitating Homer Simpson's "Doh!". Paolo Giarrusso, aka Blaisorblade (Skype ID "PaoloGiarrusso", ICQ 215621894) http://www.user-mode-linux.org/~blaisorblade ___________________________________ Yahoo! Messenger: chiamate gratuite in tutto il mondo http://it.messenger.yahoo.com |