|
From: Edward H. <qua...@op...> - 2010-10-04 20:13:19
|
I am attempting to port the Broadcom tg3 NetXtreme linux driver to rtnet. The Broadcom driver internally utilizes spin_lock_bh/spin_unlock_bh to disable/enable software interrupts. There is no rtdm lock equivalent. I believe that software interrupts are used by the driver for the following reasons: 1. To allow for simultaneous DMA read write operations 2. To use shared local variables in a critical section of the driver. 3. To receive changes to settings variables in realtime from the userpace Nextreme utilities. (not sure about this one). What RTDM lock functions should I use to substitute for spin_lock_bh/spin_unlock_bh? Should I substitute rtdm_lock_get_irqsave/rtdm_lock_put_irqrestore? Or does rtdm need to ne extended for softirqs used internally by the driver? I looked through the other device drivers and could not detect the correct substitute. RSVP. Thanks. Best Regards, Ed |
|
From: Jan K. <jan...@si...> - 2010-10-07 09:09:36
|
Am 04.10.2010 22:13, Edward Hoffman wrote: > I am attempting to port the Broadcom tg3 NetXtreme linux driver to > rtnet. Your port will be welcome! > The Broadcom driver internally utilizes > spin_lock_bh/spin_unlock_bh to disable/enable software interrupts. > There is no rtdm lock equivalent. > I believe that software interrupts are used by the driver for the > following reasons: > 1. To allow for simultaneous DMA read write operations > 2. To use shared local variables in a critical section of the driver. > 3. To receive changes to settings variables in realtime from the > userpace Nextreme utilities. (not sure about this one). > > What RTDM lock functions should I use to substitute for > spin_lock_bh/spin_unlock_bh? Should I substitute > rtdm_lock_get_irqsave/rtdm_lock_put_irqrestore? Or does rtdm need to ne > extended for softirqs used internally by the driver? > > I looked through the other device drivers and could not detect the > correct substitute. As RTDM provides no abstraction of bottom-halves, there are also no specific locking services to disable them. You have to use IRQ-disabling services unless you are in IRQ context. Jan -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux |
|
From: Gilles C. <gil...@xe...> - 2010-10-07 09:27:22
|
Jan Kiszka wrote: > Am 04.10.2010 22:13, Edward Hoffman wrote: >> I am attempting to port the Broadcom tg3 NetXtreme linux driver to >> rtnet. > > Your port will be welcome! > >> The Broadcom driver internally utilizes >> spin_lock_bh/spin_unlock_bh to disable/enable software interrupts. >> There is no rtdm lock equivalent. >> I believe that software interrupts are used by the driver for the >> following reasons: >> 1. To allow for simultaneous DMA read write operations >> 2. To use shared local variables in a critical section of the driver. >> 3. To receive changes to settings variables in realtime from the >> userpace Nextreme utilities. (not sure about this one). >> >> What RTDM lock functions should I use to substitute for >> spin_lock_bh/spin_unlock_bh? Should I substitute >> rtdm_lock_get_irqsave/rtdm_lock_put_irqrestore? Or does rtdm need to ne >> extended for softirqs used internally by the driver? >> >> I looked through the other device drivers and could not detect the >> correct substitute. > > As RTDM provides no abstraction of bottom-halves, there are also no > specific locking services to disable them. You have to use IRQ-disabling > services unless you are in IRQ context. Maybe if you want to only exclude the driver's irq, you may only disable the irq line at IC level, with rtdm_irq_disable? -- Gilles. |
|
From: Jan K. <jan...@si...> - 2010-10-07 09:22:46
|
Am 07.10.2010 11:13, Gilles Chanteperdrix wrote: > Jan Kiszka wrote: >> Am 04.10.2010 22:13, Edward Hoffman wrote: >>> I am attempting to port the Broadcom tg3 NetXtreme linux driver to >>> rtnet. >> >> Your port will be welcome! >> >>> The Broadcom driver internally utilizes >>> spin_lock_bh/spin_unlock_bh to disable/enable software interrupts. >>> There is no rtdm lock equivalent. >>> I believe that software interrupts are used by the driver for the >>> following reasons: >>> 1. To allow for simultaneous DMA read write operations >>> 2. To use shared local variables in a critical section of the driver. >>> 3. To receive changes to settings variables in realtime from the >>> userpace Nextreme utilities. (not sure about this one). >>> >>> What RTDM lock functions should I use to substitute for >>> spin_lock_bh/spin_unlock_bh? Should I substitute >>> rtdm_lock_get_irqsave/rtdm_lock_put_irqrestore? Or does rtdm need to ne >>> extended for softirqs used internally by the driver? >>> >>> I looked through the other device drivers and could not detect the >>> correct substitute. >> >> As RTDM provides no abstraction of bottom-halves, there are also no >> specific locking services to disable them. You have to use IRQ-disabling >> services unless you are in IRQ context. > > Maybe if you want to only exclude the driver's irq, you may only disable > the irq line at IC level, with rtdm_irq_disable? Usually discouraged due to potentially slow line-disabling services and the breakage this will cause when the line is shared. Jan -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux |
|
From: Gilles C. <gil...@xe...> - 2010-10-07 09:27:48
|
Jan Kiszka wrote: > Am 07.10.2010 11:13, Gilles Chanteperdrix wrote: >> Jan Kiszka wrote: >>> Am 04.10.2010 22:13, Edward Hoffman wrote: >>>> I am attempting to port the Broadcom tg3 NetXtreme linux driver to >>>> rtnet. >>> Your port will be welcome! >>> >>>> The Broadcom driver internally utilizes >>>> spin_lock_bh/spin_unlock_bh to disable/enable software interrupts. >>>> There is no rtdm lock equivalent. >>>> I believe that software interrupts are used by the driver for the >>>> following reasons: >>>> 1. To allow for simultaneous DMA read write operations >>>> 2. To use shared local variables in a critical section of the driver. >>>> 3. To receive changes to settings variables in realtime from the >>>> userpace Nextreme utilities. (not sure about this one). >>>> >>>> What RTDM lock functions should I use to substitute for >>>> spin_lock_bh/spin_unlock_bh? Should I substitute >>>> rtdm_lock_get_irqsave/rtdm_lock_put_irqrestore? Or does rtdm need to ne >>>> extended for softirqs used internally by the driver? >>>> >>>> I looked through the other device drivers and could not detect the >>>> correct substitute. >>> As RTDM provides no abstraction of bottom-halves, there are also no >>> specific locking services to disable them. You have to use IRQ-disabling >>> services unless you are in IRQ context. >> Maybe if you want to only exclude the driver's irq, you may only disable >> the irq line at IC level, with rtdm_irq_disable? > > Usually discouraged due to potentially slow line-disabling services and > the breakage this will cause when the line is shared. Why would this cause breakage when the line is shared? -- Gilles. |
|
From: Jan K. <jan...@si...> - 2010-10-07 09:33:51
|
Am 07.10.2010 11:27, Gilles Chanteperdrix wrote: > Jan Kiszka wrote: >> Am 07.10.2010 11:13, Gilles Chanteperdrix wrote: >>> Jan Kiszka wrote: >>>> Am 04.10.2010 22:13, Edward Hoffman wrote: >>>>> I am attempting to port the Broadcom tg3 NetXtreme linux driver to >>>>> rtnet. >>>> Your port will be welcome! >>>> >>>>> The Broadcom driver internally utilizes >>>>> spin_lock_bh/spin_unlock_bh to disable/enable software interrupts. >>>>> There is no rtdm lock equivalent. >>>>> I believe that software interrupts are used by the driver for the >>>>> following reasons: >>>>> 1. To allow for simultaneous DMA read write operations >>>>> 2. To use shared local variables in a critical section of the driver. >>>>> 3. To receive changes to settings variables in realtime from the >>>>> userpace Nextreme utilities. (not sure about this one). >>>>> >>>>> What RTDM lock functions should I use to substitute for >>>>> spin_lock_bh/spin_unlock_bh? Should I substitute >>>>> rtdm_lock_get_irqsave/rtdm_lock_put_irqrestore? Or does rtdm need to ne >>>>> extended for softirqs used internally by the driver? >>>>> >>>>> I looked through the other device drivers and could not detect the >>>>> correct substitute. >>>> As RTDM provides no abstraction of bottom-halves, there are also no >>>> specific locking services to disable them. You have to use IRQ-disabling >>>> services unless you are in IRQ context. >>> Maybe if you want to only exclude the driver's irq, you may only disable >>> the irq line at IC level, with rtdm_irq_disable? >> >> Usually discouraged due to potentially slow line-disabling services and >> the breakage this will cause when the line is shared. > > Why would this cause breakage when the line is shared? If the disabling context is preemptible (which is normally the motivation when avoiding irqsave), the line may remain off until that task happens to be scheduled in again. That can easily cause prio-inversions to other devices sharing the line. Jan -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux |
|
From: Gilles C. <gil...@xe...> - 2010-10-07 09:41:35
|
Jan Kiszka wrote: > Am 07.10.2010 11:27, Gilles Chanteperdrix wrote: >> Jan Kiszka wrote: >>> Am 07.10.2010 11:13, Gilles Chanteperdrix wrote: >>>> Jan Kiszka wrote: >>>>> Am 04.10.2010 22:13, Edward Hoffman wrote: >>>>>> I am attempting to port the Broadcom tg3 NetXtreme linux driver to >>>>>> rtnet. >>>>> Your port will be welcome! >>>>> >>>>>> The Broadcom driver internally utilizes >>>>>> spin_lock_bh/spin_unlock_bh to disable/enable software interrupts. >>>>>> There is no rtdm lock equivalent. >>>>>> I believe that software interrupts are used by the driver for the >>>>>> following reasons: >>>>>> 1. To allow for simultaneous DMA read write operations >>>>>> 2. To use shared local variables in a critical section of the driver. >>>>>> 3. To receive changes to settings variables in realtime from the >>>>>> userpace Nextreme utilities. (not sure about this one). >>>>>> >>>>>> What RTDM lock functions should I use to substitute for >>>>>> spin_lock_bh/spin_unlock_bh? Should I substitute >>>>>> rtdm_lock_get_irqsave/rtdm_lock_put_irqrestore? Or does rtdm need to ne >>>>>> extended for softirqs used internally by the driver? >>>>>> >>>>>> I looked through the other device drivers and could not detect the >>>>>> correct substitute. >>>>> As RTDM provides no abstraction of bottom-halves, there are also no >>>>> specific locking services to disable them. You have to use IRQ-disabling >>>>> services unless you are in IRQ context. >>>> Maybe if you want to only exclude the driver's irq, you may only disable >>>> the irq line at IC level, with rtdm_irq_disable? >>> Usually discouraged due to potentially slow line-disabling services and >>> the breakage this will cause when the line is shared. >> Why would this cause breakage when the line is shared? > > If the disabling context is preemptible (which is normally the > motivation when avoiding irqsave), the line may remain off until that > task happens to be scheduled in again. That can easily cause > prio-inversions to other devices sharing the line. Yes, we should obviously disable preemption when doing that. What would probably work is: - disabling preemption - disabling the irq line - locking the spinlock. Maybe that could be bundled in rtdm_lock_get_disable_irq(lock, irq) ? Disabling an irq line should be pretty fast on non-x86, besides, if the original code used spin_lock_bh, it is maybe because the critical section is expected to be lengthy, so irqsave/irqrestore may cause high latencies in the rest of the system. -- Gilles. |
|
From: Jan K. <jan...@si...> - 2010-10-07 10:52:39
|
Am 07.10.2010 11:41, Gilles Chanteperdrix wrote: > Jan Kiszka wrote: >> Am 07.10.2010 11:27, Gilles Chanteperdrix wrote: >>> Jan Kiszka wrote: >>>> Am 07.10.2010 11:13, Gilles Chanteperdrix wrote: >>>>> Jan Kiszka wrote: >>>>>> Am 04.10.2010 22:13, Edward Hoffman wrote: >>>>>>> I am attempting to port the Broadcom tg3 NetXtreme linux driver to >>>>>>> rtnet. >>>>>> Your port will be welcome! >>>>>> >>>>>>> The Broadcom driver internally utilizes >>>>>>> spin_lock_bh/spin_unlock_bh to disable/enable software interrupts. >>>>>>> There is no rtdm lock equivalent. >>>>>>> I believe that software interrupts are used by the driver for the >>>>>>> following reasons: >>>>>>> 1. To allow for simultaneous DMA read write operations >>>>>>> 2. To use shared local variables in a critical section of the driver. >>>>>>> 3. To receive changes to settings variables in realtime from the >>>>>>> userpace Nextreme utilities. (not sure about this one). >>>>>>> >>>>>>> What RTDM lock functions should I use to substitute for >>>>>>> spin_lock_bh/spin_unlock_bh? Should I substitute >>>>>>> rtdm_lock_get_irqsave/rtdm_lock_put_irqrestore? Or does rtdm need to ne >>>>>>> extended for softirqs used internally by the driver? >>>>>>> >>>>>>> I looked through the other device drivers and could not detect the >>>>>>> correct substitute. >>>>>> As RTDM provides no abstraction of bottom-halves, there are also no >>>>>> specific locking services to disable them. You have to use IRQ-disabling >>>>>> services unless you are in IRQ context. >>>>> Maybe if you want to only exclude the driver's irq, you may only disable >>>>> the irq line at IC level, with rtdm_irq_disable? >>>> Usually discouraged due to potentially slow line-disabling services and >>>> the breakage this will cause when the line is shared. >>> Why would this cause breakage when the line is shared? >> >> If the disabling context is preemptible (which is normally the >> motivation when avoiding irqsave), the line may remain off until that >> task happens to be scheduled in again. That can easily cause >> prio-inversions to other devices sharing the line. > > Yes, we should obviously disable preemption when doing that. What would > probably work is: > - disabling preemption > - disabling the irq line > - locking the spinlock. > > Maybe that could be bundled in rtdm_lock_get_disable_irq(lock, irq) ? > Disabling an irq line should be pretty fast on non-x86, besides, if the > original code used spin_lock_bh, it is maybe because the critical > section is expected to be lengthy, so irqsave/irqrestore may cause high > latencies in the rest of the system. Generally spoken, disabling preemption can cause high latencies as well as most critical paths include a task switch. However, anything that is more light-weight than irqsave/restore and has compatible semantics would still be welcome. It just requires some relevant performance and/or latency gain to justify complications in the locking model. We actually have two or so cases in RTnet where drivers contain long locked code paths (orders of a few hundred microseconds). In those cases, disabling the IRQ line without any preemption lock is the only way to keep system latencies in reasonable limits. But those are still corner cases. Jan -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux |
|
From: Gilles C. <gil...@xe...> - 2010-10-07 11:01:12
|
Jan Kiszka wrote: >> Maybe that could be bundled in rtdm_lock_get_disable_irq(lock, irq) ? >> Disabling an irq line should be pretty fast on non-x86, besides, if the >> original code used spin_lock_bh, it is maybe because the critical >> section is expected to be lengthy, so irqsave/irqrestore may cause high >> latencies in the rest of the system. > > Generally spoken, disabling preemption can cause high latencies as well > as most critical paths include a task switch. However, anything that is > more light-weight than irqsave/restore and has compatible semantics > would still be welcome. It just requires some relevant performance > and/or latency gain to justify complications in the locking model. > > We actually have two or so cases in RTnet where drivers contain long > locked code paths (orders of a few hundred microseconds). In those > cases, disabling the IRQ line without any preemption lock is the only > way to keep system latencies in reasonable limits. But those are still > corner cases. The interrupt latency also matters in some cases (which is why, for instance, Xenomai switches contexts with irqs on ARM and powerpc now, this does not improve scheduling latency, only irq latency). This new kind of spinlocking would be a solution to replace spinlock_bh. I suspect the drivers which will cause long critical sections are the ones where the packets needs to be copied using PIO, instead of DMA. Such drivers still exist, especially in the non-x86 word. -- Gilles. |
|
From: Jan K. <jan...@si...> - 2010-10-07 11:18:56
|
Am 07.10.2010 13:00, Gilles Chanteperdrix wrote: > Jan Kiszka wrote: >>> Maybe that could be bundled in rtdm_lock_get_disable_irq(lock, irq) ? >>> Disabling an irq line should be pretty fast on non-x86, besides, if the >>> original code used spin_lock_bh, it is maybe because the critical >>> section is expected to be lengthy, so irqsave/irqrestore may cause high >>> latencies in the rest of the system. >> >> Generally spoken, disabling preemption can cause high latencies as well >> as most critical paths include a task switch. However, anything that is >> more light-weight than irqsave/restore and has compatible semantics >> would still be welcome. It just requires some relevant performance >> and/or latency gain to justify complications in the locking model. >> >> We actually have two or so cases in RTnet where drivers contain long >> locked code paths (orders of a few hundred microseconds). In those >> cases, disabling the IRQ line without any preemption lock is the only >> way to keep system latencies in reasonable limits. But those are still >> corner cases. > > The interrupt latency also matters in some cases (which is why, for > instance, Xenomai switches contexts with irqs on ARM and powerpc now, > this does not improve scheduling latency, only irq latency). This new > kind of spinlocking would be a solution to replace spinlock_bh. We still have no BH in Xenomai to protect against (and I don't think we want them). So every porting case still needs careful analysis to translate from the Linux model to RTDM. What would be possible with a preemption-disabling spin_lock is inter-task synchronization without IRQ disabling - kind of light-weight mutexes (/wrt their worst case, ie. contention). > > I suspect the drivers which will cause long critical sections are the > ones where the packets needs to be copied using PIO, instead of DMA. > Such drivers still exist, especially in the non-x86 word. Right, but not all have excessive access times. And we should better address the expensive cases with IRQ threads (which is an overdue to-do for RTDM) so that normal mutexes can be used. Jan -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux |
|
From: Gilles C. <gil...@xe...> - 2010-10-07 12:12:12
|
Jan Kiszka wrote: > Am 07.10.2010 13:00, Gilles Chanteperdrix wrote: >> Jan Kiszka wrote: >>>> Maybe that could be bundled in rtdm_lock_get_disable_irq(lock, irq) ? >>>> Disabling an irq line should be pretty fast on non-x86, besides, if the >>>> original code used spin_lock_bh, it is maybe because the critical >>>> section is expected to be lengthy, so irqsave/irqrestore may cause high >>>> latencies in the rest of the system. >>> Generally spoken, disabling preemption can cause high latencies as well >>> as most critical paths include a task switch. However, anything that is >>> more light-weight than irqsave/restore and has compatible semantics >>> would still be welcome. It just requires some relevant performance >>> and/or latency gain to justify complications in the locking model. >>> >>> We actually have two or so cases in RTnet where drivers contain long >>> locked code paths (orders of a few hundred microseconds). In those >>> cases, disabling the IRQ line without any preemption lock is the only >>> way to keep system latencies in reasonable limits. But those are still >>> corner cases. >> The interrupt latency also matters in some cases (which is why, for >> instance, Xenomai switches contexts with irqs on ARM and powerpc now, >> this does not improve scheduling latency, only irq latency). This new >> kind of spinlocking would be a solution to replace spinlock_bh. > > We still have no BH in Xenomai to protect against (and I don't think we > want them). So every porting case still needs careful analysis to > translate from the Linux model to RTDM. What would be possible with a > preemption-disabling spin_lock is inter-task synchronization without IRQ > disabling - kind of light-weight mutexes (/wrt their worst case, ie. > contention). I was rather thinking that when porting a linux driver to rtdm, the BH code was migrated to the irq handler. So, the spinlock would allow to have mutual exclusion between the task code and the irq handler. > >> I suspect the drivers which will cause long critical sections are the >> ones where the packets needs to be copied using PIO, instead of DMA. >> Such drivers still exist, especially in the non-x86 word. > > Right, but not all have excessive access times. And we should better > address the expensive cases with IRQ threads (which is an overdue to-do > for RTDM) so that normal mutexes can be used. You really do not want irq threads on non-x86 machines: they turn irq latencies into kernel-space threads scheduling latencies. -- Gilles. |
|
From: Jan K. <jan...@si...> - 2010-10-07 12:33:16
|
Am 07.10.2010 14:11, Gilles Chanteperdrix wrote: > Jan Kiszka wrote: >> Am 07.10.2010 13:00, Gilles Chanteperdrix wrote: >>> Jan Kiszka wrote: >>>>> Maybe that could be bundled in rtdm_lock_get_disable_irq(lock, irq) ? >>>>> Disabling an irq line should be pretty fast on non-x86, besides, if the >>>>> original code used spin_lock_bh, it is maybe because the critical >>>>> section is expected to be lengthy, so irqsave/irqrestore may cause high >>>>> latencies in the rest of the system. >>>> Generally spoken, disabling preemption can cause high latencies as well >>>> as most critical paths include a task switch. However, anything that is >>>> more light-weight than irqsave/restore and has compatible semantics >>>> would still be welcome. It just requires some relevant performance >>>> and/or latency gain to justify complications in the locking model. >>>> >>>> We actually have two or so cases in RTnet where drivers contain long >>>> locked code paths (orders of a few hundred microseconds). In those >>>> cases, disabling the IRQ line without any preemption lock is the only >>>> way to keep system latencies in reasonable limits. But those are still >>>> corner cases. >>> The interrupt latency also matters in some cases (which is why, for >>> instance, Xenomai switches contexts with irqs on ARM and powerpc now, >>> this does not improve scheduling latency, only irq latency). This new >>> kind of spinlocking would be a solution to replace spinlock_bh. >> >> We still have no BH in Xenomai to protect against (and I don't think we >> want them). So every porting case still needs careful analysis to >> translate from the Linux model to RTDM. What would be possible with a >> preemption-disabling spin_lock is inter-task synchronization without IRQ >> disabling - kind of light-weight mutexes (/wrt their worst case, ie. >> contention). > > I was rather thinking that when porting a linux driver to rtdm, the BH > code was migrated to the irq handler. So, the spinlock would allow to > have mutual exclusion between the task code and the irq handler. > >> >>> I suspect the drivers which will cause long critical sections are the >>> ones where the packets needs to be copied using PIO, instead of DMA. >>> Such drivers still exist, especially in the non-x86 word. >> >> Right, but not all have excessive access times. And we should better >> address the expensive cases with IRQ threads (which is an overdue to-do >> for RTDM) so that normal mutexes can be used. > > You really do not want irq threads on non-x86 machines: they turn irq > latencies into kernel-space threads scheduling latencies. At least some ARM-variants, I bet. Anyway, the drivers that would want to make use of this will suffer from high latencies anyway. If reading a larger Ethernet frame takes hundreds of micros, you don't want IRQ context for this job (writing frames is only half of the problem). Jan -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux |
|
From: Edward H. <qua...@op...> - 2010-10-08 19:33:52
|
Jan/Gilles, Thanks for the lively debate on this. The insights that you provided on interrupt handling are invaluable. Apparently the Nextreme Broadcom Gigabit Dual NICs were developed as a great convenience for linux user space functions as they provide significant flexibility in terms of wake on lan, nway configuration, interNIC load balancing, etc. Unfortunately, in order to provide such features the entire driver is populated with functions for managing the soft interrupts that provide for these features. More significant, is that these paired NICs share buffers, so the softirqs are even propagated into all send and receive functions. So,microsurgery is not an easy option. Moreover, managing these soft interrupts appears to be generate too much latency for a real-time application. So, I have come to the conclusion that the Dual Nextreme NICS (built-in to this HP motherboard) are not a good candidate for transporting high-frequency UDP traffic, as features inherent in their architecture significantly degrade latency. If you feel this assessment is incorrect, please don't hesitate to chime in. RSVP. In lieu of this I have decided to build an rtnet device driver for the Intel Pro 1000 1GB 82574 pci-e based board. This will require an upgrade to the existing e1000e driver. The board is inexpensive($30.00) and the existing linux driver looks easier to convert to rtnet. In addition there is an already good example in the current rtnet experimental e1000e driver. I hope to complete the driver within the next week. I will provide an update when the testing is completed. Best Regards, Ed On Thu, 2010-10-07 at 14:33 +0200, Jan Kiszka wrote: > Am 07.10.2010 14:11, Gilles Chanteperdrix wrote: > > Jan Kiszka wrote: > >> Am 07.10.2010 13:00, Gilles Chanteperdrix wrote: > >>> Jan Kiszka wrote: > >>>>> Maybe that could be bundled in rtdm_lock_get_disable_irq(lock, irq) ? > >>>>> Disabling an irq line should be pretty fast on non-x86, besides, if the > >>>>> original code used spin_lock_bh, it is maybe because the critical > >>>>> section is expected to be lengthy, so irqsave/irqrestore may cause high > >>>>> latencies in the rest of the system. > >>>> Generally spoken, disabling preemption can cause high latencies as well > >>>> as most critical paths include a task switch. However, anything that is > >>>> more light-weight than irqsave/restore and has compatible semantics > >>>> would still be welcome. It just requires some relevant performance > >>>> and/or latency gain to justify complications in the locking model. > >>>> > >>>> We actually have two or so cases in RTnet where drivers contain long > >>>> locked code paths (orders of a few hundred microseconds). In those > >>>> cases, disabling the IRQ line without any preemption lock is the only > >>>> way to keep system latencies in reasonable limits. But those are still > >>>> corner cases. > >>> The interrupt latency also matters in some cases (which is why, for > >>> instance, Xenomai switches contexts with irqs on ARM and powerpc now, > >>> this does not improve scheduling latency, only irq latency). This new > >>> kind of spinlocking would be a solution to replace spinlock_bh. > >> > >> We still have no BH in Xenomai to protect against (and I don't think we > >> want them). So every porting case still needs careful analysis to > >> translate from the Linux model to RTDM. What would be possible with a > >> preemption-disabling spin_lock is inter-task synchronization without IRQ > >> disabling - kind of light-weight mutexes (/wrt their worst case, ie. > >> contention). > > > > I was rather thinking that when porting a linux driver to rtdm, the BH > > code was migrated to the irq handler. So, the spinlock would allow to > > have mutual exclusion between the task code and the irq handler. > > > >> > >>> I suspect the drivers which will cause long critical sections are the > >>> ones where the packets needs to be copied using PIO, instead of DMA. > >>> Such drivers still exist, especially in the non-x86 word. > >> > >> Right, but not all have excessive access times. And we should better > >> address the expensive cases with IRQ threads (which is an overdue to-do > >> for RTDM) so that normal mutexes can be used. > > > > You really do not want irq threads on non-x86 machines: they turn irq > > latencies into kernel-space threads scheduling latencies. > > At least some ARM-variants, I bet. Anyway, the drivers that would want > to make use of this will suffer from high latencies anyway. If reading a > larger Ethernet frame takes hundreds of micros, you don't want IRQ > context for this job (writing frames is only half of the problem). > > Jan > |
