Re: receiving an evt_timeout error code with Ricoh R5C832 PCIe controller

[Jarvis S. <sch...@gm...>]

On Thu, Aug 2, 2012 at 5:12 PM, Stefan Richter
<st...@s5...> wrote:
>
> On Aug 02 Jarvis Schultz wrote:
> > On Thu, Aug 2, 2012 at 1:54 AM, Stefan Richter
> > <st...@s5...>
> > wrote:
> > > Your kernel "3.2.0-27" is possibly based on 3.2.6 or later.  What does
> > > "grep firewire /proc/interrupts" show?
> > >
> > Here is the output of that:
> >  19:       3043          0          0          0          0          0
> >      0         21   IO-APIC-fasteoi   firewire_ohci
>
> I.e. your kernel has got the "don't use MSI on Ricoh OHCI-1394" flag
> already in its built in table.
>
> [...]
> > > Vice versa, if the quirk is built in, it can be switched off
> > > by a different value to the module parameter:
> [...]
> > I tried both of these "quirk" levels, and there is no change to the
> > behavior.
>
> Interesting; this means we could narrow the built-in quirk down to the
> chip version which was originally reported as broken with MSI, and let
> chip versions like yours use MSI again.  But this doesn't help you one
> bit, alas.
>
> [...]
> > >> System Details:
> > >> Kernel version - 3.2.0-27
> [...]
> > grep command:
> >     # CONFIG_PREEMPT_RCU is not set
> >     CONFIG_PREEMPT_NOTIFIERS=y
> >     # CONFIG_PREEMPT_NONE is not set
> >     CONFIG_PREEMPT_VOLUNTARY=y
> >     # CONFIG_PREEMPT is not set
>
> Have a look whether your distribution also offers a kernel package which
> is called "lowlatency" or "desktop" or something along these lines --- and
> in effect has got CONFIG_PREEMPT=y and "# CONFIG_PREEMPT_VOLUNTARY is not
> set".  The CONFIG_PREEMPT option is found under "Processor type and
> features -> Preemption Model -> Preemptible Kernel (Low-Latency Desktop)"
> in the kernel configuration menu when kernel sources are configured as
> preparation of a kernel build from source.
>
> CONFIG_PREEMPT means that kernel threads are put to sleep by the CPU
> scheduler when it used up its time slice and another process is waiting to
> get CPU time, very much the same way as the CPU scheduler treats userspace
> processes when they used up their time slice.
>
> I am suggesting to try such a kernel because there is a slight chance that
> reduced scheduling latency helps the Phantom Omni drivers to queue
> FireWire packets more steadily, which in turn might reduce the likelihood
> that Ricoh controller stops its DMA unrecoverably.
>
I just ran installed the most up-to-date lowlatency kernel that Ubuntu
12.04 has which was built around 3.2.0-23.  To address the next
comment, I booted into a console-only-mode, and made sure the fglrx
drivers weren't loaded.  I saw the same behavior as before.  Also, I
tried both kernels with the open source radeon drivers in a full
desktop session, and repeatedly received the same errors.
> > I attached my /proc/interrupts file as interrupts_log.txt
>
> Looks inconspicuous to me.  Just one thing:  Have you tried the mainline
> "radeon" driver instead of the closed source fglrx driver already?  3rd
> party kernel drivers tend to be latency sources due to lack of independent
> review and tie-in into mainline development.
>
> [...]
> > > Could you send the chip IDs as "lspci -nn | grep 1394" shows them?
> > >
> > Here it is:
> > 09:00.3 FireWire (IEEE 1394) [0c00]: Ricoh Co Ltd R5C832 PCIe IEEE 1394
> > Controller [1180:e832] (rev 01)
>
> Thanks for this.  Same PCI ID as in the report which led to kernel commit
> 320cfa6ce0b3, but rev 01 instead of rev 00.
>
> > >>         Phantom Omni PDD drivers version = Linux JUJU PDD pre-beta
> > >>
> > >> libraw1394 version - 2.0.5
> > >
> > > Are the Phantom Omni drivers kernel drivers or userspace drivers?  If
> > > the
> > > latter, are they using libraw1394?  You could for example check with
> > > "ldd /path/to/the/driver/binary.xy" which lists all libraries to which
> > > 'binary.xy' is dynamically linked to.  Libraries which are statically
> > > linked into a binary ( = incorporated into the binary) won't show up
> > > this
> > > way though.
> > >
> > The drivers are userspace drivers, and they are distributed as just a
> > sequence of shared libraries.  The old drivers that they released
> > depended
> > on the raw1394 kernel module.  These new "pre-beta JUJU" drivers do not.
> > I
> > originally tried to get this working with their old drivers, and
> > followed
> > the instructions at this site:
> > https://wiki.sofa-framework.org/tdev/wiki/HowTo/SensableWithoutRaw1394
> > to
> > create a "dummy" kernel module.
>
> Argh. :-)  Nitpick: "ln /dev/null /dev/raw1394" probably would have worked
> just as well as "ln /dev/fw0 /dev/raw1394".  /dev/fw* does not support
> any of the ioctl/write/read operations that are valid on /dev/raw1394.
> Anyway, the former necessity of this dummy module and dummy file is a fine
> showcase of the joys and wonders of non-free software.
>
> > I am not positive I was getting the same
> > error messages, but I was getting the same "freezing" behavior.
> >
> > > The term "JUJU" implies that the drivers are not using libraw1394.
> > > But if
> > > they do, you should try to update to libraw1394 2.0.7 or later because
> > > it
> > > has got changes which affect isochronous streaming.  Current and
> > > stable is
> > > 2.1.0.  Of course an update of the system-wide installed libraw1394
> > > would
> > > only have effect if the Phantom Omni drivers are dynamically linked to
> > > it.
> > >
> > I am a little confused by their terminology.  Their binaries are
> > absolutely
> > linked to libraw1394 libraries.  I believe they are using the term JUJU
> > because they no longer use the kernel module raw1394.
>
> They never directly used the kernel module, they used libraw1394.
> (Which is good because the developers of raw1394 intended the raw1394
> kernel interface to be private to the raw1394<-->libraw1394 pair.  This
> in turn facilitated the transition to the binary incompatible
> <linux/firewire-cdev.h> kernel interface provided by firewire-core
> while retaining compatibility with libraw1394 based application
> software.  libraw1394 v2 transparently uses raw1394's or firewire-core's
> mutually incompatible interfaces depending on what libraw1394 detects to
> be
> available at runtime.)
>
> So this "JUJU" designation either refers to the accomplishment of removing
> that silly check for presence of raw1394, or it means that they added a
> new
> <linux/firewire-cdev.h> backend but are still keeping a libraw1394 based
> backend as an alternative or for parts of their functionality.
>
> Or it means that they noticed that libraw1394 does not behave 100% the
> same when running on top of firewire-core versus raw1394, and added some
> workarounds in their own code (instead of submitting fixes to upstream
> libraw1394 or kernel, or at least reporting such issues).
>
> I guess one could find out more about it by running it all in a debugger
> or by inserting some simple debug logging into libraw1394.  But I don't
> think at this point that it would get us nearer to a fix for the Ricoh
> problem (or what I believe to be a Ricoh problem).
>
> > Not really sure if
> > this is correct.  I just downloaded and compiled libraw1394 2.1.0, and
> > made
> > sure that the drivers were finding the new versions that I installed
> > (using
> > ldd and ldconfig).  I still get the same error message.
>
> OK.  Either the libraw1394 2.0.7 change which I alluded to simply does not
> have a positive effect here ---- which is not surprising since that change
> was implemented to address a different problem than yours ---, or the
> drivers bypass libraw1394 when running on top of firewire-core, or both.
>
> > > http://www.sensable.com/support-download-pdd.htm looks like the
> > > Phantom
> > > Omni drivers are not free, not open source, and not even made
> > > available
> > > to anybody but customers of theirs.  In which case it would be hard to
> > > find out what kinds of I/O these drivers need and which userspace/
> > > kernelspace code paths they go through, obviously.
> >
> > You are absolutely correct.  I have to provide them with my device
> > serial
> > number, just to access the driver download, and even then, I only get
> > binaries and libraries, not source code.  I guess it's not quite
> > "OpenHaptics"...
>
> Well, at least they do provide something for Linux users among their
> customers, although in a not particularly pragmatic form.
That is absolutely right, the Linux support is definitely appreciated!

As a final update, I just did some testing with our lab's server
machine.  The machine has the same version for the kernel, the
drivers, the shared libraries the drivers depend on, and the
application I have been using to test (i.e. quite similar software
builds).  The firewire controller on that machine is

06:0c.0 FireWire (IEEE 1394) [0c00]: Texas Instruments TSB43AB22A
IEEE-1394a-2000 Controller (PHY/Link) [iOHCI-Lynx] [104c:8023]

Every test application that sensable/ OpenHaptics provides worked
without any errors.

> --
> Stefan Richter
> -=====-===-- =--- ---==
> http://arcgraph.de/sr/