C-states of Intel CPU's cause firewire transfer corruptions

[Nizamov S. <niz...@gm...>]

Hello,

recently I tried to connect my dc1394 compliant FW400 camera from my
older laboratory PC (dual-core P4-3200 GHz/ 2Gb RAM), running 32-bit
WinXP, to a newer much more powerful workstation running Linux (64-bit
Ubuntu 12.04). On old PC this CCD camera was connected using VT6307
card. I used this setup for more than the year and it showed no
corruptions at all, despite the stream was topping the FW-400 speed -
12(16)-bit grayscale 1 MP @ 16 fps, 4092 bit packet size. I save data
stream for hours, later process and visualize them and pretty much
sure that there were no corruptions.

However, after connecting the same camera using the same VT6307 card
to the quad-core p5@2.8GHz, 8 Gb RAM (Linux), I noticed that there are
corruptions in the acquired frames. Corruptions are in form of lines,
which are not overwritten by the newer frame data and contain the
older buffer data (or zeros, if buffer was clean). The length of this
line is nearly of the buffer size of VT6307 (circa 2 kbytes or 1k
pixels). I wrote a simple test program using dc1394 library, which
would always clean out the buffer after acquiring the frame. Because
my camera has an DAC offset it never outputs the pixel values less
than circa 190. So it was easy to distinguish real pixel data (in the
range of 190-4096) from corruptions. With the same VT6307 card and CCD
camera on my newer Linux workstation, I noticed that about 1 byte in
15000 bytes would be lost (actually in my case that means that
approximately 1 line of pixels would be zeroed out in every 5-10
frames).

I guess that this is very small error rate and in most applications it
is practically unnoticeable (e.g. in case of video, if one frame
contains a very small random portion of older frame and these frames
do not differ much). But in my use-case it is not acceptable and I
tried to fix the issue. I tried VT6306 and VT6307 cards. I changed PCI
ports, made sure that there is no other PCI devices on the bus,
adjusted all possible PCI bus and device settings including exotic
ones (memory/address stepping), played with giving real-time priority
to my test program, higher interrupt priority to the firewire module,
tried to force VT6307 to run in OHCI1.1 mode, recompile dc1394
libraries, coriander etc. I tried a fresh Linux kernel 3.5. I got the
smaller rate of corruptions but never could get rid of them
completely. I could not find the LSI chips with 4kb buffer therefore I
bought FW card with 2 Kb buffer, but with much more robust TI chip
(43AB23). Things get actually worse - if with VT6306/VT6307 I would
get zeros instead of actual pixel data, now alongside such pixels
there would be also a few pixels with values higher than 4096 - also
corruptions, but of another kind.

At the same time I was evaluating a newly bought USB2 camera (not
dcam/iidc compatible, with proprietary library). It worked flawlessly
on my older laboratory PC and with my notebook (the same 64-bit ubuntu
12.04 as my workstation). But it mysteriously failed with my
workstation too! All the controls would work perfectly but I could get
data stream out of it only at USB1 speeds - something like 1 frame per
second. Looking through the vendors site I found an application note
which blamed that newer Intel CPU's introduced so known C-states for
power consumption reduction. They claimed that even if a single core
of CPU gets into such state, not only its frequency, but the bandwidth
of buses will be reduced too! Being a little bit skeptic, I
nevertheless disabled these C-states. It turns out that disabling them
in BIOS is not enough, linux kernel loads module intel_idle which
re-enables these C-states. The solution is loading the kernel with
"intel_idle.max_cstate=0" option. Check in dmesg for the "intel_idle:
disabled" line to see if it worked.

And it really solved both my problems - both USB2 and FW400 cameras
work on my workstation with practically saturation level transfer
speeds (more than 30MB/s) and without any corruptions!

Hopefully it will help someone to fix their problem with firewire corruptions.

With best regards,
S.Nizamov