#328 Wacom Bamboo one S, kernel freeze on linux49

[Antonín Dach]

I don't want to reopen issue #311 https://sourceforge.net/p/linuxwacom/bugs/311/, so I am creating a new one

What tablet: Wacom One By Wacom CTL-471-EU,

As from kernel 4.9.0 to 4.9.2 I am havin issue with the tablet.

After pluging it to usb2 it works for a while (circa 25 seconds), however whenever I use a graphical drawing program Krita or Pinta it causes it to freeze entire system much faster.

kernel: BUG: unable to handle kernel NULL pointer dereference at 0000000000000001

I will keep investigating, and after few days I will post a recording of the device input.

[Tom R]

I had a similer problem with my new Wacom Cintiq Pro 13 DTH-1320-EU recently in Ubuntu 16.04 with kernel 4.4.0. Same problem, was working for about 15sec before os freeze.

I found the bug below when connecting a second monitor and thought it was the root cause since mine it is a pen display. I updated to kernel 4.8.17 and installed input-wacom drives again and it has been working for a week so far.
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1559308

[Antonín Dach]

I have done the recording, judging by the nature of the last journal message, I think it's some devilish Xorg incompatibility. To reproduce full system crash, open a Krita program :) and run the evemu recordings over the krita (version 3.1.1
) program.

In my opinion it's unrelated to @t0m's issue. As this bug is fresh.

System:    Host: normandy Kernel: 4.9.2-1-MANJARO x86_64 (64 bit gcc: 6.2.1) Desktop: i3 4.13
           Distro: Manjaro Linux
Machine:   Device: laptop System: LENOVO product: 20B20015MC v: ThinkPad Edge E545
           Mobo: LENOVO model: 20B20015MC UEFI [Legacy]: LENOVO v: HRET24WW (1.12) date: 01/24/2014
Battery    BAT0: charge: 39.7 Wh 100.0% condition: 39.7/38.9 Wh (102%) model: SANYO 45N1043 status: Full
CPU:       Dual core AMD A8-4500M APU with Radeon HD Graphics (-HT-MCP-) cache: 4096 KB
           flags: (lm nx sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 svm) bmips: 7588
           clock speeds: max: 1900 MHz 1: 1400 MHz 2: 1400 MHz 3: 1600 MHz 4: 1600 MHz
Graphics:  Card: Advanced Micro Devices [AMD/ATI] Trinity [Radeon HD 7640G] bus-ID: 00:01.0
           Display Server: X.Org 1.19.1 driver: radeon Resolution: 1366x768@59.99hz
           GLX Renderer: Gallium 0.4 on AMD ARUBA (DRM 2.48.0 / 4.9.2-1-MANJARO, LLVM 3.9.1)
           GLX Version: 3.0 Mesa 13.0.3 Direct Rendering: Yes
Audio:     Card-1 Advanced Micro Devices [AMD] FCH Azalia Controller driver: snd_hda_intel bus-ID: 00:14.2
           Card-2 Advanced Micro Devices [AMD/ATI] Trinity HDMI Audio Controller
           driver: snd_hda_intel bus-ID: 00:01.1
           Sound: Advanced Linux Sound Architecture v: k4.9.2-1-MANJARO
Network:   Card-1: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
           driver: r8169 v: 2.3LK-NAPI port: 1000 bus-ID: 01:00.0
           IF: enp1s0 state: down mac: f8:a9:63:53:62:b2
           Card-2: Broadcom Limited BCM43142 802.11b/g/n driver: wl bus-ID: 02:00.0
           IF: wlp2s0 state: up mac: 48:5a:b6:e3:11:e7
Drives:    HDD Total Size: 500.1GB (37.0% used)
           ID-1: /dev/sda model: WDC_WD5000LPVX size: 500.1GB
Partition: ID-1: / size: 452G used: 166G (39%) fs: ext4 dev: /dev/dm-0
           ID-2: /boot size: 247M used: 117M (50%) fs: ext2 dev: /dev/sda1
           ID-3: swap-1 size: 7.48GB used: 0.00GB (0%) fs: swap dev: /dev/sda5
Sensors:   None detected - is lm-sensors installed and configured?
Info:      Processes: 137 Uptime: 9 min Memory: 661.2/7129.5MB Init: systemd Gcc sys: 6.3.1

Only wacom package installed
Name : libwacom
Version : 0.22-1
Description : Library to identify Wacom tablets and their features

[Bleak Grey]

I'm experiencing the same issue on newly installed Ubuntu 16.10 with my Wacom One S. It works as expected for some time but then whole system freezes and stops responding.
I tried reinstalling libwacom and switching multiple kernels including 4.9 and 4.10rc7, but the freeze still occurs.

Sadly, this issue renders my Wacom unusable on Linux. Is there any possible solution yet?

[Antonín Dach]

I am guesing, but you could try uninstalling all wacom packages and tehn try to install a older kernel like 4.6.
If it would work, we would know that it's not tied with other userspace library or something.
Then the absolute time wasting procedure of a bisecting a kernel would do the trick, it would tell us exactly the commit that broke the driver. But like I said, compiling the kernels 15 times takes ages.

Another aproach would be compiling the module itself and they bump the versions per kernel release and I have no idea if inserting module would work on a newer kernel release but why not? :O

I will try something after the finals ends, I don't have that much of a time.

I am hoping the maintainers will be able to reproduce the crash with my udev recordings, maybe they know how to debug these thing.

[Jason Gerecke]

Unfortunately, I've had no luck reproducing this on my system (see below) with the larger CTL-671. I've been able to paint in Krita for several minutes without any kind of instability. Looking at the logs you've posted, it looks like the page table(?) for the Xorg process is getting corrupted somehow. I'm not sure if this kind of problem would be caused by a buggy X program, buggy X driver, or buggy kernel driver -- I'll have to do more reading.

Could you try downgrading your kernel to confirm that it fixes your issue? I believe Manjaro will allow you to install AUR softwarelike "downgrade" which you can use to revert the kernel to a pre-4.9 version. Also, can you confirm whether the xf86-input-wacom or xf86-input-libinput packages are installed, which driver is in use for your tablet (run journalctl -b0 | grep "Using input driver" | grep Bamboo), and what version of krita and xorg-server you're using?

System:

• Arch Linux
• linux 4.9.6-1
• xf86-input-wacom 0.34.0-1
• libinput 1.6.0-1
• xf86-input-libinput 0.23.0-1
• krita 3.1.2.1-1
• xorg-server 1.19.1-1

[Antonín Dach]

Hi Jason,

Weird you can't reproduce it on your system, I for once am not having xf86-input-wacom package installed and just using the native wacom.ko inside the shipped Manjaro kernel.

I can confirm that on linux 4.8.17 the tablet no longer crashes the system, that's why I opened this ticket as I believe this is very fresh bug inside the input-wacom..

Kernel 4.9.6 and 4.10rc5 crashes entire system.

(I own a Desktop PC as well, it crashes aswell with the kernel 4.9 and up)

[Jason Gerecke]

Thanks for that confirmation, Antonin. There were quite a few changes made between 4.8 and 4.9, so it might be a little difficult to pin down exactly what the trigger is. How comfortable are you with the idea of compiling/installing the input-wacom driver? If you are okay with it, I'd like to walk you through the process of doing a "git bisect" to determine exactly which change introduced the break, and hopefully better understand what is going wrong.

The process can be a bit time intensive (since you'll probably have to compile/install/test the kernel five or so times to isolate the change); it might take an hour or so.

[Antonín Dach]

I am doing it now, but man, it ain't easy, all the bisects are comming good, and I keep skipping the right half that won't even compile.. So It taking forever, Bisects good, 5x skip then it came as good again and it keep saying 6 steps remaining... Jeezus I just keep getting errors about devm_add_action_ro_reset redefinition..

I might be stuck.. (So weird that the last commit compiles but for debugging purposes it's hell :C )

I will keep trying.

EDIT: motHer of all the devillish thing, I HAVE TO KEEP TRACK OF USING ./configure and ./autogen.sh GOD damn, why would someone leave the damn thing in there if it tries to compile the code and fail.. "game an impression of untestable commit"

[Jason Gerecke]

Sounds like quite the battle you're engaged in! :eek:

The devm changes were pretty intrusive, but I'm a little surprised that you're running into compile errors... We try to keep the tree buildable at all points since bisecting a broken build is nobody's definition of fun. I wonder if there's a way to have Travis compile intermediate commits to help ensure it doesn't accidentaly happen. Keep me updated, and best of luck...

[Antonín Dach]

Yeah it's harder than I thought.

The kernel panics started to be more random, I do plenty of misteps, once the tablet worked for 10minutes and then failed when repluging from usb2 to usb3.

Depressing that it could have been issue that was fixed and I incoreclty assumed that it was my issue.

I will keep trying.

EDIT: I am nearly at the end, few more commits

[Antonín Dach]

OK I have completed my task..

I am including my entire bisect log

I have come to the mergin phase where I cannot compile the code.

There are only 'skip'ped commits left to test.
The first bad commit could be any of:
165ef5752e1d4bf5c228bf6d493b843b422a1c4b
1670ce97476075cb437822098d11a9dcb6fd58cf
2e1c4398d1d7bca88cffe79165f3a8d1398496fe
89799354bed8694055d81d7b619cf2cf5cf8a3c6
We cannot bisect more!

The last good commit found in log "d42c7284afc9ccf8036bdb285719ed7619bc6018" was well tested and did now fail after longer period of time and multiple repluggin in ports usb2 -> usb3 without a panic.

I am not 100% sure but I am running without crash on that commit so far.. :)

Good luck finding the bug. the output when compilation fail

/input-wacom/3.17/wacom_sys.c:980:19: error: redefinition of ‘devm_add_action_or_reset’
 static inline int devm_add_action_or_reset(struct device *dev,
                   ^~~~~~~~~~~~~~~~~~~~~~~~
In file included from ./include/linux/input.h:22:0,
                 from ./include/linux/hid.h:35,
                 from /home/frdm/src/input-wacom/3.17/wacom_wac.h:13,
                 from /home/frdm/src/input-wacom/3.17/wacom_sys.c:14:
./include/linux/device.h:693:19: note: previous definition of ‘devm_add_action_or_reset’ was here
 static inline int devm_add_action_or_reset(struct device *dev,
                   ^~~~~~~~~~~~~~~~~~~~~~~~
/home/frdm/src/input-wacom/3.17/wacom_sys.c: In function ‘wacom_show_remote_mode’:
/home/frdm/src/input-wacom/3.17/wacom_sys.c:1292:11: warning: comparison is always true due to limited range of data type [-Wtype-limits]
  if (mode >= 0 && mode < 3)
           ^~

but further version compiles fine.

[Jason Gerecke]

Thanks for the hard work! The bisect points to one of the merge commits as being a probable cause, but this doesn't make sense since those don't introduce any code changes. Its likely that the apparently intermittent nature of the bug caused you to make some bad commits as 'good' since they appeared to work fine. Determining which is going to be extra-tricky though.

As to the source of the devm_add_action_or_reset errors that you were getting, those appear to be a result of poor branch merging on our (read: "my") part. The backports of the "jiri/for-4.9" branch were integrated in to master before the "jiri/for-4.9" branch itself. Normally this wouldn't matter, except that the branch introduces a new "4.5" directory to ensure new kernels don't try to compile incompatible code. This means that the build can fail for some of the commits prior to the branch merge :( Mea culpla.

To work around the devm_add_action_or_reset issue and focus on the 4.9 branch where this bug was probably introduced, you might try running something like git bisect reset && git bisect start 27809b2 ea3af31. This should run into fewer problems since it will avoid the broken backports, though you're probably still going to see cases where the tablet appears to work at first only for it to fail after multiple minutes or a disconnect/connect.

[Antonín Dach]

Hi Jason,

You were absolutelly right I have missed a bad commit.
I have succesfully finished bisecting and found the trouble maker!

37703c4b3158fde034da44fa38d13b1bff1b4acd is the first bad commit
commit 37703c4b3158fde034da44fa38d13b1bff1b4acd
Author: Jason Gerecke <killertofu@gmail.com>
Date:   Mon Aug 8 12:06:30 2016 -0700

    HID: wacom: Augment 'oVid' and 'oPid' with heuristics for HID_GENERIC

    The 'oVid' and 'oPid' variables used by wacom_are_sibling are a hacky
    solution to the problem of the driver historically having few good
    heuristics to use in determining if two devices should be considered
    siblings or not. While it works well enough for explicitly supported
    devices, it offers no help for HID_GENERIC devices. Now that we have
    a bit more information (e.g. direct/indirect) available to us though,
    we should make use of it it to improve the pairing of such devices.

    Signed-off-by: Jason Gerecke <jason.gerecke@wacom.com>
    Reviewed-by: Benjamin Tissoires <benjamin.tissoires@redhat.com>
    Signed-off-by: Jiri Kosina <jkosina@suse.cz>
    [aaron.skomra@wacom.com: Imported into input-wacom repository (41372d5)
    Signed-off-by: Aaron Armstrong Skomra <aaron.skomra@wacom.com>

:040000 040000 bffa3f7ac59cad64674f248606bd331ec9517669 df8c532ad4c396f38cd

I am alsow adding a bisect log :)

Is this better?

[Sean Engelhardt]

Hi. Ex experience the same issue with a Wacom Bamboo Pen.
I am on
ArchLinux,
Kernel: 4.9.11-1-ARCH
libwacom 0.24-1
xf86-input-wacom 0.34.0-1

My wacom tablet is now basically unusable. Is there still no fix?

[Antonín Dach]

Hi, I believe they are working on a fix :) in the mean time, you can use the older driver by compiling the source code at the specific last known commit. For example this one 1f49cce2565845f38eca503cf9e5e7f399b9089e was the last known to work for my tablet.

You could have the same issue as me.

If you want to work, try uninstalling xf86-input-wacom,, clone the repository of input-wacom and compile it at the mentioned commit.

THe module will insert itself after reboot so you can at least draw.

[Jason Gerecke]

Still working on this issue yes. I don't see anything suspicious in the commit, but did recently hear about another memory corruption that maybe related.

[Jason Gerecke]

Antonin (and others) -- would it be possible for you to try installing the xf86-input-wacom 0.34.2 release that was made a few days ago? It fixes an issue with our driver making changes to internal bits of the X server at the wrong time. This bug has caused errors that appear to be completely unrelated to the actual issue, and I wonder if it might also be responsible for this. After installing the update, remove any input-wacom driver present by running sudo make uninstall from within the "input-wacom" source directory and then reboot.

Instructions on installing the xf86-input-wacom driver from source can be found at http://linuxwacom.sourceforge.net/wiki/index.php/Xf86-input-wacom

[Antonín Dach]

I have troubles running autogeneration, the condition were not met,

configure: error: Package requirements (xorg-server >= 1.7.0 xproto xext kbproto inputproto randrproto) were not met:

No package 'xorg-server' found

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

But I have xorg-server installed, any ideas on arch based distros?

I have xorg-server package at version 1.19 :C what should I do?

input-wacom uninstalled and all dependencies installed

[Inna Rozen]

I experience the same problem.

Kernel: 4.9.0-2-amd64 x86_64 (64 bit)
Display Server: X.Org 1.19.2

dpkg -s xserver-xorg-input-wacom | grep '^Version:'
Version: 0.34.0-1

My Xorg.0.log is full of messages like

[  1171.596] (EE) Wacom Bamboo One S Pen eraser: Error reading wacom device : No such device

and

[  1061.819] (EE) ******** WriteToClient called from input thread *********
[  1061.819] (EE) 
[  1061.819] (EE) Backtrace:
[  1061.819] (EE) 0: /usr/lib/xorg/Xorg (xorg_backtrace+0x4a) [0x55b21d38d46a]
////(here goes a long backtrace)
[  1061.819] (EE) 
[  1061.819] (EE) BUG: triggered 'if (in_input_thread())'
[  1061.819] (EE) BUG: ../../../../os/io.c:656 in WriteToClient()
[  1061.819] (EE) ******** WriteToClient called from input thread *********

Full Xorg.0.log.old attached

[Jason Gerecke]

Antonin, the error message you are seeing is a result of not having the xorg-server development headers installed. Assuming Manjaro uses the same package names as Arch, try installing the "xorg-server-devel" package. Unfortunately I don't have a complete list of the required development packages for Manjaro/Arch -- let me know if you run into any more "No package" errors.

Inna, the "WriteToClient" errors that you're seeing in your Xorg.0.log should be fixed by updating to xf86-input-wacom-0.34.2. I'm hoping it also fixes the the freeze issue that this bug is about. Please try installing the driver from source as outlined in http://linuxwacom.sourceforge.net/wiki/index.php/Xf86-input-wacom

[Antonín Dach]

Hi, I feel silly forgeting about the devel package, what the heck was I thinking of..

However I must say the the xf86-input-wacom-0.34.2 is better but It still crashes after while :(.

I am possitive I am running it, since

~$ cat /sys/module/wacom/version
v2.00

and the tablet is able to go for entire half an hour without a kernel panic.

[Inna Rozen]

I compiled driver from source and it removed "WriteToClient" to client errors, but the freezes are still present, though slightly different. Now it takes more time to get the freezes and there is a chance of unfreezing if the tablet is unplugged when freezes started. The Xorg.0.log is still full of such messages:

[   724.299] (EE) Wacom Bamboo One S Pen eraser: Error reading wacom device : No such device

[Jason Gerecke]

EDIT: Actually Antonin, I just updated one of my Arch linux boxes and it appears that 0.34.2 is now in the repository. You might check to see if its available on Manjaro as well. Should save the trouble of compiling :) Fingers crossed it fixes this.

[Antonín Dach]

I just build the master and crashed after 35 minutes :( WAY Better but still.

[Antonín Dach]

Behaviour is now somewhat different, I am able to use the computer for about 30s (also happened before but now it's the same all the attempts) after the cursor stops moving via tablet, after that it fails compeltely. (memory leak?).
I could do some core dumping, or maybe puting a printk somewhere ? If you gave me instruction :)

One other thought, maybe it's the xorg that crashes this time and go to full kernel panic few seconds later?

I am probably unhelpful, so I'm willing to test further or apply anypatch you throw at me.