Re: [Valgrind-users] compat_ioctl cmd does not match even if it shows same value

[John R. <jr...@bi...>]

> long mgr_compat_ioctl(struct file *pFile, unsigned int cmd, unsigned long arg)
> {
>      int err = -EFAULT;
>      //unsigned int openLink = kIoctlOpenEthernetLink;
>      unsigned int openLink = 3222829167;

"3222829167" is not a legal 32-bit value in source code.  You MUST use "3222829167u"
with a trailing 'u'.  Do not post or reply to this mailing list until you have changed
the source, re-compiled, re-linked, and re-run the test.

> 
>      printk("cmd 0x%x/%u, openLink 0x%x/%u\n", cmd, cmd, openLink, openLink);

//  [409204.415924/1] cmd 0xc018786f/3222829167, openLink 0xc018786f/3222829167

The documentation for ioctl(), as shown by running the command "man 2 ioctl",
says that the prototype is:

        int ioctl(int fd, unsigned long request, ...);

Notice that the second parameter has type "unsigned long", which is 64 bits.
In contrast, the definition of this function 'mgr_compat_ioctl' has a second
parameter type of "unsigned int", which is 32 bits.  Beware of this difference.
[Also note that the width of the return value differs: 'int' in the documentation,
'long' in the implementation; but this should not matter here.]

Try an experiment: change the implementation to:

    long mgr_compat_ioctl(struct file *pFile, unsigned long cmd, unsigned long arg)

where the second parameter has 'long' width instead of 'int'.  If necessary,
(for instance, in the "struct file_operations mgr_fops" table of function pointers)
then cast the type of the function pointer &mgr_compat_ioctl to hide the difference
in the type of the second parameter.  At the same time, change the printk format
conversions corresponding to 'cmd' to become "0x%lx/%lu" which has two 'l' modifiers.

The experiment might reveal that the second parameter to mgr_compat_ioctl is being passed
a 64-bit value whose upper 32 bits are not all zero.

> 
>      if (openLink == cmd) {
>          printk("match\n");
>      } else {
>          printk("no match: cmd 0x%x/%u, openLink 0x%x/%u\n", cmd, cmd, openLink, openLink);
>      }

Post the assembly-language code for the function mgr_compat_ioctl.  The safest way
is to use "(gdb) disassemble mgr_compat_ioctl" or "objdump --disaassemble",
but "gcc -S" usually is OK.  We need to see EXACTLY what instructions are generated.

--