valgrind-users

[Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Troels W. H. <tr...@th...>]

Valgrind 1.9.6 runs much better on RH8+glibc 2.3.2/RH9, but it doesn't
appear to be perfect yet. I have reproduced this problem on both of the
mentioned platforms.

This is the expected behavior of the program.

$ python dbboguserrno.py
Traceback (most recent call last):
  File "dbboguserrno.py", line 4, in ?
    filetype = db.DB_BTREE)
  File "/usr/lib/python2.2/site-packages/rpmdb/dbshelve.py", line 69, in
open
    d.open(filename, dbname, filetype, flags, mode)
rpmdb._rpmdb.DBNoSuchFileError: (2, 'No such file or directory')

This is what happens in Valgrind. RH8 and RH9 uses BerkeleyDB 4.0.14,
with a custom compiled Python interpreter and BerkeleyDB 3.3.11 I get
"bsddb3._db.DBError: (9, 'Bad file descriptor -- nosuchfile.db: Bad file
descriptor')" errors instead.

$ valgrind -v python dbboguserrno.py
==7099== Memcheck, a.k.a. Valgrind, a memory error detector for
x86-linux.
==7099== Copyright (C) 2002, and GNU GPL'd, by Julian Seward.
==7099== Using valgrind-1.9.6, a program instrumentation system for
x86-linux.
==7099== Copyright (C) 2000-2002, and GNU GPL'd, by Julian Seward.
==7099== Startup, with flags:
==7099==    --suppressions=/usr/local/lib/valgrind/default.supp
==7099==    -v
==7099== Reading suppressions file: /usr/local/lib/valgrind/default.supp
==7099== Estimated CPU clock rate is 702 MHz
==7099==
==7099== Reading syms from /usr/bin/python
==7099==    object doesn't have any debug info
==7099== Reading syms from /lib/ld-2.3.2.so
==7099==    object doesn't have any debug info
==7099== Reading syms from /usr/local/lib/valgrind/vgskin_memcheck.so
==7099== Reading syms from /usr/local/lib/valgrind/valgrind.so
==7099== Reading syms from /lib/libdl-2.3.2.so
==7099==    object doesn't have any debug info
==7099== Reading syms from /usr/local/lib/valgrind/libpthread.so
==7099== Reading syms from /lib/libutil-2.3.2.so
==7099==    object doesn't have any debug info
==7099== Reading syms from /lib/libm-2.3.2.so
==7099==    object doesn't have a symbol table
==7099==    object doesn't have any debug info
==7099== Reading syms from /lib/libc-2.3.2.so
==7099==    object doesn't have a symbol table
==7099==    object doesn't have any debug info
==7099== Reading syms from
/usr/lib/python2.2/lib-dynload/structmodule.so
==7099== Conditional jump or move depends on uninitialised value(s)
==7099==    at 0x40008D55: elf_dynamic_do_rel.7 (in /lib/ld-2.3.2.so)
==7099==    by 0x4000942A: _dl_relocate_object_internal (in
/lib/ld-2.3.2.so)
==7099==    by 0x40377D90: (within /lib/libc-2.3.2.so)
==7099==    by 0x4000B115: _dl_catch_error_internal (in
/lib/ld-2.3.2.so)
==7099==
==7099== Conditional jump or move depends on uninitialised value(s)
==7099==    at 0x40008D59: elf_dynamic_do_rel.7 (in /lib/ld-2.3.2.so)
==7099==    by 0x4000942A: _dl_relocate_object_internal (in
/lib/ld-2.3.2.so)
==7099==    by 0x40377D90: (within /lib/libc-2.3.2.so)
==7099==    by 0x4000B115: _dl_catch_error_internal (in
/lib/ld-2.3.2.so)
==7099==
==7099== Conditional jump or move depends on uninitialised value(s)
==7099==    at 0x40009517: _dl_relocate_object_internal (in
/lib/ld-2.3.2.so)
==7099==    by 0x40377D90: (within /lib/libc-2.3.2.so)
==7099==    by 0x4000B115: _dl_catch_error_internal (in
/lib/ld-2.3.2.so)
==7099==    by 0x403774AE: _dl_open (in /lib/libc-2.3.2.so)
==7099==
==7099== Conditional jump or move depends on uninitialised value(s)
==7099==    at 0x40009565: _dl_relocate_object_internal (in
/lib/ld-2.3.2.so)
==7099==    by 0x40377D90: (within /lib/libc-2.3.2.so)
==7099==    by 0x4000B115: _dl_catch_error_internal (in
/lib/ld-2.3.2.so)
==7099==    by 0x403774AE: _dl_open (in /lib/libc-2.3.2.so)
==7099== Reading syms from
/usr/lib/python2.2/lib-dynload/_codecsmodule.so
==7099== Reading syms from
/usr/lib/python2.2/site-packages/rpmdb/_rpmdb.so
==7099== Reading syms from /usr/lib/librpm-4.1.so
==7099== Reading syms from /usr/lib/librpmdb-4.1.so
==7099== Reading syms from /usr/lib/librpmio-4.1.so
==7099== Reading syms from /usr/lib/libpopt.so.0.0.0
==7099== Reading syms from /lib/librt-2.3.2.so
==7099==    object doesn't have any debug info
==7099== Reading syms from /usr/lib/libelf.so.0.8.2
==7099== Reading syms from /usr/lib/libbz2.so.1.0.2
==7099== Reading syms from /usr/lib/python2.2/lib-dynload/cPickle.so
==7099== Reading syms from /usr/lib/python2.2/lib-dynload/cStringIO.so
Traceback (most recent call last):
  File "dbboguserrno.py", line 4, in ?
    filetype = db.DB_BTREE)
  File "/usr/lib/python2.2/site-packages/rpmdb/dbshelve.py", line 69, in
open
    d.open(filename, dbname, filetype, flags, mode)
rpmdb._rpmdb.DBAgainError: (11, 'Resource temporarily unavailable --
nosuchfile.db: Resource temporarily unavailable')
==7099==
==7099== ERROR SUMMARY: 48 errors from 4 contexts (suppressed: 2 from 1)
==7099==
==7099== 12 errors in context 1 of 4:
==7099== Conditional jump or move depends on uninitialised value(s)
==7099==    at 0x40009565: _dl_relocate_object_internal (in
/lib/ld-2.3.2.so)
==7099==    by 0x40377D90: (within /lib/libc-2.3.2.so)
==7099==    by 0x4000B115: _dl_catch_error_internal (in
/lib/ld-2.3.2.so)
==7099==    by 0x403774AE: _dl_open (in /lib/libc-2.3.2.so)
==7099==
==7099== 12 errors in context 2 of 4:
==7099== Conditional jump or move depends on uninitialised value(s)
==7099==    at 0x40009517: _dl_relocate_object_internal (in
/lib/ld-2.3.2.so)
==7099==    by 0x40377D90: (within /lib/libc-2.3.2.so)
==7099==    by 0x4000B115: _dl_catch_error_internal (in
/lib/ld-2.3.2.so)
==7099==    by 0x403774AE: _dl_open (in /lib/libc-2.3.2.so)
==7099==
==7099== 12 errors in context 3 of 4:
==7099== Conditional jump or move depends on uninitialised value(s)
==7099==    at 0x40008D59: elf_dynamic_do_rel.7 (in /lib/ld-2.3.2.so)
==7099==    by 0x4000942A: _dl_relocate_object_internal (in
/lib/ld-2.3.2.so)
==7099==    by 0x40377D90: (within /lib/libc-2.3.2.so)
==7099==    by 0x4000B115: _dl_catch_error_internal (in
/lib/ld-2.3.2.so)
==7099==
==7099== 12 errors in context 4 of 4:
==7099== Conditional jump or move depends on uninitialised value(s)
==7099==    at 0x40008D55: elf_dynamic_do_rel.7 (in /lib/ld-2.3.2.so)
==7099==    by 0x4000942A: _dl_relocate_object_internal (in
/lib/ld-2.3.2.so)
==7099==    by 0x40377D90: (within /lib/libc-2.3.2.so)
==7099==    by 0x4000B115: _dl_catch_error_internal (in
/lib/ld-2.3.2.so)
--7099--
--7099-- supp:    2 __pthread_mutex_unlock/_IO_funlockfile
==7099==
==7099== IN SUMMARY: 48 errors from 4 contexts (suppressed: 2 from 1)
==7099==
==7099== malloc/free: in use at exit: 120602 bytes in 1147 blocks.
==7099== malloc/free: 31726 allocs, 30579 frees, 2322038 bytes
allocated.
==7099==
--7099--     TT/TC: 0 tc sectors discarded.
--7099--            11223 chainings, 0 unchainings.
--7099-- translate: new     14012 (209064 -> 2646829; ratio 126:10)
--7099--            discard 0 (0 -> 0; ratio 0:10).
--7099--  dispatch: 6900000 jumps (bb entries), of which 1304733 (18%)
were unchained.
--7099--            140/88816 major/minor sched events.  17845 tt_fast
misses.
--7099-- reg-alloc: 1950 t-req-spill, 497014+11920 orig+spill uis, 70141
total-reg-r.
--7099--    sanity: 141 cheap, 6 expensive checks.
--7099--    ccalls: 51904 C calls, 58% saves+restores avoided (178738
bytes)
--7099--            71568 args, avg 0.87 setup instrs each (17770 bytes)
--7099--            0% clear the stack (155712 bytes)
--7099--            19332 retvals, 34% of reg-reg movs avoided (12842
bytes)

-- 
Troels Walsted Hansen

Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Nicholas N. <nj...@ca...>]

On Wed, 7 May 2003, Troels Walsted Hansen wrote:

> This is the expected behavior of the program.
>
> rpmdb._rpmdb.DBNoSuchFileError: (2, 'No such file or directory')
>
> This is what happens in Valgrind. RH8 and RH9 uses BerkeleyDB 4.0.14,

> rpmdb._rpmdb.DBAgainError: (11, 'Resource temporarily unavailable --
> nosuchfile.db: Resource temporarily unavailable')

It's possible that Valgrind's doing something slightly different with file
handling that means you get a different behaviour.  Do you have more
information about what exactly is happening to prompt this error?  Eg. is
it trying to open a file with the open() system call, or something like
that?  It's very difficult to know what the problem is from your
description so far.

N

RE: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Troels W. H. <tr...@th...>]

Sorry, I managed to forget to attach the script.

Here's the script, as well as two straces, created with the below
commands, on a RH8+glibc 2.3.3 machine.

strace python dbboguserrno.py 2> strace-normal.txt
strace valgrind python dbboguserrno.py 2> strace-valgrind.txt

Both traces show the below syscall, but the instance running in Valgrind
seems to get EAGAIN instead...

open("nosuchfile.db", O_RDONLY|O_NONBLOCK|O_LARGEFILE) = -1 ENOENT (No
such file or directory)

Troels

> -----Original Message-----
> From: Nicholas Nethercote [mailto:nj...@he...] On 
> Behalf Of Nicholas Nethercote
> Sent: 7. mai 2003 10:43
> To: Troels Walsted Hansen
> Cc: val...@li...
> Subject: Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind
> 
> 
> On Wed, 7 May 2003, Troels Walsted Hansen wrote:
> 
> > This is the expected behavior of the program.
> >
> > rpmdb._rpmdb.DBNoSuchFileError: (2, 'No such file or directory')
> >
> > This is what happens in Valgrind. RH8 and RH9 uses 
> BerkeleyDB 4.0.14,
> 
> > rpmdb._rpmdb.DBAgainError: (11, 'Resource temporarily unavailable --
> > nosuchfile.db: Resource temporarily unavailable')
> 
> It's possible that Valgrind's doing something slightly 
> different with file
> handling that means you get a different behaviour.  Do you have more
> information about what exactly is happening to prompt this 
> error?  Eg. is
> it trying to open a file with the open() system call, or 
> something like
> that?  It's very difficult to know what the problem is from your
> description so far.
> 
> N
> 

Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Julian S. <js...@ac...>]

I imagine this is similar to other problems I fixed ... there is two
implementations of errno, in effect, and threaded one and a non-threaded
one and they are getting mixed up.  I'll look into it.  Thx.

J


On Wednesday 07 May 2003 10:01 am, Troels Walsted Hansen wrote:
> Sorry, I managed to forget to attach the script.
>
> Here's the script, as well as two straces, created with the below
> commands, on a RH8+glibc 2.3.3 machine.
>
> strace python dbboguserrno.py 2> strace-normal.txt
> strace valgrind python dbboguserrno.py 2> strace-valgrind.txt
>
> Both traces show the below syscall, but the instance running in Valgrind
> seems to get EAGAIN instead...
>
> open("nosuchfile.db", O_RDONLY|O_NONBLOCK|O_LARGEFILE) = -1 ENOENT (No
> such file or directory)
>
> Troels
>
> > -----Original Message-----
> > From: Nicholas Nethercote [mailto:nj...@he...] On
> > Behalf Of Nicholas Nethercote
> > Sent: 7. mai 2003 10:43
> > To: Troels Walsted Hansen
> > Cc: val...@li...
> > Subject: Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind
> >
> > On Wed, 7 May 2003, Troels Walsted Hansen wrote:
> > > This is the expected behavior of the program.
> > >
> > > rpmdb._rpmdb.DBNoSuchFileError: (2, 'No such file or directory')
> > >
> > > This is what happens in Valgrind. RH8 and RH9 uses
> >
> > BerkeleyDB 4.0.14,
> >
> > > rpmdb._rpmdb.DBAgainError: (11, 'Resource temporarily unavailable --
> > > nosuchfile.db: Resource temporarily unavailable')
> >
> > It's possible that Valgrind's doing something slightly
> > different with file
> > handling that means you get a different behaviour.  Do you have more
> > information about what exactly is happening to prompt this
> > error?  Eg. is
> > it trying to open a file with the open() system call, or
> > something like
> > that?  It's very difficult to know what the problem is from your
> > description so far.
> >
> > N

RE: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Troels W. H. <tr...@th...>]

Thanks. Let me know if you want me to test any changes.

Troels

-----Original Message-----
From: val...@li...
[mailto:val...@li...] On Behalf Of Julian
Seward
Sent: 8. mai 2003 10:04
To: Troels Walsted Hansen; val...@li...
Subject: Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind



I imagine this is similar to other problems I fixed ... there is two
implementations of errno, in effect, and threaded one and a non-threaded
one and they are getting mixed up.  I'll look into it.  Thx.

J

Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Julian S. <js...@ac...>]

Thanks for chasing this down.  The attached patch fixes it for me on
R H 9 and SuSE 8.2.  Perhaps you can try it?

J


On Wednesday 07 May 2003 10:01 am, Troels Walsted Hansen wrote:
> Sorry, I managed to forget to attach the script.
>
> Here's the script, as well as two straces, created with the below
> commands, on a RH8+glibc 2.3.3 machine.
>
> strace python dbboguserrno.py 2> strace-normal.txt
> strace valgrind python dbboguserrno.py 2> strace-valgrind.txt
>
> Both traces show the below syscall, but the instance running in Valgrind
> seems to get EAGAIN instead...
>
> open("nosuchfile.db", O_RDONLY|O_NONBLOCK|O_LARGEFILE) = -1 ENOENT (No
> such file or directory)
>
> Troels
>
> > -----Original Message-----
> > From: Nicholas Nethercote [mailto:nj...@he...] On
> > Behalf Of Nicholas Nethercote
> > Sent: 7. mai 2003 10:43
> > To: Troels Walsted Hansen
> > Cc: val...@li...
> > Subject: Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind
> >
> > On Wed, 7 May 2003, Troels Walsted Hansen wrote:
> > > This is the expected behavior of the program.
> > >
> > > rpmdb._rpmdb.DBNoSuchFileError: (2, 'No such file or directory')
> > >
> > > This is what happens in Valgrind. RH8 and RH9 uses
> >
> > BerkeleyDB 4.0.14,
> >
> > > rpmdb._rpmdb.DBAgainError: (11, 'Resource temporarily unavailable --
> > > nosuchfile.db: Resource temporarily unavailable')
> >
> > It's possible that Valgrind's doing something slightly
> > different with file
> > handling that means you get a different behaviour.  Do you have more
> > information about what exactly is happening to prompt this
> > error?  Eg. is
> > it trying to open a file with the open() system call, or
> > something like
> > that?  It's very difficult to know what the problem is from your
> > description so far.
> >
> > N

RE: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Troels W. H. <tr...@th...>]

Hi Julian,

I can confirm that your patch solved my testcase. I tested Valgrind
1.9.6 + patch (applied with a small offset) on RH9.

However... I managed to recreate the exact same problem with another
testcase. The new testcase contains the same code, but runs in a
different thread. Script is attached.

$ python dbboguserrno2.py
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.2/threading.py", line 408, in __bootstrap
    self.run()
  File "dbboguserrno2.py", line 8, in run
    filetype = db.DB_BTREE)
  File "/usr/lib/python2.2/site-packages/rpmdb/dbshelve.py", line 69, in
open
    d.open(filename, dbname, filetype, flags, mode)
DBNoSuchFileError: (2, 'No such file or directory')


$ valgrind python dbboguserrno2.py
==24099== Memcheck, a.k.a. Valgrind, a memory error detector for
x86-linux.
==24099== Copyright (C) 2002, and GNU GPL'd, by Julian Seward.
==24099== Using valgrind-1.9.6, a program instrumentation system for
x86-linux.
==24099== Copyright (C) 2000-2002, and GNU GPL'd, by Julian Seward.
==24099== Estimated CPU clock rate is 436 MHz
==24099== For more details, rerun with: -v
==24099==
==24099== valgrind's libpthread.so: IGNORED call to:
pthread_attr_destroy
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.2/threading.py", line 408, in __bootstrap
    self.run()
  File "dbboguserrno2.py", line 8, in run
    filetype = db.DB_BTREE)
  File "/usr/lib/python2.2/site-packages/rpmdb/dbshelve.py", line 69, in
open
    d.open(filename, dbname, filetype, flags, mode)
DBAgainError: (11, 'Resource temporarily unavailable')

Troels

-----Original Message-----
From: val...@li...
[mailto:val...@li...] On Behalf Of Julian
Seward
Sent: 10. mai 2003 02:03
To: Troels Walsted Hansen; val...@li...
Subject: Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind



Thanks for chasing this down.  The attached patch fixes it for me on
R H 9 and SuSE 8.2.  Perhaps you can try it?

J


On Wednesday 07 May 2003 10:01 am, Troels Walsted Hansen wrote:
> Sorry, I managed to forget to attach the script.
>
> Here's the script, as well as two straces, created with the below
> commands, on a RH8+glibc 2.3.3 machine.
>
> strace python dbboguserrno.py 2> strace-normal.txt
> strace valgrind python dbboguserrno.py 2> strace-valgrind.txt
>
> Both traces show the below syscall, but the instance running in
Valgrind
> seems to get EAGAIN instead...
>
> open("nosuchfile.db", O_RDONLY|O_NONBLOCK|O_LARGEFILE) = -1 ENOENT (No
> such file or directory)
>
> Troels

Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Julian S. <js...@ac...>]

> However... I managed to recreate the exact same problem with another
> testcase. The new testcase contains the same code, but runs in a
> different thread. Script is attached.

Er, yes.  There's definitely some still wrong with errno handling
on both my glibc-2.3.2 systems.  The attached test program I made
doesn't work properly -- the perror calls print different stuff when 
running on V.  I'll look into it more tomorrow night.  The strange
thing is the numbers returned by the errno uses are right, it's just
that when perror reads errno it gets junk.

Nick, can you lmk if it works on R H 7.1?  Thx,

J




#include <pthread.h>
#include <stdio.h>
#include <errno.h>

void* thr2 ( void* v )
{
  FILE* f = fopen("bogus2", "r");
  printf("f2 = %p, errno2 = %d\n", f, errno);
  perror("wurble2");
  return NULL;
}

void* thr3 ( void* v )
{
  FILE* f = fopen("bogus3", "r");
  printf("f3 = %p, errno3 = %d\n", f, errno);
  perror("wurble3");
  return NULL;
}

int main ( void )
{
  FILE* f;
  pthread_t tid2, tid3;
  pthread_create(&tid2, NULL, &thr2, NULL);
  pthread_create(&tid3, NULL, &thr3, NULL);
  f = fopen("bogus", "r");
  printf("f1 = %p, errno1 = %d\n", f, errno);
  perror("wurble1");
  pthread_join(tid2, NULL);
  pthread_join(tid3, NULL);
  return 0;
}

Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Tom H. <th...@cy...>]

In message <200...@ac...>
        Julian Seward <js...@ac...> wrote:

> Er, yes.  There's definitely some still wrong with errno handling
> on both my glibc-2.3.2 systems.  The attached test program I made
> doesn't work properly -- the perror calls print different stuff when 
> running on V.  I'll look into it more tomorrow night.  The strange
> thing is the numbers returned by the errno uses are right, it's just
> that when perror reads errno it gets junk.

The errno handling in glibc 2.3.2 is horrible. Some of the routines
in the library don't go through the procedure linkage table when they
call the helper function that gets the address of the current thread's
errno, so they always get glibc's version of that routine instead of
the one that you have tried to replace it with.

This works normally because libpthread calls __libc_pthread_init at
startup and gives it a descriptor block which includes a function
pointer which glibc uses to find the thread descriptor and hence the
errno to use, but if you don't call that function, and if your thread
descriptor don't match the real libpthread ones, then it all falls
apart.

This is why wine now deliberately writes a jmp instruction over the
first instruction of glibc's internal errno address getting routine
so that it guarantee to get control at that point...

Tom

-- 
Tom Hughes (th...@cy...)
Software Engineer, Cyberscience Corporation
http://www.cyberscience.com/

Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Julian S. <js...@ac...>]

> This is why wine now deliberately writes a jmp instruction over the
> first instruction of glibc's internal errno address getting routine
> so that it guarantee to get control at that point...

Well, I've peered at glibc-2.3.2 internals regarding errno and concluded
that it's a *$%*£@ nightmare.

Perhaps Wine's brute-force solution is a better way.

So I peered at the sources for wine-20030508 (a snapshot from a
couple of days ago).  But after some greppery I cannot find the place
where this patching is done.  Can you point me in the right place?

Thanks,

J

Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Dan K. <da...@ke...>]

Julian Seward wrote:
>>This is why wine now deliberately writes a jmp instruction over the
>>first instruction of glibc's internal errno address getting routine
>>so that it guarantee to get control at that point...
> 
> 
> Well, I've peered at glibc-2.3.2 internals regarding errno and concluded
> that it's a *$%*£@ nightmare.
> 
> Perhaps Wine's brute-force solution is a better way.
> 
> So I peered at the sources for wine-20030508 (a snapshot from a
> couple of days ago).  But after some greppery I cannot find the place
> where this patching is done.  Can you point me in the right place?

Before you do this, could you check with the libc-alpha mailing
list and see what Ulrich Dreper has to say about the best way
to address the issue?  Can't hurt...
- Dan


-- 
Dan Kegel
http://www.kegel.com
http://counter.li.org/cgi-bin/runscript/display-person.cgi?user=78045

Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Tom H. <th...@cy...>]

In message <200...@ac...>
          Julian Seward <js...@ac...> wrote:

> So I peered at the sources for wine-20030508 (a snapshot from a
> couple of days ago).  But after some greppery I cannot find the place
> where this patching is done.  Can you point me in the right place?

It's in scheduler/sysdeps.c in the SYSDEPS_InitErrno function.

Tom

-- 
Tom Hughes (th...@cy...)
Software Engineer, Cyberscience Corporation
http://www.cyberscience.com/

RE: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Troels W. H. <tr...@th...>]

I just dicovered that RH has updated the glibc errata for RH8 to version
2.3.2-4.80.6.
https://rhn.redhat.com/errata/RHSA-2003-089.html

There is also a 2.3.2-27.9 version for RH9:
https://rhn.redhat.com/errata/RHBA-2003-136.html

The changelog for the RH8 package contains the entry below, and Valgrind
1.9.6 now works unpatched. The bugzilla entry for that change is at=20
http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=3D86437

"* Sun Apr 06 2003 Jakub Jelinek <ja...@re...> 2.3.2-4.80.5
- make sure all calls to __errno_location() and __h_errno_location()
  jump through .plt to make older wine happy"

This is probably no help to you Julian (since I guess this is a RH
specific compability patch that won't be included in mainline glibc),
but it certainly makes me happy.

Troels

> -----Original Message-----
> From: val...@li...=20
> [mailto:val...@li...] On Behalf=20
> Of Julian Seward
> Sent: 15. mai 2003 01:13
> To: Tom Hughes; val...@li...
> Subject: Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind
>=20
>=20
>=20
> > This is why wine now deliberately writes a jmp instruction over the
> > first instruction of glibc's internal errno address getting routine
> > so that it guarantee to get control at that point...
>=20
> Well, I've peered at glibc-2.3.2 internals regarding errno=20
> and concluded that it's a *$%*=A3@ nightmare.

Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Nicholas N. <nj...@ca...>]

On Mon, 12 May 2003, Julian Seward wrote:

> Er, yes.  There's definitely some still wrong with errno handling
> on both my glibc-2.3.2 systems.  The attached test program I made
> doesn't work properly -- the perror calls print different stuff when
> running on V.  I'll look into it more tomorrow night.  The strange
> thing is the numbers returned by the errno uses are right, it's just
> that when perror reads errno it gets junk.
>
> Nick, can you lmk if it works on R H 7.1?  Thx,

Hmm, I don't think so:

[~/grind/head] a.out
f1 = 0x804b8a8, errno1 = 4
wurble1: Interrupted system call
f2 = (nil), errno2 = 2
wurble2: No such file or directory
f3 = (nil), errno3 = 2
wurble3: No such file or directory

[~/grind/head] vg6 a.out
==557== Memcheck, a.k.a. Valgrind, a memory error detector for x86-linux.
==557== Copyright (C) 2002-2003, and GNU GPL'd, by Julian Seward.
==557== Using valgrind-cvs-head-2003-04-23, a program supervision framework for x86-linux.
==557== Copyright (C) 2000-2003, and GNU GPL'd, by Julian Seward.
==557== Estimated CPU clock rate is 1405 MHz
==557== For more details, rerun with: -v
==557==
f1 = 0x40fd8194, errno1 = 0
wurble1: Success
f2 = (nil), errno2 = 2
wurble2: No such file or directory
f3 = (nil), errno3 = 2
wurble3: No such file or directory
==557==
==557== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 6 from 2)
==557== malloc/free: in use at exit: 564 bytes in 2 blocks.
==557== malloc/free: 9 allocs, 7 frees, 2408 bytes allocated.
==557== For a detailed leak analysis,  rerun with: --leak-check=yes
==557== For counts of detected errors, rerun with: -v

Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Julian S. <js...@ac...>]

On Monday 12 May 2003 8:23 am, Tom Hughes wrote:
> In message <200...@ac...>
>
>         Julian Seward <js...@ac...> wrote:
> > Er, yes.  There's definitely some still wrong with errno handling
> > on both my glibc-2.3.2 systems.  The attached test program I made
> > doesn't work properly -- the perror calls print different stuff when
> > running on V.  I'll look into it more tomorrow night.  The strange
> > thing is the numbers returned by the errno uses are right, it's just
> > that when perror reads errno it gets junk.
>
> The errno handling in glibc 2.3.2 is horrible. Some of the routines
> in the library don't go through the procedure linkage table when they
> call the helper function that gets the address of the current thread's
> errno, so they always get glibc's version of that routine instead of
> the one that you have tried to replace it with.
>
> This works normally because libpthread calls __libc_pthread_init at
> startup and gives it a descriptor block which includes a function
> pointer which glibc uses to find the thread descriptor and hence the
> errno to use, but if you don't call that function, and if your thread
> descriptor don't match the real libpthread ones, then it all falls
> apart.

Thanks so much for this info; I do indeed implement __errno_location()
but I wasn't sure it's always getting used.  This passing of a 
descriptor block does sound like NPTL, tho, and we're operating 
glibc-2.3.2 in old PosixThreads mode so far.  We have confirmation
from Nick that the errno test program fails on glibc-2.2.X, so
something else must be broken.

> This is why wine now deliberately writes a jmp instruction over the
> first instruction of glibc's internal errno address getting routine
> so that it guarantee to get control at that point...

Hmm.  

J

Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Tom H. <th...@cy...>]

In message <200...@ac...>
        Julian Seward <js...@ac...> wrote:

> Thanks so much for this info; I do indeed implement __errno_location()
> but I wasn't sure it's always getting used.  This passing of a 
> descriptor block does sound like NPTL, tho, and we're operating 
> glibc-2.3.2 in old PosixThreads mode so far.  We have confirmation
> from Nick that the errno test program fails on glibc-2.2.X, so
> something else must be broken.

The idea may have come from NPTL but the glibc 2.3.2 that RedHat
shipped as an update for RH8 comes with a version of linuxthreads
that works in the same way.

The main problem is that there isn't an errno_location function
pointer in the descriptor block - instead it uses the thread_descr
function to get the address of the thread descriptor and then expects
to find the thread's errno at a particular offset in that block.

Tom

-- 
Tom Hughes (th...@cy...)
Software Engineer, Cyberscience Corporation
http://www.cyberscience.com/

Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Julian S. <js...@ac...>]

I think I have fixed this problem.  Can you build from cvs and see
if the situation has improved?

Thanks,

J


Building valgrind from the current cvs sources ... is easy.
Here's how:

cvs -d:pserver:ano...@cv...:/cvsroot/valgrind login

When prompted for a password for anonymous, simply press the Enter key.

cvs -z3 -d:pserver:ano...@cv...:/cvsroot/valgrind co 
valgrind
cd valgrind
./autogen.sh
./configure --prefix=....
make
make install

And you should be in business.

8 Dec 2002


On Wednesday 07 May 2003 10:01, Troels Walsted Hansen wrote:
> Sorry, I managed to forget to attach the script.
>
> Here's the script, as well as two straces, created with the below
> commands, on a RH8+glibc 2.3.3 machine.
>
> strace python dbboguserrno.py 2> strace-normal.txt
> strace valgrind python dbboguserrno.py 2> strace-valgrind.txt
>
> Both traces show the below syscall, but the instance running in Valgrind
> seems to get EAGAIN instead...
>
> open("nosuchfile.db", O_RDONLY|O_NONBLOCK|O_LARGEFILE) = -1 ENOENT (No
> such file or directory)
>
> Troels
>
> > -----Original Message-----
> > From: Nicholas Nethercote [mailto:nj...@he...] On
> > Behalf Of Nicholas Nethercote
> > Sent: 7. mai 2003 10:43
> > To: Troels Walsted Hansen
> > Cc: val...@li...
> > Subject: Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind
> >
> > On Wed, 7 May 2003, Troels Walsted Hansen wrote:
> > > This is the expected behavior of the program.
> > >
> > > rpmdb._rpmdb.DBNoSuchFileError: (2, 'No such file or directory')
> > >
> > > This is what happens in Valgrind. RH8 and RH9 uses
> >
> > BerkeleyDB 4.0.14,
> >
> > > rpmdb._rpmdb.DBAgainError: (11, 'Resource temporarily unavailable --
> > > nosuchfile.db: Resource temporarily unavailable')
> >
> > It's possible that Valgrind's doing something slightly
> > different with file
> > handling that means you get a different behaviour.  Do you have more
> > information about what exactly is happening to prompt this
> > error?  Eg. is
> > it trying to open a file with the open() system call, or
> > something like
> > that?  It's very difficult to know what the problem is from your
> > description so far.
> >
> > N

Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Dan K. <da...@ke...>]

Julian Seward wrote:
> I think I have fixed this problem.  Can you build from cvs and see
> if the situation has improved?

Nope, the CVS situation hasn't improved :-)

I still can't retrieve sources from CVS:

Logging in to :pserver:ano...@cv...:2401/cvsroot/valgrind
CVS password:
cvs [login aborted]: end of file from server (consult above messages if any)

Does it actually work for you?  I had the impression the cvs
server's been down for weeks.
- Dan

-- 
Dan Kegel
http://www.kegel.com
http://counter.li.org/cgi-bin/runscript/display-person.cgi?user=78045

Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Dan K. <da...@ke...>]

Dan Kegel wrote:
> Julian Seward wrote:
> 
>> I think I have fixed this problem.  Can you build from cvs and see
>> if the situation has improved?
> 
> 
> Nope, the CVS situation hasn't improved :-)
> 
> I still can't retrieve sources from CVS:

Scratch that.  I can now.  Must have been a cosmic ray.
- Dan

-- 
Dan Kegel
http://www.kegel.com
http://counter.li.org/cgi-bin/runscript/display-person.cgi?user=78045

Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Steve G <lin...@ya...>]

>Does it actually work for you?  I had the impression 
>the cvs server's been down for weeks.

I've had mixed results getting cvs over the last month. I
tried 5 times this week and only got in twice. The times I
got in were slow to start and then seemed OK once it was
underway.

-Steve Grubb

__________________________________
Do you Yahoo!?
SBC Yahoo! DSL - Now only $29.95 per month!
http://sbc.yahoo.com

Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Erik C. <er...@ar...>]

On Sat, Jul 12, 2003 at 07:01:30PM -0700, Dan Kegel wrote:
> Julian Seward wrote:
> > I think I have fixed this problem.  Can you build from cvs and see
> > if the situation has improved?
> 
> Nope, the CVS situation hasn't improved :-)
> 
> I still can't retrieve sources from CVS:
> 
> Logging in to :pserver:ano...@cv...:2401/cvsroot/valgrind
> CVS password:
> cvs [login aborted]: end of file from server (consult above messages if any)

I get that if I have the CVS_RSH environment variable set to ssh
(for other projects).  Unset the variable and it works (for me).

-- 
Erik Corry er...@ar...

Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Dima B. <di...@cs...>]

I get the following error with the current CVS sources of valgrind:

==20126== Memcheck, a.k.a. Valgrind, a memory error detector for x86-linux.
==20126== Copyright (C) 2002-2003, and GNU GPL'd, by Julian Seward.
==20126== Using valgrind-cvs-head-2003-07-13, a program supervision framework for x86-linux.
==20126== Copyright (C) 2000-2003, and GNU GPL'd, by Julian Seward.
==20126== Estimated CPU clock rate is 266 MHz
==20126== For more details, rerun with: -v
==20126== 
Root = /root/mcoda/root
mam_journalinit:env_open: Function not implemented
mammoth: mam_journal.c:52: mam_journalenvinit: Assertion `0' failed.
Aborted

This is using Redhat 9, gcc 322, glibc 2.3.2, nptl, and BerkeleyDB 1.4.25.

Thanks
ttyl
Dima

On Sun, Jul 13, 2003 at 02:28:45AM +0100, Julian Seward wrote:
> 
> I think I have fixed this problem.  Can you build from cvs and see
> if the situation has improved?
> 
> Thanks,
> 
> J
> 
> 
> Building valgrind from the current cvs sources ... is easy.
> Here's how:
> 
> cvs -d:pserver:ano...@cv...:/cvsroot/valgrind login
> 
> When prompted for a password for anonymous, simply press the Enter key.
> 
> cvs -z3 -d:pserver:ano...@cv...:/cvsroot/valgrind co 
> valgrind
> cd valgrind
> ./autogen.sh
> ./configure --prefix=....
> make
> make install
> 
> And you should be in business.
> 
> 8 Dec 2002
> 
> 
> On Wednesday 07 May 2003 10:01, Troels Walsted Hansen wrote:
> > Sorry, I managed to forget to attach the script.
> >
> > Here's the script, as well as two straces, created with the below
> > commands, on a RH8+glibc 2.3.3 machine.
> >
> > strace python dbboguserrno.py 2> strace-normal.txt
> > strace valgrind python dbboguserrno.py 2> strace-valgrind.txt
> >
> > Both traces show the below syscall, but the instance running in Valgrind
> > seems to get EAGAIN instead...
> >
> > open("nosuchfile.db", O_RDONLY|O_NONBLOCK|O_LARGEFILE) = -1 ENOENT (No
> > such file or directory)
> >
> > Troels
> >
> > > -----Original Message-----
> > > From: Nicholas Nethercote [mailto:nj...@he...] On
> > > Behalf Of Nicholas Nethercote
> > > Sent: 7. mai 2003 10:43
> > > To: Troels Walsted Hansen
> > > Cc: val...@li...
> > > Subject: Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind
> > >
> > > On Wed, 7 May 2003, Troels Walsted Hansen wrote:
> > > > This is the expected behavior of the program.
> > > >
> > > > rpmdb._rpmdb.DBNoSuchFileError: (2, 'No such file or directory')
> > > >
> > > > This is what happens in Valgrind. RH8 and RH9 uses
> > >
> > > BerkeleyDB 4.0.14,
> > >
> > > > rpmdb._rpmdb.DBAgainError: (11, 'Resource temporarily unavailable --
> > > > nosuchfile.db: Resource temporarily unavailable')
> > >
> > > It's possible that Valgrind's doing something slightly
> > > different with file
> > > handling that means you get a different behaviour.  Do you have more
> > > information about what exactly is happening to prompt this
> > > error?  Eg. is
> > > it trying to open a file with the open() system call, or
> > > something like
> > > that?  It's very difficult to know what the problem is from your
> > > description so far.
> > >
> > > N
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email sponsored by: Parasoft
> Error proof Web apps, automate testing & more.
> Download & eval WebKing and get a free book.
> www.parasoft.com/bulletproofapps1
> _______________________________________________
> Valgrind-users mailing list
> Val...@li...
> https://lists.sourceforge.net/lists/listinfo/valgrind-users

-- 
Dima Brodsky                                   di...@cs...
                                               http://www.cs.ubc.ca/~dima
201-2366 Main Mall
Department of Computer Science                 (604) 822-2895 (DSG Lab)
University of British Columbia, Canada         (604) 822-5485 (FAX)

"The price of reliability is the pursuit of the utmost simplicity.
 It is a price which the very rich find the most hard to pay."
						  (Sir Antony Hoare, 1980)

RE: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Troels W. H. <tr...@th...>]

Summary: it works.

My original testsystem is using glibc 2.3.2-4.80.6 which contains a
workaround for the problem. Fortunately I had another RH8 box around
with stock glibc (2.2.93-5) installed.

$ cat /etc/redhat-release
Red Hat Linux release 8.0 (Psyche)
$ uname -a
Linux testus 2.4.18-26.8.0 #1 Mon Feb 24 10:21:42 EST 2003 i686 i686
i386 GNU/Linux
$ rpm -q glibc
glibc-2.2.93-5
$ valgrind --version
valgrind-1.9.6

--> both testcases succeed

$ rpm -q glibc
glibc-2.3.2-4.80

--> both testcases fail

$ valgrind --version
valgrind-cvs-head-2003-07-13

--> both testcases succeed

Troels

> -----Original Message-----
> From: val...@li... 
> [mailto:val...@li...] On Behalf 
> Of Julian Seward
> Sent: 13. juli 2003 03:29
> To: Troels Walsted Hansen
> Cc: val...@li...
> Subject: Re: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind
> 
> 
> 
> I think I have fixed this problem.  Can you build from cvs and see
> if the situation has improved?

RE: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind

[Troels W. H. <tr...@th...>]

For completeness (and because I wanted to upgrade the machine anyway), I
also tested with the latest kernel+glibc, and it worked fine.

$ uname -a
Linux testus 2.4.20-18.8 #1 Thu May 29 07:40:27 EDT 2003 i686 i686 i386
GNU/Linux
$ rpm -q glibc
glibc-2.3.2-4.80.6

Troels

> -----Original Message-----
> From: val...@li... 
> [mailto:val...@li...] On Behalf 
> Of Troels Walsted Hansen
> Sent: 14. juli 2003 10:37
> To: js...@ac...
> Cc: val...@li...
> Subject: RE: [Valgrind-users] BerkeleyDB gets bogus errno in Valgrind
> 
> 
> Summary: it works.
> 
> My original testsystem is using glibc 2.3.2-4.80.6 which contains a
> workaround for the problem. Fortunately I had another RH8 box around
> with stock glibc (2.2.93-5) installed.

[Valgrind-users] Valgrind newb here needs help ;-)

[Tristan V. B. <va...@to...>]

Hi everyone!
	I develop jukebox apliences under linux and am considering
valgrind as a memory debugging tool. Personaly I do not have a vast
knowlage
on cpu's and all that jazz, acronymes et al so please bear with me.


My questions are the following:

    Considering that gcc doesn't generate opcodes for anything other
than i386 if not specified and that valgrind takes the i486 instruction
set as a baseline:

        Would it be a fatal mistake to assume that valgrind will support
    the i386 instruction set as a consiquence of supporting the i486
    instruction set ((i486 >= i386) ? backwards compatible) ?

    or would I have to recompile all my code with `-march=i486' ?

===================== from `info gcc' (-mcpu=option) ================
     While picking a specific CPU TYPE will schedule things
     appropriately for that particular chip, the compiler will not
     generate any code that does not run on the i386 without the
     `-march=CPU TYPE' option being used.  `i586' is equivalent to
     `pentium' and `i686' is equivalent to `pentiumpro'.  `k6' is the
     AMD chip as opposed to the Intel ones.
=====================================================================

    I'm a real dunce when it comes to automake and friends ;-)
I ran `./configure --prefix=/usr/local' followed by `make'.
(after that I tried a slew of options such as --host, --build,
--target all of which I cant seem to get right even while
searching through the config.sub).

This is the compile command issued by make that failed:
======================================================================
source='vg_intercept.c' object='vg_intercept.o' libtool=no \
depfile='.deps/vg_intercept.Po' tmpdepfile='.deps/vg_intercept.TPo' \
depmode=gcc /bin/sh ../depcomp \
gcc -DHAVE_CONFIG_H -I. -I. -I.. -I./demangle -I../include    
-DVG_LIBDIR="\"/usr/local/lib"\" -Winline -Wall -Wshadow -O
-fomit-frame-pointer -mpreferred-stack-boundary=2 -g
-fno-omit-frame-pointer -c `test -f 'vg_intercept.c' || echo
'./'`vg_intercept.c

This is the error isued:
=================================================
vg_intercept.c:419: conflicting types for `msgsnd'
/usr/include/sys/msg.h:74: previous declaration of `msgsnd'

and theese are the prototypes:
============ vg_intercept.c:419 =================
#ifdef GLIBC_2_1
int msgsnd(int msgid,       void *msgp, size_t msgsz, int msgflg)
#else
int msgsnd(int msgid, const void *msgp, size_t msgsz, int msgflg)
#endif

============ /usr/include/sys/msg.h ============
extern int msgsnd __P ((int __msqid, __const void *__msgp, size_t
__msgsz,
                        int __msgflg));

I have to admit that after 3 years of seeing this damn macro in header
files I still haven't figured out exactly what it does (`__P') but I
have
a hunch that it removes some of theese leading underscores.

What I do find strange is that in my `msg.h' msgp is `const' and 
that comes from GLIBC_2_1 !!! but if that conditional directive
was buggy; nobody else would be able to compile valgrind either 
(I think). (BTW, GLIBC_2_1 is defined in the config.h)

I am using:
gcc          2.95.3
Linux kernel 2.4.16
glibc        2.1.3-5
valgrind     1.9.6 (latest)
(missing anything ?)

Any help with my compile situation is greatly apreciated.


Regards
			-Tristan


-- 
Education is imposed ignorance. -- Noam Chomsky