valgrind-users — General discussion list for Valgrind users

[Valgrind-users] regtest failure on Centos

[Mark R. <ma...@cs...>]

Running CentOS 7.3.1611 on amd64 box.  Valgrind release 3.12.0 with no mods.

 

Builds without error and runs all regtests without error except for helgrind/tests/bar_bad (and, I assume, that’s why the drd versions fail as well.)

 

Running bar_bad without Valgrind appears to work fine.  When run under Valgrind seems to run find until the end, then there is a long pause – as much as five minutes – and it produces the attached stderr.diff.  I’ve run it directly under valgrind (i.e. no perl vg_regtest) and attached that log as well.

 

Any ideas or suggestions for further debug info?

 

Thank you,

Mark Roberts

UW PLSE 

[Valgrind-users] Valgrind error in lockfree::queue

[Yuval L. <yu...@ya...>]

Hi All,
This is a follow to a discussed that took place yesterday on the irc channel, but would still like to followup on the ML for better tracking. There is a 4 years old ticket on boost's lockfree queue, here: https://svn.boost.org/trac/boost/ticket/8395 
According to the discussion there there is unintialized memory being used, but they failed to change the code in a way that the valgrind error would go away. OTOH, given that the boost lockfree queue is being used under different compilers/OSs etc. it is probably not a real issue.
Was wondering if you can assist in resolving that either by providing info in the boost ticket above on how to solve that in the boost code, or maybe change something in valgrind that could detect this is not a real issue.

For you convenience attaching some info from the execution of the code from the above ticket. gcc version 4.9.2 (Debian 4.9.2-10), valgrind-3.10.0.

Valgrind output: ==9101== Conditional jump or move depends on uninitialised value(s)
==9101==    at 0x401CD1: boost::atomics::detail::base_atomic<boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int*, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node>, void, 8u, false>::compare_exchange_weak(boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int*, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node>&, boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int*, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node> const&, boost::memory_order, boost::memory_order) volatile (gcc-atomic.hpp:812)
==9101==    by 0x401934: boost::atomics::detail::base_atomic<boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int*, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node>, void, 8u, false>::compare_exchange_weak(boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int*, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node>&, boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int*, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node>, boost::memory_order) volatile (gcc-atomic.hpp:822)
==9101==    by 0x401152: bool boost::lockfree::queue<int*, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::do_push<false>(int* const&) (queue.hpp:311)
==9101==    by 0x400C28: boost::lockfree::queue<int*, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::push(int* const&) (queue.hpp:271)
==9101==    by 0x400948: main (test_lockfree.cpp:9)
==9101==
==9101== Conditional jump or move depends on uninitialised value(s)
==9101==    at 0x401155: bool boost::lockfree::queue<int*, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::do_push<false>(int* const&) (queue.hpp:311)
==9101==    by 0x400C28: boost::lockfree::queue<int*, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::push(int* const&) (queue.hpp:271)
==9101==    by 0x400948: main (test_lockfree.cpp:9)
Assembly code:
/usr/include/boost/atomic/detail/gcc-atomic.hpp:812
 storage_type expected_s = 0, desired_s = 0;
 memcpy(&expected_s, &expected, sizeof(value_type));
 memcpy(&desired_s, &desired, sizeof(value_type));
 const bool success = __atomic_compare_exchange_n(&v_, &expected_s, desired_s, true,
 atomics::detail::convert_memory_order_to_gcc(success_order),
 atomics::detail::convert_memory_order_to_gcc(failure_order));
 401cb5: 48 8b 4d e0      mov -0x20(%rbp),%rcx
 401cb9: 48 8b 75 d8      mov -0x28(%rbp),%rsi
 401cbd: 48 8d 55 e8      lea -0x18(%rbp),%rdx
 401cc1: 48 8b 02         mov (%rdx),%rax
 401cc4: f0 48 0f b1 0e   lock cmpxchg %rcx,(%rsi)
 401cc9: 48 89 c1         mov %rax,%rcx
 401ccc: 0f 94 c0         sete %al
 401ccf: 84 c0            test %al,%al
 401cd1: 75 03            jne 401cd6 <_ZNV5boost7atomics6detail11base_atomicINS_8lockfree6detail10tagged_ptrINS3_5queueIPiNS_9parameter5void_ES9_S9_E4nodeEEEvLj8ELb0EE21compare_exchange_weakERSC_RKSC_NS_12memory_orderESH_+0xc0>
 401cd3: 48 89 0a         mov %rcx,(%rdx)
 401cd6: 88 45 ff         mov %al,-0x1(%rbp)

Please let me know if more info is needed.
Best Regards,
Yuval

Re: [Valgrind-users] invalid free() from static destructor on exit()

[Ivo R. <iv...@iv...>]

2017-03-08 11:56 GMT+01:00 Julian Seward <js...@ac...>:
> On 08/03/17 07:16, A. Lester Buck III wrote:
>> I don't understand the last line:
>>
>> Address 0xf95f6a0 is in the BSS segment of /usr/local/lib/libaws-cpp-\
>> sdk-core.so
>
> What this means is, the address that has been passed to free -- that is,
> 0xf95f6a0 -- is zero-initialised data in libaws-cpp-sdk-core.so and so
> has not been allocated on the heap.  It's (the address of) a global
> variable in libaws-cpp-sdk-core.so, in other words.
>
>> How do I use this to figure out where/why the free() error is happening?
>
> You can poke around more with GDB by rerunning the program on V with
> --vgdb-error=0 and following the directions it gives.  GDB will stop at
> the error (at all errors) and you can examine the program state at your
> leisure.

If this is your first time debugging a program under gdb,
you might want to read this manual section:
http://valgrind.org/docs/manual/manual-core-adv.html#manual-core-adv.gdbserver

I.

Re: [Valgrind-users] invalid free() from static destructor on exit()

[Julian S. <js...@ac...>]

On 08/03/17 07:16, A. Lester Buck III wrote:
> I don't understand the last line:
> 
> Address 0xf95f6a0 is in the BSS segment of /usr/local/lib/libaws-cpp-\
> sdk-core.so

What this means is, the address that has been passed to free -- that is,
0xf95f6a0 -- is zero-initialised data in libaws-cpp-sdk-core.so and so
has not been allocated on the heap.  It's (the address of) a global
variable in libaws-cpp-sdk-core.so, in other words.

> How do I use this to figure out where/why the free() error is happening?

You can poke around more with GDB by rerunning the program on V with
--vgdb-error=0 and following the directions it gives.  GDB will stop at
the error (at all errors) and you can examine the program state at your
leisure.

J

[Valgrind-users] invalid free() from static destructor on exit()

[A. L. B. I. <val...@nb...>]

I am writing a text-to-speech loadable module for the open source
telephony server Freeswitch, using the new AWS Polly text-to-speech
service.  The AWS C++ SDK is C++11, templatized, etc., and I am wrapping
it in C with calls to factory methods returning opaque pointers to
objects.  I am not actually executing any AWS functions at the moment,
just the static allocation of these two variables that set up the SDK
and the client.

static Aws::SDKOptions options;
static Aws::Client::ClientConfiguration config;

The rest of my Freeswitch module code is stubbed out, and makes no AWS
calls.  When I start and stop Freeswitch, the stub Polly module loads
and unloads.  At server exit, I get a free() error.  After recompiling
everything with -O0, I get this single invalid output from valgrind:

==5892== Invalid free() / delete / delete[] / realloc()
==5892==    at 0x4C29E90: free (vg_replace_malloc.c:473)
==5892==    by 0xF467A9D: Aws::Allocator<char>::deallocate(char*, unsigned long) (AWSAllocator.h:74)
==5892==    by 0xF467778: std::basic_string<char, std::char_traits<char>, Aws::Allocator<char> >::_Rep::_M_destroy(Aws::Allocator<char> const&) (basic_string.tcc:449)
==5892==    by 0xF4674F3: std::basic_string<char, std::char_traits<char>, Aws::Allocator<char> >::_Rep::_M_dispose(Aws::Allocator<char> const&) (basic_string.h:249)
==5892==    by 0xF467386: std::basic_string<char, std::char_traits<char>, Aws::Allocator<char> >::~basic_string() (basic_string.h:547)
==5892==    by 0xF46706F: Aws::Client::ClientConfiguration::~ClientConfiguration() (ClientConfiguration.h:48)
==5892==    by 0x68B9B28: __run_exit_handlers (exit.c:82)
==5892==    by 0x68B9B74: exit (exit.c:104)
==5892==    by 0x68A3B4B: (below main) (libc-start.c:321)
==5892==  Address 0xf95f6a0 is in the BSS segment of /usr/local/lib/libaws-cpp-sdk-core.so
==5892== 
==5892== 

I got the glibc source for my Linux and the last line of libc-start.c is
line 321, the return statement from main().

There are a bunch of small allocation errors in the rest of Freeswitch,
but no more "Invalid free/delete/delete[]/realloc()" errors.  I also
wrote a minimal dlopen/dlsym/dlclose main program to exercise the
loadable module, and got no free() errors from the static destructor,
though the build systems are somewhat different from my toy main and the
entire Freeswitch server.

The AWS C++ SDK has a flexible memory allocation system, but I am not
using any of that and thus it defaults to the system built in
malloc/free/new/delete.

I don't understand the last line:

Address 0xf95f6a0 is in the BSS segment of /usr/local/lib/libaws-cpp-\
sdk-core.so

Is there anything unusual in the trace?  How do I use this to figure out
where/why the free() error is happening?

Thanks!

Re: [Valgrind-users] Migrating Valgrind sources from SVN to GIT

[Ivo R. <iv...@iv...>]

2017-02-24 20:21 GMT+01:00 Ivo Raisr <iv...@iv...>:
> Dear Valgrind community,
>
> We are pleased to announce an imminent migration of Valgrind sources
> from existing Subversion SCM to modern git SCM, as discussed during
> our FOSDEM 2017 Valgrind devroom.

So far a number of very useful comments have been received.
Necessary infrastructure changes in Valgrind code repo and Valgrind
www repo were prepared,
together with the migration recipe.

Please review them all here: https://github.com/ivosh/valgrind-git-migration

1. Migration recipe itself:
https://github.com/ivosh/valgrind-git-migration/blob/master/migration-recipe.txt

2. Necessary Valgrind code repo changes:
https://github.com/ivosh/valgrind-git-migration/blob/master/001-Change-SVN-to-GIT-in-various-places.patch
https://github.com/ivosh/valgrind-git-migration/blob/master/002-Update-auxprogs-update-demangler-for-Valgrind-in-GIT.patch
https://github.com/ivosh/valgrind-git-migration/blob/master/003-Fix-nightly-build-script-to-work-with-git.patch
https://github.com/ivosh/valgrind-git-migration/blob/master/004-fix-verbose-version-reporting-for-git.patch
https://github.com/ivosh/valgrind-git-migration/blob/master/005-Change-Subversion-to-GIT-in-various-places.patch
https://github.com/ivosh/valgrind-git-migration/blob/master/006-Convert-release-HOWTO.txt-from-SVN-to-GIT.patch

3. Website changes (residing in valgrind-www SVN repository):
https://github.com/ivosh/valgrind-git-migration/blob/master/valgrind-www-changes.patch

Please provide your comments and suggestions.
We are very close to the final migration!

I.

Re: [Valgrind-users] New leaks when running valgrind in a docker container

[Dan K. <da...@ke...>]

On Sat, Mar 4, 2017 at 9:46 AM, Ulrich Hierl <kni...@fr...> wrote:
> 101 bytes in 1 blocks are possibly lost in loss record 241 of 365
> at : operator new(unsigned long)
> by : std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&)
> by : char* std::string::_S_construct<char const*>(char const*, char const*, std::allocator<char> const&, std::forward_iterator_tag)
> by : testing::internal::MakeAndRegisterTestInfo(char const*, char const*, char const*, char const*, testing::internal::CodeLocation, void const*, void (*)(), void (*)(), testing::internal::TestFactoryBase*) (gtest.cc:2555)
> by : __static_initialization_and_destruction_0(int, int) (ClassFilesManipulator_expensive_tests.cpp:166)

That looks like an expected and benign leak from google test.
I'd just suppress it, and anything else from testing::internal.
- Dan

[Valgrind-users] New leaks when running valgrind in a docker container

[Ulrich H. <kni...@fr...>]

Hi valgrind folk,

I am currently transforming my build-slave from a virtual machine to a 
docker container. One step of my build is to run valgrind. The problem 
is that I now get new memory leaks that do not occur on the virtual 
machine.

The errors come from libc and std::string, here is an example:

==11397== 101 bytes in 1 blocks are possibly lost in loss record 241 of 365
==11397==    at 0x4C29180: operator new(unsigned long) 
(vg_replace_malloc.c:324)
==11397==    by 0x62ABE98: std::string::_Rep::_S_create(unsigned long, 
unsigned long, std::allocator<char> const&) (in 
/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.20)
==11397==    by 0x4FD2F1: char* std::string::_S_construct<char 
const*>(char const*, char const*, std::allocator<char> const&, 
std::forward_iterator_tag) (basic_string.tcc:138)
==11397==    by 0x62ADC45: std::basic_string<char, 
std::char_traits<char>, std::allocator<char> >::basic_string(char 
const*, std::allocator<char> const&) (in 
/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.20)
==11397==    by 0x4D6644: 
testing::internal::MakeAndRegisterTestInfo(char const*, char const*, 
char const*, char const*, testing::internal::CodeLocation, void const*, 
void (*)(), void (*)(), testing::internal::TestFactoryBase*) (gtest.cc:2555)
==11397==    by 0x4B0798: __static_initialization_and_destruction_0(int, 
int) (ClassFilesManipulator_expensive_tests.cpp:166)
==11397==    by 0x4B8F8B: 
_GLOBAL__sub_I_ClassFilesManipulator_expensive_tests.cpp 
(ClassFilesManipulator_expensive_tests.cpp:269)
==11397==    by 0x55FE6C: __libc_csu_init (in 
/home/jenkins/CppLibraries/CppCodeBase/Generated/CompiledBinaries/LinuxMakeGccDebug/CodeAssistant_expensive_tests)
==11397==    by 0x6A30AD4: (below main) (libc-start.c:246)

Now I am not sure if this is a problem of running valgrind in a docker 
container or if my container has a different libc then my virtual 
machine which may cause the problem.

Do you have experience with valgrind in a container and are there any 
known issues?


Cheers Knitschi

Re: [Valgrind-users] Problem of running mozilla thunderbird under valgrind on Debian GNU/Linux 4.x kernel.

[ISHIKAWA,chiaki <ish...@yk...>]

After looking for a suitable place to post lengthy log files,
I filed https://bugs.kde.org/show_bug.cgi?id=377006
in valgrind bugzilla with a set of log files.

Sorry, I have forgotten to post a notice to this mailing list.

If someone can figure out a possible way to proceed with debugging, I 
would appreciate to hear it.

TIA


On 2017/02/22 2:25, ISHIKAWA,chiaki wrote:
> Sorry for top-posting, but thank you for the suggestions.
>
> So far, I figured out that the maps are different under 3.19.5, 4.7.0.1,
> and 4.9.6 versions of linux kernel.
>
> Also, I have figured out the SIGSEGV problem is timing-related/race
> under 4.7.0.1
> (Worst bug in terms of reproducibility).
> If managed to attach to the thunderbird binary executing under valgrind
> using --vgdb=y and --vgdb-error=0 and single step (or step over
> functions) through thunderbird to figure out what kind of thunderbird
> behavior may trigger valgrind problem.
> Then I noticed the SIGSEGV did not occur when I stepped through the code
> (over the fork) while if I simply run the thunderbird code by "cont" all
> the way, SIGSEGV occurs :-(
> To sum up, under 4.7.0.1, the SIGSEGV seems to occur near the fork()
> system call.
> Thunderbird invokes a small glxtest program which checks for the
> graphics driver info (for debugging?). And fork() is reached before
> SIGSEGV is observed under 4.7.0.1.
> [I thought that I was homing on a possible bug.]
> But under 4.9.6, the SIGSEGV seems to occur way before this fork() is
> executed and it is very difficult yet to figure out where the SIGSEGV
> occurs.
>
> Under 3.19.5, thunderbird runs under valgrind just fine.
>
> From the way it goes, I will be able to post the logs with some results
> from additional probes at the beginning of next week.
>
>
> On 2017/02/20 8:30, John Reiser wrote:
>> On 02/18/2017 11:38 PM, ISHIKAWA,chiaki wrote:
>>>   BTW, DOES ANYONE HAVE A GOOD IDEA ABOUT HOW TO CAPTURE the mapped
>>> file, etc WHEN SIGSEGV happens? It is very dynamic and by the time I
>>> am ready to type in shell commands, the child binary that experienced
>>> it may be gone. Yes, I have not been able to figure out exactly which
>>> process under the test
>>> suite setup started by thunderbird (under valgrind) is experiencing a
>>> difficulty.
>>> I guess some clever hacking via gdb gets me started there?
>>> BTW, valgrind's --gdb-* options are meant to debug the target under
>>> valgrind, NOT the segfault of valgrind itself, correct?
>>> [And the whole thing including valgrind works under kernel 3.19.5 and
>>> not under later kernel drives me crasy.]
>>
>> This gdb command will stop execution and print a message when SIGSEGV
>> happens:
>>     (gdb) handle SIGSEGV stop print
>> When the SIGSEGV happens then you will have to focus keyboard input to
>> that process.
>> (The above 'handle' command is the default anyway, so if the automation
>> for your
>> test harness snatches control, then you still might not get a chance for
>> manual input.)
>> There is no way to ask of gdb, "Please run these commands upon SIGSEGV."
>>
>> You can write a script for the entire input to gdb: gdb -batch -x script
>> -e executable
>> (beware: it is very brittle) but gdb cannot switch its input stream
>> (such as back and forth between the script and the terminal)
>> while it is running.  "gdb -batch -x script -e executable" might be your
>> best option,
>> but it will take some patience.  There is no way for the script to
>> check that gdb is waiting for input after SIGSEGV, so you just have to
>> assume
>> that the SIGSEGV is going to happen after your 'run' command in the
>> input.
>>
>> Yes, valgrind's --gdb-* options are for debugging the target under
>> valgrind,
>> and are NOT for debugging valgrind itself.
>>
>>
>> If you run "strace -f -o strace.out -e trace=execve valgrind
>> --trace-children=yes ..."
>> then the output in strace.out will tell you which process receives the
>> SIGSEGV.
>> The "-e trace=execve" is a filter which restricts tracing to execve only;
>> otherwise the output will be very long because it contains every system
>> call
>> for every process.
>>
>
>
>

Re: [Valgrind-users] Clarification on functions in pub_tool_tooliface.h

[Ivo R. <iv...@iv...>]

2017-03-03 0:01 GMT+01:00 Mike Lui <mik...@gm...>:
> I did a poor job explaining my inquiry. I'm writing a valgrind tool that's
> tracking the IOPS/FLOPS/reads/writes reported by VEX IR during
> instrumentation, and doing some minor analysis on the IR with a
> instrument_basic_block callback (as documented in
> http://www.valgrind.org/docs/manual/writing-tools.html)

That is interesting. How are you going to represent the results?
Cannot be the existing tools "persuaded" to do what you need?

> So my understanding right now is that as long as a syscall wrapper exists,
> registering to *_mem_write will give me metadata about userspace memory
> modified by supported syscalls, otherwise I use the normal
> instrument_basic_block callback.

Alright.

I.

Re: [Valgrind-users] Clarification on functions in pub_tool_tooliface.h

[Mike L. <mik...@gm...>]

I did a poor job explaining my inquiry. I'm writing a valgrind tool that's
tracking the IOPS/FLOPS/reads/writes reported by VEX IR during
instrumentation, and doing some minor analysis on the IR with a
instrument_basic_block callback (as documented in
http://www.valgrind.org/docs/manual/writing-tools.html )

I was interested how to also see which data was written to by any syscalls
(not to instrument syscalls), and saw the aforementioned callbacks in
pub_tool_tooliface.h
After reading the comments, my confusion was whether registering a callback
to (*_mem_write) would instrument ALL memory operations (rendering my
instrument_basic_block unnecessary for processing writes).

So my understanding right now is that as long as a syscall wrapper exists,
registering to *_mem_write will give me metadata about userspace memory
modified by supported syscalls, otherwise I use the normal
instrument_basic_block callback. Because I'm not currently interested in
events that happen within the core framework (besides thread swapping,
which I already detect with VG_(get_running_tid)), I wouldn't need to
register to other aforementioned callbacks.



On Thu, Mar 2, 2017 at 5:34 PM Ivo Raisr <iv...@iv...> wrote:

> 2017-03-02 23:07 GMT+01:00 Mike Lui <mik...@gm...>:
> > Hi Ivo,
> >
> > I didn't get a response, but I found a snippet in the original paper:
> > https://www.cs.columbia.edu/~junfeng/09fa-e6998/papers/valgrind.pdf
> >
> > Table 1 says these functions are only for syscalls and the valgrind
> core. Is
> > that still correct?
>
> Hi Mike,
>
> As comment on line 517 in pub_tool_tooliface.h says, these are
> callback functions
> which the Valgrind core (framework) use to pass various events to the
> analysis tool.
> Tool can be one of memcheck, helgrind, drd, cachegrind, callgrind,
> massif, ... [1]
> Various tools are interested in slightly different events, so they do
> or don't register
> event callbacks. Syscall is just one type of such an event.
>
> > callbacks specifically refer to events within the core, right?
> >> > Does this mean that they bear no relevance to profiling the user
> application?
>
> User application usually does not interact with these events.
> What are you trying to achieve? What is your goal? Asking too detailed
> question
> without knowing the context (what are you up to) does not usually lead to
> meaningful answers.
>
> >> > Am I correct that a "mem_write" callback only gets called for memory
> >> > write that happens during translation, internal signal handling, etc,
> along
> >> > with the "new_mmap" calls?
>
> Nope. pre_mem_write and post_mem_write are callbacks for events emited
> by Valgrind core (framework) when it intercepted a syscall on behalf
> of the application.
> Because Valgrind has no idea what is happening during a syscall inside
> kernel,
> this is described by so called "syscall wrappers". They tend to describe
> things
> such as "this particular piece of memory is going to be written to  by
> kernel during syscall (pre_mem_write)"
> and "this particular piece of memory was written to by kernel during
> syscall (post_mem_write)".
>
> >  If I were to instrument the application under test then I'd go through
> the callback given in "basic_tool_funcs", correct?
>
> Instrumentation happens automatically, depending on the analysis tool
> selected at Valgrind startup.
> I recommend you to read Valgrind intro [1] and core [2], at least the
> initial part.
>
> I.
>
> [1] http://valgrind.org/docs/manual/manual-intro.html
> [2] http://valgrind.org/docs/manual/manual-core.html
>

Re: [Valgrind-users] Clarification on functions in pub_tool_tooliface.h

[Ivo R. <iv...@iv...>]

2017-03-02 23:07 GMT+01:00 Mike Lui <mik...@gm...>:
> Hi Ivo,
>
> I didn't get a response, but I found a snippet in the original paper:
> https://www.cs.columbia.edu/~junfeng/09fa-e6998/papers/valgrind.pdf
>
> Table 1 says these functions are only for syscalls and the valgrind core. Is
> that still correct?

Hi Mike,

As comment on line 517 in pub_tool_tooliface.h says, these are
callback functions
which the Valgrind core (framework) use to pass various events to the
analysis tool.
Tool can be one of memcheck, helgrind, drd, cachegrind, callgrind,
massif, ... [1]
Various tools are interested in slightly different events, so they do
or don't register
event callbacks. Syscall is just one type of such an event.

> callbacks specifically refer to events within the core, right?
>> > Does this mean that they bear no relevance to profiling the user application?

User application usually does not interact with these events.
What are you trying to achieve? What is your goal? Asking too detailed question
without knowing the context (what are you up to) does not usually lead to
meaningful answers.

>> > Am I correct that a "mem_write" callback only gets called for memory
>> > write that happens during translation, internal signal handling, etc, along
>> > with the "new_mmap" calls?

Nope. pre_mem_write and post_mem_write are callbacks for events emited
by Valgrind core (framework) when it intercepted a syscall on behalf
of the application.
Because Valgrind has no idea what is happening during a syscall inside kernel,
this is described by so called "syscall wrappers". They tend to describe things
such as "this particular piece of memory is going to be written to  by
kernel during syscall (pre_mem_write)"
and "this particular piece of memory was written to by kernel during
syscall (post_mem_write)".

>  If I were to instrument the application under test then I'd go through the callback given in "basic_tool_funcs", correct?

Instrumentation happens automatically, depending on the analysis tool
selected at Valgrind startup.
I recommend you to read Valgrind intro [1] and core [2], at least the
initial part.

I.

[1] http://valgrind.org/docs/manual/manual-intro.html
[2] http://valgrind.org/docs/manual/manual-core.html

Re: [Valgrind-users] Clarification on functions in pub_tool_tooliface.h

[Mike L. <mik...@gm...>]

Hi Ivo,

I didn't get a response, but I found a snippet in the original paper:
https://www.cs.columbia.edu/~junfeng/09fa-e6998/papers/valgrind.pdf

Table 1 says these functions are only for syscalls and the valgrind core.
Is that still correct?

Mike

On Thu, Mar 2, 2017 at 4:28 PM Ivo Raisr <iv...@iv...> wrote:

2017-02-25 3:55 GMT+01:00 Mike Lui <mik...@gm...>:
> Being unable to search through gmane for now, I turn to the users group.
>
> I'd like some clarification regarding some functions I'm seeing in the
tool
> interface.
>
> track_new_mem_startup
> ....
> track_new_mem_mmap
> ...
> track_pre_mem_read
> track_pre_mem_write
> track_post_mem_write
> track_post_reg_write
>
> These callbacks specifically refer to events within the core, right? Does
> this mean that they bear no relevance to profiling the user application?
>
> Am I correct that a "mem_write" callback only gets called for memory write
> that happens during translation, internal signal handling, etc, along with
> the "new_mmap" calls? If I were to instrument the application under test
> then I'd go through the callback given in "basic_tool_funcs", correct?
>
> I only asks because I just noticed these functions, and the wording in the
> comments wasn't incredibly clear.

Did you get any response for your query? If not, I will chime in.
I.

Re: [Valgrind-users] Clarification on functions in pub_tool_tooliface.h

[Ivo R. <iv...@iv...>]

2017-02-25 3:55 GMT+01:00 Mike Lui <mik...@gm...>:
> Being unable to search through gmane for now, I turn to the users group.
>
> I'd like some clarification regarding some functions I'm seeing in the tool
> interface.
>
> track_new_mem_startup
> ....
> track_new_mem_mmap
> ...
> track_pre_mem_read
> track_pre_mem_write
> track_post_mem_write
> track_post_reg_write
>
> These callbacks specifically refer to events within the core, right? Does
> this mean that they bear no relevance to profiling the user application?
>
> Am I correct that a "mem_write" callback only gets called for memory write
> that happens during translation, internal signal handling, etc, along with
> the "new_mmap" calls? If I were to instrument the application under test
> then I'd go through the callback given in "basic_tool_funcs", correct?
>
> I only asks because I just noticed these functions, and the wording in the
> comments wasn't incredibly clear.

Did you get any response for your query? If not, I will chime in.
I.

Re: [Valgrind-users] [Valgrind-developers] Migrating Valgrind sources from SVN to GIT

[Ivo R. <iv...@iv...>]

2017-02-27 21:45 GMT+01:00 Matthias Schwarzott <zz...@ge...>:
>
> I wonder if the parameter 10 in --abbrev=10 should be skipped, now that
> git has an automatic estimation for the number of digits to be used.

Are you referring to "--short=10" command line option for "git rev-parse"?
The manual [1] states that if no length is specified than 7 is used.
Could you give us a reference where the automatic length estimation is
described?
I.

[1] https://git-scm.com/docs/git-rev-parse/

Re: [Valgrind-users] Migrating Valgrind sources from SVN to GIT

[Ivo R. <iv...@iv...>]

2017-02-28 23:13 GMT+01:00 Tom Hughes <to...@co...>:
> On 28/02/17 20:56, Ivo Raisr wrote:
>
>> So we decided to stick with existing (SVN) workflow which translates to
>> "rebased branches at the top of the tree".
>> Our valgrind.git config will have (after the final migration happens):
>>
>> [receive]
>>         denyNonFastforwards = true
>
>
> That doesn't actually prevent people pushing merges though, it just stops
> history being rewritten - the push to the remote can only move the remote
> forward but the pushed commits can include merges.

Good point.
So I think it's the time to start gathering config for AdaCore git hooks [1] [2]
which is used at sourceware.org.

----------------------------------
[hooks]
    from-domain = sourceware.org
    mailinglist = val...@li...

    # Allow to include debugging output in commit messages.
    max-rh-line-length = 0

    # Forces to rebase changes before pushing to master and release branches.
    reject-merge-commits = refs/heads/master, refs/heads/VALGRIND_.*

    commit-url = "https://sourceware.org/git/?p=valgrind.git;h=%(rev)s"

    # No emails from private user branches.
    no-emails = /refs/heads/user/.*
----------------------------------

Your thoughts?
I.


[1] https://github.com/adacore/git-hooks
[2] https://sourceware.org/gdb/wiki/GitHooksUsersGuide

Re: [Valgrind-users] Migrating Valgrind sources from SVN to GIT

[Tom H. <to...@co...>]

On 28/02/17 20:56, Ivo Raisr wrote:

> So we decided to stick with existing (SVN) workflow which translates to
> "rebased branches at the top of the tree".
> Our valgrind.git config will have (after the final migration happens):
>
> [receive]
>         denyNonFastforwards = true

That doesn't actually prevent people pushing merges though, it just 
stops history being rewritten - the push to the remote can only move the 
remote forward but the pushed commits can include merges.

Tom

-- 
Tom Hughes (to...@co...)
http://compton.nu/

Re: [Valgrind-users] [Valgrind-developers] Migrating Valgrind sources from SVN to GIT

[Ivo R. <iv...@iv...>]

2017-02-28 22:16 GMT+01:00 Christian Borntraeger <bor...@de...>:
> The given workflow requires that there is no other push between the pull
> and the push. If for some reason somebody else pushes some update after
> you have commited, you can recover with a rebase. There is a nice combine
> of pull + rebase:
> git pull --rebase
> and I think we might just want to document that.

Definitely yes. Would you mind suggesting something like
docs/internals/git-HOWTO.txt,
possibly taking into account existing docs/internals/svn-HOWTO.txt.

Thank you,
I.

Re: [Valgrind-users] [Valgrind-developers] Migrating Valgrind sources from SVN to GIT

[Christian B. <bor...@de...>]

On 02/28/2017 09:56 PM, Ivo Raisr wrote:
[...]
> 
> So we decided to stick with existing (SVN) workflow which translates to
> "rebased branches at the top of the tree".

Which is a perfectly fine model that makes sense in our project.


> Our valgrind.git config will have (after the final migration happens):
> 
> [receive]
>         denyNonFastforwards = true
> 
> So based on this information, could you suggest different git commands
> (with some reasoning) than initially outlined by me?

The given workflow requires that there is no other push between the pull
and the push. If for some reason somebody else pushes some update after 
you have commited, you can recover with a rebase. There is a nice combine
of pull + rebase:
git pull --rebase
and I think we might just want to document that.

Re: [Valgrind-users] [Valgrind-developers] Migrating Valgrind sources from SVN to GIT

[Ivo R. <iv...@iv...>]

2017-02-27 10:02 GMT+01:00 Christian Borntraeger <bor...@de...>:
> For a pull like approach I would suggest to either
> a: do not this step (Linux style git handling)
> b: use git pull --rebase (flat history style)
>
>
>> build + test
>> git commit
>> [git push - if you have write access]
>
> If we give write accesses, then I suggest to use the --rebase variant.
> Not sure what git server is running, but git-o-lite for example allows
> to reject non-fast-forward merges, so that we do not fill the
> history with single patch merge commits.
> Merge commits can be very useful though, if the merge commit contains
> a description of a multititude of patches.

Are you referring here to the git merge model?

We had a discussion with Mark Wielaard previously which git model to use:
----------------------
> > And given the git model we should also
> > decide whether we want a merge model or that we ask people to only push
> > rebased branches at the top of the tree. The merge model is more
> > flexible and more accurately shows the development history if features
> > get developed/integrated in parallel. But having just one straight
> > linear commit history often helps with bisecting and precisely
> > pin-pointing when bugs were introduced. I know sourceware can setup
> > commit/push hooks to make sure one or the other model is followed.
>
> I think the second one reflects what we have been using in SVN.
> Any reason to change that other than that it's more GIT'ish?

I think that is a fine reason and probably works out nicely when
multiple people push to the same repository. The "more GIT'sh" way is
(IMHO) slightly nicer when there is a single person that pushes to a
repo. Merges often represent work others did and show where they started
off and where they integrated their work. But this can (IMHO) get
slightly messy if multiple people push to the same repo.
---------------------

So we decided to stick with existing (SVN) workflow which translates to
"rebased branches at the top of the tree".
Our valgrind.git config will have (after the final migration happens):

[receive]
        denyNonFastforwards = true

So based on this information, could you suggest different git commands
(with some reasoning) than initially outlined by me?

I.

Re: [Valgrind-users] macos 10.12 not working

[Ivo R. <iv...@iv...>]

2017-02-28 8:52 GMT+01:00 James Orr <jam...@wu...>:
> I believe this is a known issue with MacOSX Sierra.

Patches are welcome!
I.

Re: [Valgrind-users] macos 10.12 not working

[James O. <jam...@wu...>]

I believe this is a known issue with MacOSX Sierra.

On Tue, Feb 28, 2017 at 1:45 AM, Thomas Manninger <DBG...@gm...> wrote:

> Hi,
>
> i am using valgrind on macos 10.12.3 (intel Core i5).
> After compiling the newest valgrind version from svn, i cannot start
> valgrind for any application.
>
> I get the following error:
> $ sudo ./valgrind -v /Users/user/Documents/application
> valgrind: mmap-FIXED(0x0, 253952) failed in UME (load_segment1) with error
> 12 (Cannot allocate memory).
>
> $ ./valgrind --version
> valgrind-3.13.0.SVN
>
> How can i solve this problem?
> Thanks for help!
>
> best regards,
> Thomas
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Valgrind-users mailing list
> Val...@li...
> https://lists.sourceforge.net/lists/listinfo/valgrind-users
>
>

[Valgrind-users] macos 10.12 not working

[Thomas M. <DBG...@gm...>]

<html><head></head><body><div style="font-family: Verdana;font-size: 12.0px;"><div>Hi,</div>

<div>&nbsp;</div>

<div>i am using valgrind on macos 10.12.3 (intel Core i5).</div>

<div>After compiling the newest valgrind version from svn, i cannot start valgrind for any application.</div>

<div>&nbsp;</div>

<div>I get the following error:</div>

<div>&#36; sudo ./valgrind -v /Users/user/Documents/application<br/>
valgrind: mmap-FIXED(0x0, 253952) failed in UME (load_segment1) with error 12 (Cannot allocate memory).</div>

<div>&nbsp;</div>

<div>&#36; ./valgrind --version<br/>
valgrind-3.13.0.SVN</div>

<div>&nbsp;</div>

<div>How can i solve this problem?<br/>
Thanks for help!</div>

<div>&nbsp;</div>

<div>best regards,</div>

<div>Thomas</div></div></body></html>

Re: [Valgrind-users] [Valgrind-developers] Migrating Valgrind sources from SVN to GIT

[Christian B. <bor...@de...>]

On 02/24/2017 08:21 PM, Ivo Raisr wrote:
> Dear Valgrind community,
> 
> We are pleased to announce an imminent migration of Valgrind sources
> from existing Subversion SCM to modern git SCM, as discussed during
> our FOSDEM 2017 Valgrind devroom.

Very nice, thank you.

> 
> What is going on now?
> ~~~~~~~~~~~~~~~~~
> The migration has just started. We are now in beta testing stage.
> We still use the official SVN Valgrind repository for our work until
> the final migration step. If you have some patches ready now, send
> them for review.
> You can contribute to the migration process - read below.
> 
> What will be migrated:
> ~~~~~~~~~~~~~~~~~
> Valgrind and VEX sources. Precisely sources available today under
> svn://svn.valgrind.org/valgrind and svn://svn.valgrind.org/vex, including
> all production release branches and tags. Valgrind and VEX repos will
> be merged into one, so no more SVN externals.
> 
> Where I will find the new repo:
> ~~~~~~~~~~~~~~~~~~~~~~~
> At sourceware.org. Precisely at:
> git://sourceware.org/git/valgrind.git/
> http://sourceware.org/git/valgrind.git/
> Right now a snapshot of SVN sources as of 2017-02-21 is available for
> you to test.
> 
> How the test migration was performed:
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> See recipes at https://github.com/ivosh/valgrind-git-migration
> 
> What is the plan for the migration to go forward:
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 1. Test migration has been performed and initial tests were successful.
> 2. The test repo is now available to test and play for others with - see below
>     for details.
> 3. Prepare www (website) and nightly build script changes and have
> them reviewed.
> 4. Proceed once 2+3 are successfully done.
> 5. Announce the final migration.
> 6. Completely eradicate contents in the GIT repository so the migration
>     can start from scratch.
> 7. Switch SVN valgrind+vex repo readonly.
> 8. Perform the final migration to sourceware.org.
> 9. Enable email notifications from new git repo.
> 10. Push www and nightly script changes to the new repo.
> 
> What will not be migrated:
> ~~~~~~~~~~~~~~~~~~~~
> - Valgrind www (website) repo. Not now, but later.
> - Non production release branches and tags from old SVN Valgrind+VEX repos.
>   If you need to preserve some other branches or tags, let us know:
>   https://sourceware.org/git/?p=valgrind.git;a=heads
>   https://sourceware.org/git/?p=valgrind.git;a=tags
> 
> I have a write access to existing SVN repo. What shall I do for the new one?
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Please contact Julian Seward. He will point you to specific instructions.
> 
> What will be my simple workflow in new git SCM?
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Not much will be changed from the way we worked in SVN.
> We still prepare patches, send them for review, have someone
> with write access to push them. A minimalistic workflow would be:
> 
> git clone git://sourceware.org/git/valgrind.git/ valgrind
> edit/compile
> git status/add/show
> git pull origin/master

For a pull like approach I would suggest to either
a: do not this step (Linux style git handling)
b: use git pull --rebase (flat history style)


> build + test
> git commit
> [git push - if you have write access]

If we give write accesses, then I suggest to use the --rebase variant.
Not sure what git server is running, but git-o-lite for example allows
to reject non-fast-forward merges, so that we do not fill the
history with single patch merge commits.
Merge commits can be very useful though, if the merge commit contains
a description of a multititude of patches.

> 
> There are a lot of good tutorials on simple git workflows, so please
> have a look. If you are using something more complicated, please
> share with us and ideally send us a write up.
> 
> I would like to help with the migration.
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Yes, please! Send us your positive and negative feedback.
> For example:
> - It worked for me!
> - This and that did not work for me...
> - How do I do such and such thing now?
> 
> The test repository is there for you to play with. The contents
> will be deleted before the final migration so no reason to worry about
> potential mistakes. It is also quite likely that the contents will be
> regenerated during the beta testing, to fix any problems found.
> 
> We also need a help documenting possible workflows. Especially when
> preparing a release - we need to test and document how to work with branches
> and releases.

Re: [Valgrind-users] [Valgrind-developers] Migrating Valgrind sources from SVN to GIT

[Ivo R. <iv...@iv...>]

2017-02-26 9:08 GMT+01:00 Austin English <aus...@gm...>:
> On Sun, Feb 26, 2017 at 1:35 AM, Austin English <aus...@gm...> wrote:
> Assuming leaving git broken until the migration is okay (which seems
> fine, given that it currently is and I'm decently sure I'm the only
> person using this ;) ), here's a patch that works for a pure git
> repository.
>
> Please do not apply until the git migration is complete.

Thank you for the patch.
I've added it to the migration recipe:
    https://github.com/ivosh/valgrind-git-migration/commit/69a007175fa0c2186a498a8d55e6fb0319e35456
and pushed it also to the test repo at sourceware.org.

I.