|
From: Nicholas N. <nj...@cs...> - 2007-03-28 03:33:21
|
Hi all,
Julian and I have been discussing a list of Valgrind design and development
goals. We're thinking this would be suitable for putting on the website
somewhere. This started after I looked at GCC's design goals (which are
really vague and thus not much good to anyone), and I realised we didn't
have any written down.
Below is a tentative statement of Valgrind design and development goals
and rationales for discussion. The rationale for each point is
presented in double parenthese, ((like this)).
There are 10 points. Following those are some background comments
on techniques that have been used to achieve these goals.
The idea is not to set anything in stone, but to give an indication of the
relative priorities, which can serve as a useful guide when considering
changes and new features. Comments are welcome.
Nick
------ Highest priority ------
1 Robustness: should be able to correctly run as many programs as
possible on the platforms we support.
((Systems which cannot be relied on to handle the vast majority of
presented workloads soon fall out of favour with users.))
2 Accuracy of outputs: debugging and profiling information generated
by the tools should be sufficiently accurate as to be both useful to
and credible to users. For example, bug detectors should have
minimal false positives.
((Tools which produce unreliable or non-credible results soon fall
out of favour with users.))
3 Design simplicity: the design and implementation should be easy to
understand, maintain, test and verify.
((Our engineering resources are very limited, so the code base
should be structured to make best use of them. Also, a simple code
base is more accessible to newcomers. Simplicity also usually
helps robustness and accuracy (1 and 2).))
4 Instrumentation capabilities: firstly, provide enough capabilities
to keep Memcheck going. Then conservatively add capabilities as required
by other tools.
((We aim for Valgrind to be an effective framework for building
dynamic analysis tools, so it needs to provide instrumentation
capabilities as required by current and emerging tools.))
------ Medium priority ------
5 Performance (speed and memory usage) of heavyweight tools. This
covers the speed of both the Valgrind core and the tool components.
((All else being equal, faster and less space hungry tools are able
to handle larger workloads and so are more useful. Also, users
prefer fast tools.))
6 Usability: it should be possible for users to use the tools without
excessive complication or inconvenience.
((Tools which are difficult or inconvenient to use soon fall out
of favour with users.))
7 New tools: encourage development of new tools as needs and
opportunities develop.
((The needs of users and the general computing landscape changes
slowly over time, and it is important to remain relevant.))
8 Portability: want to avoid platform-specific techniques.
((Platform specific techniques and assumptions are clearly a
hindrance to portablility. We have also found them to sometimes
reduce stability.))
------ Low priority ------
9 Performance of lightweight tools.
((Heavyweight tools (eg. Memcheck) are both harder to construct and
more valuable to users than simple ones (eg, instruction counters,
memory trace generators), and where design choices conflict they have
usually been resolved in favour of supporting heavyweight tools
better.))
10 New platforms: although we want portability, we don't want to
support a lot of platforms, because it's hard work, especially to
do it to a high quality.
((Same as 3: our engineering resources are limited and so we should
focus effort on the most widely used platforms.))
Here are some techniques that in the past have been used to achieve
the above goals. This list does not claim to be complete.
- Valgrind's code representation (IR) favours powerful instrumentation
capabilities. This allows heavyweight tools such as Memcheck to be
built and have reasonable performance. However, it gives poor
performance for lightweight tools such as trace collectors. Such
lightweight tools will have better performance if written in other
DBI frameworks such as Pin or DynamoRIO.
- Valgrind does not use libc or any other library itself. This used
to not be true, but it caused robustness and portability problems.
- Valgrind makes very few assumptions about memory layout. This used
to not be true, but it caused robustness and portability problems.
For example, Memcheck's two-level shadow memory representation means
its shadow memory can be laid out very flexibly, but it is not
particularly fast. A "half-and-half" representation that stores
shadow memory can be faster, but fails on some programs, some Linux
kernel configurations, and is incompatible with other OSes such as
Mac OS X.
- Valgrind used to use x86 segmentation to prevent a client program
from accidentally clobbering Valgrind data. However, this is not
portable to other platforms, and it was removed once support for
other platforms was added. Having such platform-specific features
hinders portability and makes testing harder.
- Valgrind serialises thread execution. For subtle atomicity reasons,
this is necessary for tools (like Memcheck) that use shadow values.
It means they can not use more than one processor at a time on
multiprocessor machines.
- Assertions and sanity checkers. The code base contains a large
number of assertions. Additionally, many subsystems have sanity
check code which periodically checks important data structures in
detail. In some cases these checkers are permanently enabled, even
at the cost of some performance. These include: the IR intermediate
representation, the address space manager (m_aspacemgr), the
translation table manager (m_transtab), some parts of Memcheck. One
side effect is that Valgrind almost never segfaults - instead if it
fails it does so by failing one of these assertions or sanity
checks.
|