Re: [Valgrind-users] Relational interface to Valgrind tools and technical support question

[Philippe W. <phi...@sk...>]

On Sat, 2014-01-04 at 14:14 +0200, Marios Fragkoulis wrote:


> As part of my PhD research I am developing an interactive SQL
> relational
> interface to Valgrind tools so that tool end users can execute SQL
> queries
> against tool's data structures [1]. The query library currently
> supports Memcheck,
> Cachegrind, and Callgrind.
...
> 1. As I am currently at the point where I am evaluating the usefulness
> of such an
> interface, I was wondering whether you can think of specific use cases
> from your experience with Valgrind?
I once had to investigate which piece of guest code was creating a
lot of partially defined bytes, so do a kind of 
  select count(*), stacktrace/origin from partially_defined_bytes
  group by stacktrace/origin
 /// this is invalid sql :)
as the partially defined bytes were taking a lot of memory in valgrind
data structures.

So, this sql interface could possibly have been used.
No idea of other use cases at this moment.

Note that an alternative implementation would be to have valgrind
outputting a file (e.g. csv) to be loaded in a real relational db.

I guess that the advantage of your current implementation is that
it could be used interactively during the run (not only at the end)
and so the sql interface might then possibly be used to enrich
valgrind during run (e.g. to implement more sophisticated error types).
All this is still vague to me however, if you have already some use
cases, that might give some light about what it could be used for
(and we could discuss all that at the valgrind BoF in valgrind
devroom in fosdem2014 
   https://fosdem.org/2014/schedule/track/valgrind/

You could send a small description/few slides to
the BoF coordinator with the use cases you have already seen.
(will you go to FOSDEM ?)

> 
> 
> 2. For the query shown above, I am getting a couple of unintuitive
> results when
> instrumenting egrep with Memcheck on Linux Debian - kernel 2.6.32.
> While a specific byte's VAbits shows that it is part defined, yet the
> respective Vbits is
> V_BITS8_UNDEFINED. Am I doing sth wrong? Below is the code used to
> retrieve
> this information (please bear with me :-) ). It follows closely
> Memcheck's code so it should look
> familiar.

The secondary table entries are maintaining the V bits for the partially
defined bytes (pdb). These entries can become 'stale' : if a pdb is
overwritten by a fully defined or fully undefined byte, the entry
in the sec table is not cleaned up (for speed reason).
Instead, from time to time, the sec v bit table is garbage collected
(see gcSecVBitTable).

However, that can only explain VAbits telling fully defined/undefined
and Vbits giving partially defined bytes.

But having a Vbit secondary entry indicating fully undefined 
for VAbits telling partially undefined looks like a bug to me.
However, such inconsistency is very likely to be detected by
Valgrind self checks (see the asserts in get_sec_vbits8 and
set_sec_vbits8).

Maybe you could check  using vgdb monitor command get_vbits what
the "original memcheck data structure" gives, compared to the
sql interface ?

Or maybe the case you describe is rather fully defined or undefined
in VAbits, and partially defined in Vbits ? Then this is normal.

Philippe

> 
> 
> static short getVAbits(Addr base) {
>   UChar vabits = -1;
>   if (inPrim(base)) {
>     UWord pm_off = base >> 16;
>     UWord sm_off = SM_OFF(base);
> #if VG_DEBUG_MEMORY >= 1
>     tl_assert(pm_off < N_PRIMARY_MAP);
> #endif
>     SecMap *sm = primary_map[ pm_off ];
>     vabits = sm->vabits8[ sm_off ];
>   } else {
>     AuxMapEnt  key;
>     AuxMapEnt* res;
>     Word       i;
>     tl_assert(base > MAX_PRIMARY_ADDRESS);
>     base &= ~(Addr)0xFFFF;
>     for (i = 0; i < N_AUXMAP_L1; i++) {
>       if (aux_primary_L1_map[i].base == base) {
>         break;
>       }
>     }
>     if (i < N_AUXMAP_L1) {
>       res = aux_primary_L1_map[i].ent;
>     } else {
>       key.base = base;
>       key.sm   = 0;
>       res = VG_(OSetGen_Lookup)(aux_primary_L2_map, &key);
>     }
>     if ((res) && (res->sm) && (res->sm !=
> &distinguished_sec_map[SM_DIST_NOACCESS])) {
>       UWord sm_off = SM_OFF(base);
>       vabits = res->sm->vabits8[ sm_off ];
>     }
>   }
>   return vabits;
> };
> 
> 
> static short extract_vabits2(Addr base, UChar vabits8) {
>    UInt shift = (base & 3) << 1;          // shift by 0, 2, 4, or 6
>    vabits8 >>= shift;                     // shift the two bits to the
> bottom
>    return 0x3 & vabits8;                  // mask out the rest
> };
> 
> 
> static short getVAbits2(Addr base, int indexB) {    // indexB possible
> values [0-3]
>   UChar vabits8 = getVAbits(base);
>   return extract_vabits2(base + indexB, vabits8);
> };
> 
> 
> static UWord getVbits8(Addr base, int indexB) {
>   UChar vabits2 = getVAbits2(base, indexB);
>   if (vabits2 == VA_BITS2_PARTDEFINED) {
>     Addr aAligned = VG_ROUNDDN(base, BYTES_PER_SEC_VBIT_NODE);
>     Int amod     = base % BYTES_PER_SEC_VBIT_NODE;
>     SecVBitNode* n = VG_(OSetGen_Lookup)(sec_vbit_table, &aAligned);
>     UChar vbits8;
>     tl_assert2(n, "get_sec_vbits8: no node for address %p (%p)\n",
> aAligned, base);
>     /* Shouldn't be fully defined or fully undefined -- those cases
> shouldn't
>      * make it to the secondary V bits table.
>      */
>     vbits8 = n->vbits8[amod];
>     tl_assert(V_BITS8_DEFINED != vbits8 && V_BITS8_UNDEFINED !=
> vbits8);
>     return vbits8;
>   }
>   return -1;
> };
> 
> 
> Best regards,
> Marios Fragkoulis
> PhD researcher
> Department of Management Science and Technology
> Athens University of Economics and Business
> http://istlab.dmst.aueb.gr/content/members/m_mfrag.html
> https://github.com/mfragkoulis
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT 
> organizations don't have a clear picture of how application performance 
> affects their revenue. With AppDynamics, you get 100% visibility into your 
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> _______________________________________________ Valgrind-users mailing list Val...@li... https://lists.sourceforge.net/lists/listinfo/valgrind-users