[Vesta-devel] [ vesta-Feature Requests-1121926 ] Evaluator dependency consistency checking

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 454-5900

Feature Requests item #1121926, was opened at 2005-02-13 14:13
Message generated for change (Comment added) made by xorian
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=410430&aid=1121926&group_id=34164

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Evaluator
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: Kenneth C. Schalk (xorian)
Assigned to: Nobody/Anonymous (nobody)
Summary: Evaluator dependency consistency checking

Initial Comment:
It's currently difficult to check the correctness of a
change to the evaluator's dependency analysis code. 
Since we're currently planning a major overhaul of the
data structures used by that code, it would be good to
provide some help with this.

Yuan, Irina, and I discussed one idea earlier this
week: When the evaluator gets a cache hit, save the
value returned by the cache and evaluate the function
anyway.  Compare the results (or at least their
dependency sets).  If they're different, tell the user
exactly how they differ.

It would probably be fine to make this feature
something that's compiled out by default (like the
DumpVal function I added recently).  it would probably
be something only be useful when modifying the
evaluator.  The cache lookups would have to be done
with the "old" PKs (if changing the PK seeds in
evaluatorRunToolVersion and evaluatorFunctionVersion).

----------------------------------------------------------------------

>Comment By: Kenneth C. Schalk (xorian)
Date: 2006-11-09 18:05

Message:
Logged In: YES 
user_id=304837

I have made progress on this and plan on making it official
shortly.  Here's the current version in the branch I've been
working on:

/vesta/vestasys.org/vesta/eval/84.dep_check/4

Regarding the two types of unexpected differences I
mentioned in my previous update:

1. The differences across pickling/unpickling seem to come
from values which have a name in the calling context are
stored and retrieved.  If you have a function like this:

  f() { return ./x; }

Then the pickled value stored in the cache doesn't actually
include the contents of ./x.  Instead it is simply a record
that the value can be gotten from ./x in the calling
context.  The unpickling code will look up this name and
copy the value.  The unpickling code was calling Valc::Copy
with more=true, which causes it to copy the "lenDps" of a
ListVC or BindingVC.  However, this isn't necessary when a
value has a path.  Simply changing the unpickling code to
pass more=false when copying such a value made the
difference disappear.

2. The "N/nm" dependencies in the run_both function of the
mtex bridge which seem to randomly appear and disappear
seems to be a buglet that was lying dormant.  I disabled the
use of the various changes to the dependency analysis code
which we had made, and the -dependency-check output still
showed those differences.  (In other words, it was always
there but -dependency-check allowed us to see it for the
first time.)  While this deserves further investigation,
it's not a reason to hold back the changes to the dependency
analysis code.

----------------------------------------------------------------------

Comment By: Kenneth C. Schalk (xorian)
Date: 2005-06-27 21:20

Message:
Logged In: YES 
user_id=304837

I've re-worked the dependency comparison code to perform a
more extensive comparison in:

/vesta/vestasys.org/vesta/eval/56.dep_tree/9

Unfortunately, this seems to be revealing obscure
differences which point to potential bugs.  I was hoping to
use this to validate some later changes to the evaluator,
but instead it seems to be showing differences between the
cache entries created by eval/56 and eval/56.dep_tree/9.
I'm still investigating, and it's not yet clear to me
whether these differences are due to the changes in the
eval/56.dep_tree branch or bugs that have been lurking prior
to that.

One kind of difference I have observed is an apparently
random addition or deletion of a dependence on the value of
the "nm" parameter to the "run_both" function in the mtex
bridge:

----- Dependency Differences In -----: 
/vesta/vestasys.org/bridges/mtex/1/build.ves/run_both(),
line 25, col 13
Cache Hit PK: 90d6d99c78f2fdbc 284f008bc9e2e439
Cache Index: 13265

result[1]->dps :

+  0. N/nm : "VCacheMonitor.1.mtex"

----- Dependency Differences In -----: 
/vesta/vestasys.org/bridges/mtex/1/build.ves/run_both(),
line 25, col 13
Cache Hit PK: 833280d2d4443237 f6ce9f37e3e082d4
Cache Index: 13261

result[1]->dps :

-  0. N/nm : "FlushCache.1.mtex"

Both of these indicate a difference in the dependencies on
the second element of the list returned by the function.  In
the first case, the dependence on nm was present in the
re-evaluation but not in the cache entry.  In the second,
the reverse is true.

Another kind of difference I've observed is an apparent
change by the process of pickling and unpickling a result
value.  The dependency comparison code actually pickles and
then unpickles the new value before comparing to the old
cached value, to get something close to the process that the
value from a cache hit would have gone through.  While
working on eval/56.dep_tree/9, I decided to try comparing
the dependencies of the value before pickling to those of
the value after unpickling.  Surprisingly, I found some
small differences:

----- Dependency Differences In -----: 
/vesta/vestasys.org/bridges/c_like/0/build.ves() (special)
Cache Hit PK: 37e613486b914599 3f25ad1185c11c02
Cache Index: 540

pickle/C/program/root->lenDps :

+  0. N/components/glibc-kernheaders/root/root : [ ... ]

pickle/C/shared_object/root->lenDps :

+  0. N/components/glibc-kernheaders/root/root : [ ... ]

pickle/C/expert/root->lenDps :

+  0. N/components/glibc-kernheaders/root/root : [ ... ]

These indicate differences in the length dependency set for
the variable "root" captured from the definition context of
the functions C/program, C/shared_object, and C/expert.

----------------------------------------------------------------------

Comment By: Kenneth C. Schalk (xorian)
Date: 2005-06-24 17:36

Message:
Logged In: YES 
user_id=304837

Irina Furman implemented a version of this a while ago.  You
can find it in:

/vesta/vestasys.org/vesta/eval/56.dep_tree/3

I've been doing some further work (mostly minor improvements
and fixes) in that branch.  However, in testing it, I've
decided that the current code provides insufficient
information, and may not even do enough checking.

Currently, only the dps of the result values are compared.
If the result of a function is a composite value, the
dependency comparison should really recurse through its
sub-values and compare the dps, lenDps, and path of each.
Those become part of the secondary key for caching purposes.
When modifying the evaluator, subtle differences could come
up in any of them.

I came to this conclusion because in testing these changes I
ran into a case where I was about to do exactly this kind of
detailed comparison by hand.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=410430&aid=1121926&group_id=34164