Say foo() calls bar(), but foo()'s result does not depend
upon bar()'s result. There is no dependence for foo() on
the function body of bar(). So, if bar() is changed, foo()
can still cache hit. However, a fatal bug may be
introduced into bar(). Foo()'s cache hit would hide this
fatal bug. So a developer could introduce a bug, but still
build successfully. The bug would only surface once
the cache is flushed (likely after the developer released
a seemingly functional build).
Fundamentally, the issue is that the fatality of a function
is not considered to be part of its result. And the cache
does not record dependencies upon a function's fatality.
Ken has prepared a reproducer, included inline below.
It might at first appear sufficient to record dependencies
upon all function bodies that are called, regardless of
any dependency upon their results, but, as Ken pointed
out, this is not sufficient. The secondary inputs of foo()
can affect the code flow, and therefore, whether bar() is
called. A cache entry can be created for a foo() call
which does not call bar(), and has no knowledge that bar
() has any potential to be called. There is no easy way
to record the dependence on the secondary input which
affected the code flow.
One solution might be to defer the fatality of a function.
Return a "fatal" indication as the function result, and
propagate this through the evaluation. Only if the build
result depends on the failing function's return value
would the build fail. Unfortunately, this approach is
nearly impossible to implement. The evaluator's
determination of what the build depends on is
conservative, meaning it might record more
dependencies than are necessary or a courser-grained
dependency than necessary. This would leave open the
possibility that a call of foo() which does call bar()
is "conservatively" detected as being dependent on bar
(), and therefore can be fatal based on bar(), when it is
not truly dependent. In this case, a cache entry could
exist for a foo() call which does not call bar() and hits for
this falsely-dependent-and-therefore-fatal build. So, the
evaluator's dependence analysis cannot be used to
determine the fatality of a bug. (At least that's the
conclusion I got to.)
It would seem the only feasible solution is to remove
fatality from the build process altogether. This means
not just failing _run_tool() invocations, but any SDL
errors for a function as well. This would not exactly be
a backward compatible change, but it would only
change the behavior of previously-failing builds. This is
probably not a real problem. SDL could check the
return values for failure (ERR?), but special-casing
failure is not required. Even if not explicitly checked,
the error would be reported to stderr. If the SDL is not
written to be resilient to errors, this error would likely
lead to many others downstream. As with many
compilers/interpreters, only the first bug is guaranteed
to be meaningful, but it would be a potential benefit to
see multiple bugs from a single evaluation. (Or the
evaluator could hide these.) The additional stderr output
would be clutter, but existing error reports have a clutter
issue already. It would be good for the evaluator to flag
the error and disable subsequent generation of cache
entries. This would enable regeneration of the error
output. Also, a flag to make errors fatal would be fine
too -- wait, this gets us right back where we started --
yes, but now there is a non-fatal mode of operation that
is guaranteed to produce a reproducible result.
So, maybe all we need is to add a "-non-fatal" flag that
will, on previously-fatal-error, disable caching, and return
out of the enclosing function with an ERR return value.
Builds without this flag would have the reproducibility
issue, but with this flag would not. And no current
behavior would be affected. A project must consider a
build "good", independent of the build's fatality without -
non-fatal, as it is not possible to prevent these "hidden"
errors from sneaking in. Any rebuilds of successful
builds, as well as integration builds (which merge
successful builds), should use -non-fatal. General
building for development might or might not use this
flag, but code should not be released that has visible
errors. Looking at errors from things like integration
builds is still useful. Unfortunately, it would be difficult
to distinguish hidden errors from those released by
someone who did not follow proper procedures, so there
could still be some misdirected blame. And this very
complex issue is exposed to the user through the -non-
fatal flag. But it seems to be our best option.
Note: The -k option to "vesta" controls fatality of
_=_print("good function called");
// Note: the function has to be passed in a complex
// like a binding so it doesn't become part of the PK of
// Return something which doesn't depend on the
function so we
// don't record any dependency on it.
return <call_but_dont_use([f=good]), // This
populates the cache.
call_but_dont_use([f="This isn't even a function!"])>;
// This would fail, but it is instead a cache hit!!!
Log in to post a comment.