[Sbcl-devel] error reporting, mostly (was Re: changes to list.lisp)

[William H. N. <wil...@ai...>]

On Sat, Feb 01, 2003 at 03:05:55PM -0600, Robert E. Brown wrote:
> Thanks for taking a look at the patch for list.lisp.  I did not know about
> %COERCE-CALLABLE-TO-FUN, but it sounds like the right thing.  If you'd like
> me to submit a better patch, just let me know.

You don't need to send me a new patch unless you want to make some
changes other than those we've discussed. If you're happy with the
approach I plan to take, I don't mind just working from what I have.

> While on the topic of compiler notes, let me say that I'm generally a bit
> bothered by how verbose Python is while compiling SBCL.  Let's say I want to
> use SBCL for a big project with several dozen programmers.  I've never had
> the opportunity to do this with a Lisp system, but with C or C++, it's
> reasonable to set up automatic nightly compilation of all the code.  When
> this is done, errors and warnings in the nightly compiles result in email to
> those who maintain or last changed a chunk of code.  In this sort of
> environment everyone tries to make the compiler generate as few messages as
> possible, since they tend to obscure warnings that indicate real problems.

It is verbose. The closest thing to a saving grace is that errors and
warnings and style warnings are all summarized separately and
automatically in the return values from COMPILE-FILE, so some of the
reporting you describe can be implemented without much trouble. Using
this mechanism, SBCL's build already stops on (full, not just STYLE-)
WARNINGs, except for the few specific :IGNORE-FAILURE-P workarounds
used in build-order.lisp-expr to work around problems we haven't been
able to fix yet.

STYLE-WARNINGs are hard to completely avoid, because it's reasonable
to emit them for things like references to not-defined-yet functions,
but valid code can intentionally have references to not-defined-yet
functions,. I don't know how to fix that better without new design
insights.
 
Even full WARNINGs have some design issues. Usually we'd like type
mismatches detectable at compile time to cause full WARNINGs. But then
what about code which is unreachable? In macroexpansions or other
automatically generated code, it can be very natural to create
provably unreachable code which would cause type errors if it were
reached. Python can prove some easy cases unreachable, but of course
it's not feasible to make the compiler find all such proofs for
itself, so for sufficiently difficult cases you get full WARNINGs, in
which COMPILE and COMPILE-FILE return FAILURE-P true for perfectly
safe and valid code...

Note that even C seems to have some issues along these lines. What
we're doing in the GC (e.g. gencgc.c) is only semi-portable, but it is
meaningful and basically correct. While the GC code compiles with
fewer warnings than it did years ago, because I and other maintainers
(especially dan_b in this case) basically agree with you that compiler
warnings are to be eliminated, it still causes many warnings,
especially
   warning: decimal constant is so large that it is unsigned
That's sort of a reasonable warning, but it would be nice to have some
way to avoid it when we're doing it intentionally and correctly.
Unfortunately, I've never found even a gcc-specific way of avoiding
it, and I'd be surprised to find there's a portable way of avoiding it.

So while this topic is one that we take seriously, we seem to have
more hard problems than easy answers.

All that said, there are also some cases where we do have relatively
easy answers, we just haven't implemented them yet. For example, SBCL
gives STYLE-WARNINGs when redefining functions and methods, which is a
pain when you reload a file. We should probably find some way to stop
the bogus redefinition warnings in that case, and it's probably not
too hard. One possibility is remembering the location of the original
definition, and not warning when the new definition is the same place
in the same file as the old definition.

Incidentally, current SBCL is likely to be an imperfect implementation
for monolithic programs at the several-dozen-programmer scale in more
ways than this. We work on what we're motivated to work on, and so far
none of the regular contributors have been, as far as I can see, very
motivated by problems at that scale. Thus, for example, you probably
wouldn't be very happy with our current 32-bit address space
limitations. And even for our much-smaller projects (such as SBCL
itself) none of us is particularly happy about the compile time.

Also incidentally, for such a large project as you describe, I for one
would probably not use *any* Common Lisp implementation in the role
than you envisage. A single monolithic program of millions of lines
can be done in CL, but statically linked and
more-closely-statically-checked languages have some advantages which
grow with size. CL might still be useful in other ways of attacking
such a big problem, e.g. by using CL to implement a specialized
language in which the rest of the project is done, or by using CL as a
a soft customization layer atop a monolithic program. But I'd try hard
to avoid making such a large monolithic program at all (although I
concede that sometimes it can be a natural choice); and then if even
after trying hard to avoid it I found I couldn't avoid it, I'd
probably look for another language to do it in.

> Also, in order to analyze the compiler's output, you want the warnings and
> error messages in a format that's easy for software to parse.  Hence, the
> compiler should have a mode where it outputs warnings and errors as
> s-expressions.  This makes it much easier to track changes in compiler
> output and match them up with code checkins.

I can see that such a feature could be useful. I put something similar
into clocc ansi-test to track known bugs, for similar reasons.
However, for dealing with compilation warnings, I'd be more
comfortable with an approach which let you add declarations to the
code itself to tell the compiler about things so that it doesn't warn
you, and so that other programmers can see it too. Unfortunately
standard CL declaration technology doesn't cover everything, as above.
More design work for somebody, maybe. (Proof-carrying code?)
 
> Anyway, I guess I expect to see no more than a handful of compiler warning
> messages, even when compiling a system much bigger than SBCL.  That's why
> I'm happy to learn about Common Lisp and the internals of SBCL while
> tracking down compiler notes that I feel are easily gotten rid of.

I'm generally interested in cleanups, and there seems to've been
enough self-selection in the rest of the SBCL developer and user
community that they tend to be interested in them too.

-- 
William Harold Newman <wil...@ai...>
Furious activity is no substitute for understanding. -- H. H. Williams
PGP key fingerprint 85 CE 1C BA 79 8D 51 8C  B9 25 FB EE E0 C3 E5 7C