Re: [lisp-snmp] Patch to asn.1 to change from parsergen to a modified cl-yacc

[John F. <jf...@ms...>]

Dear Chun Tian,

"Chun Tian (binghe)" <bin...@gm...> writes:
[...]
> |1) For the question about nicknames of generated packages, I think my
> current solution is the best, my ideas on this design:
>
> On ASN.1 package trunk, the ASN.1 "module" is directly mapped into
> Lisp "package", this change let us could have two OID instances with
> the same name but in different modules. One example is
> "netSnmpAgentOIDs.linux" (in NET-SNMP-TC) and "ucdSnmpAgent.linux" (in
> UCD-SNMP-MIB). A new OID function syntax has been introduced:
>
> ASN.1 47 > (oid "NET-SNMP-TC::linux")
> #<OBJECT-ID NET-SNMP-TC::linux (10) [0]>

The problem with this design is that the MIB module name is made
directly into a Lisp package.

This means that you cannot safely have a MIB module named COMMON-LISP or
CFFI or PPCRE or anything which is already a Lisp package. This is
particularly important when the end-user is able to load in arbitrary
MIB files.

That's why the "asn.1/" prefix to generated package names is a good
idea. It is already there. Just drop the nickname and we are fine.

To support the cool (oid ..) syntax you can easily add "asn.1/" to the
start of the string processed by OID method.

[...]

> Due to this design, the implementation of OID function is very easy,
> and highly depend on what Common Lisp already have, and it's delivery
> friendly (need only symbols but the CL compiler/interpreter). So the
> "short" package name is necessary, no matter if they may clash with
> other CL packages. (Actually the clash chance is very small, because
> most formally defined MIB module has a "MIB" as part of its module
> name).

Yes, it is fine for most MIBs. But suppose the end-user (our customer)
loads in an MIB called SYSTEM, it would cause all sorts of mayhem in
the SYSTEM package.

[...]

> |2) For the "EVAL" issue and your related questions. I think, the only
> issue you have solved is to use cl-yacc instead of lispworks'
> parsergen, then all supported CL platform can use the ASN.1 compiler.
>
> I don't quite like the use of function EVAL at runtime, because this
> will cause some delivery issues. I don't know much about Allegro CL,
> but to support EVAL on runtime in delivered LispWorks images, the
> whole Common Lisp runtime and interpreter will be hold in target
> executions, this may increase the file size. And EVAL itself also
> suffer from environment-related problems, one of which is the symbol
> intern issues you have mentioned.

The symbol intern issues can easily be fixed, and it is wrong for extra
symbols to be interned in the ASN.1 package whether or not we use eval.

> The better way to load MIB definitions at runtime may be to use
> COMPILE-FILE and LOAD instead of EVAL, if a new lisp file is generated
> from the ASN.1 definitions, we should be able to COMPILE-FILE it
> first, and then LOAD it.

Compile file is more tricky than eval because it requires the compiler.

[...]

> The best way to load MIB definitions at runtime, I think, is to
> compile the ASN.1 definitions into another form: the AST itself or a
> modified version. Then we can use a pure-lisp ASN.1 "interpreter" to
> read this ACT, and create correspond OID variables at runtime. This is
> a delivery-free solution, we only need something like MAKE-INSTANCE,
> and DEFCLASS sometimes.
>
> What your opinion here?

I think that is extra work. I would not be against it if you did it but
I don't think it is a good use of my time. In our images we keep eval,
but not the compiler, so eval is a good option for us and compile-file
is a very bad option.

>
> -----
> |3) I'd very like your functions to convert LispWorks PARSERGEN syntax
> into CL-YACC syntax.
>
> The reason to keep PARSERGEN: The ASN.1 compiler based on LispWorks'
> parsergen must be faster than those based on CL-YACC, and on LispWorks
> the CL-YACC package is not needed.

Are you sure? I don't think that the speed is important and I am not
sure that there will be a big difference in speed as they use very
similar algorithms.

> PARSERGEN will compile LALR(1) syntax into a series of Lisp functions
> which do the actual parsing job, these functions will be compiled and
> saved into syntax.fasl and on runtime they can do things very fast (no
> depend on these LALR definitions any more). Compare to that, CL-YACC
> is NOT a LALR compiler, it just compile LALR syntax definitons into a
> private-format "parser" object, and use it on runtime, this method
> should be slower than LispWorks' primitive parser functions.

Are you sure?

LALR is just about building a table. Does Lispworks really compile down
the table into separate functions? I doubt it.

Why is the speed of loading in MIB files important anyway? I think that
cl-yacc is sufficiently fast and the extra complexity of trying to
maintain separate identical grammars is just going to cause trouble down
the line.

[...]