Re: [lisp-snmp] Patch to asn.1 to change from parsergen to a modified cl-yacc

[Chun T. (binghe) <bin...@gm...>]

Hi, John

On 2009-2-9, at 13:56, John Fremlin wrote:

> Dear Chun Tian,
>
> "Chun Tian (binghe)" <bin...@gm...> writes:
> [...]
>> |1) For the question about nicknames of generated packages, I think  
>> my
>> current solution is the best, my ideas on this design:
>>
>> On ASN.1 package trunk, the ASN.1 "module" is directly mapped into
>> Lisp "package", this change let us could have two OID instances with
>> the same name but in different modules. One example is
>> "netSnmpAgentOIDs.linux" (in NET-SNMP-TC) and  
>> "ucdSnmpAgent.linux" (in
>> UCD-SNMP-MIB). A new OID function syntax has been introduced:
>>
>> ASN.1 47 > (oid "NET-SNMP-TC::linux")
>> #<OBJECT-ID NET-SNMP-TC::linux (10) [0]>
>
> The problem with this design is that the MIB module name is made
> directly into a Lisp package.
>
> This means that you cannot safely have a MIB module named COMMON- 
> LISP or
> CFFI or PPCRE or anything which is already a Lisp package. This is
> particularly important when the end-user is able to load in arbitrary
> MIB files.
>
> That's why the "asn.1/" prefix to generated package names is a good
> idea. It is already there. Just drop the nickname and we are fine.
>
> To support the cool (oid ..) syntax you can easily add "asn.1/" to the
> start of the string processed by OID method.

It's not so easy when you've found following syntax is also well  
supported:

ASN.1 10 > (oid "SNMPv2-SMI::mib-2.SNMPv2-MIB::system")
#<OBJECT-ID SNMPv2-MIB::system (1) [9]>

>
>
> [...]
>
>> Due to this design, the implementation of OID function is very easy,
>> and highly depend on what Common Lisp already have, and it's delivery
>> friendly (need only symbols but the CL compiler/interpreter). So the
>> "short" package name is necessary, no matter if they may clash with
>> other CL packages. (Actually the clash chance is very small, because
>> most formally defined MIB module has a "MIB" as part of its module
>> name).
>
> Yes, it is fine for most MIBs. But suppose the end-user (our customer)
> loads in an MIB called SYSTEM, it would cause all sorts of mayhem in
> the SYSTEM package.

Well, contact me if you really found your customers use such a MIB  
module name, I think we can talk about this issue later:)

> [...]
>
>> The best way to load MIB definitions at runtime, I think, is to
>> compile the ASN.1 definitions into another form: the AST itself or a
>> modified version. Then we can use a pure-lisp ASN.1 "interpreter" to
>> read this ACT, and create correspond OID variables at runtime. This  
>> is
>> a delivery-free solution, we only need something like MAKE-INSTANCE,
>> and DEFCLASS sometimes.
>>
>> What your opinion here?
>
> I think that is extra work. I would not be against it if you did it  
> but
> I don't think it is a good use of my time. In our images we keep eval,
> but not the compiler, so eval is a good option for us and compile-file
> is a very bad option.

OK, I'll do the "extra" work, to make my packages perfect.

>
>
>>
>> -----
>> |3) I'd very like your functions to convert LispWorks PARSERGEN  
>> syntax
>> into CL-YACC syntax.
>>
>> The reason to keep PARSERGEN: The ASN.1 compiler based on LispWorks'
>> parsergen must be faster than those based on CL-YACC, and on  
>> LispWorks
>> the CL-YACC package is not needed.
>
> Are you sure? I don't think that the speed is important and I am not
> sure that there will be a big difference in speed as they use very
> similar algorithms.

You're right. After actual test, no big performance difference between  
cl-yacc and parsergen on lispworks, but I have other reasons to keep  
the current way:

>
>
>> PARSERGEN will compile LALR(1) syntax into a series of Lisp functions
>> which do the actual parsing job, these functions will be compiled and
>> saved into syntax.fasl and on runtime they can do things very fast  
>> (no
>> depend on these LALR definitions any more). Compare to that, CL-YACC
>> is NOT a LALR compiler, it just compile LALR syntax definitons into a
>> private-format "parser" object, and use it on runtime, this method
>> should be slower than LispWorks' primitive parser functions.
>
> Are you sure?
>
> LALR is just about building a table. Does Lispworks really compile  
> down
> the table into separate functions? I doubt it.

It's true. Both ZEBU and PARSERGEN will compile parsers into separate  
parser functions:

ASN.1 13 > (apropos "ASN.1-PARSER")
ASN.1-PARSER-ACTION103 (defined)
ASN.1-PARSER-ACTION42 (defined)
ASN.1-PARSER-ACTION93 (defined)
ASN.1-PARSER-ACTION61 (defined)
ASN.1-PARSER-ACTION6 (defined)
ASN.1-PARSER (defined)
ASN.1-PARSER-ACTION12 (defined)
ASN.1-PARSER-ACTION13 (defined)
ASN.1-PARSER-ACTION20 (defined)
ASN.1-PARSER-ACTION11 (defined)
ASN.1-PARSER-ACTION26 (defined)
ASN.1-PARSER-ACTION37 (defined)
ASN.1-PARSER-ACTION34 (defined)
ASN.1-PARSER-ACTION55 (defined)
ASN.1-PARSER-ACTION59 (defined)
ASN.1-PARSER-ACTION0 (defined)
ASN.1-PARSER-ACTION100 (defined)
ASN.1-PARSER-ACTION101 (defined)
ASN.1-PARSER-ACTION41 (defined)
ASN.1-PARSER-ACTION4 (defined)
ASN.1-PARSER-ACTION131 (defined)
ASN.1-PARSER-ACTION32 (defined)
ASN.1-PARSER-ACTION53 (defined)
ASN.1-PARSER-ACTION31 (defined)
ASN.1-PARSER-ACTION86 (defined)
ASN.1-PARSER-ACTION94 (defined)
ASN.1-PARSER-ACTION95 (defined)
ASN.1-PARSER-ACTION81 (defined)
ASN.1-PARSER-ACTION9 (defined)
ASN.1-PARSER-ACTION104 (defined)
ASN.1-PARSER-ACTION119 (defined)
ASN.1-PARSER-ACTION22 (defined)
ASN.1-PARSER-ACTION33 (defined)
ASN.1-PARSER-ACTION80 (defined)
ASN.1-PARSER-ACTION76 (defined)
ASN.1-PARSER-ACTION54 (defined)
ASN.1-PARSER-ACTION75 (defined)
ASN.1-PARSER-ACTION79 (defined)
ASN.1-PARSER-ACTION2 (defined)
ASN.1-PARSER-ACTION62 (defined)
ASN.1-PARSER-ACTION71 (defined)
ASN.1-PARSER-ACTION7 (defined)
ASN.1-PARSER-ACTION60 (defined)
ASN.1-PARSER-ACTION21 (defined)
ASN.1-PARSER-ACTION77 (defined)
ASN.1-PARSER-ACTION44 (defined)
ASN.1-PARSER-ACTION78 (defined)
ASN.1-PARSER-ACTION69 (defined)
ASN.1-PARSER-ACTION1 (defined)

>
>
> Why is the speed of loading in MIB files important anyway? I think  
> that
> cl-yacc is sufficiently fast and the extra complexity of trying to
> maintain separate identical grammars is just going to cause trouble  
> down
> the line.

Well, I think I'm not maintaining two grammars, just one, and your  
translation function from parsergen to cl-yacc is excellent. I want to  
keep this way since it works well. I'm a LispWorks customer and love  
it. It's my responsibility to give LispWorks users best performance  
and minimum dependency when using my packages.

--
Chun Tian (binghe)
NetEase.com, Inc.
P. R. China