From: Tomohiro M. <to...@cx...> - 2012-09-03 10:57:48
|
Hi all, I've recently came up with a new idea: user extensible packages. The mechanism is similar to user extensible sequences (SB-SEQUENCE). ANSI packages would be instances of STANDARD-PACKAGE class which is a subclass of PACKAGE class. MAKE-PACKAGE, FIND-PACKAGE, INTERN, and others functions around packages will be still functions, but they rather delegate to their generic functions like MAKE-PACKAGE-UNDER, FIND-PACKAGE-UNDER, INTERN-UNDER. Such generic functions will take another argument of the context package. With user extensible packages, we can implement hierarchical packages and locally-renamed packages without changing the compiler. Cool! Isn't it? So, I tried to make SB!XC:PACKAGE located in src/code/package.lisp a class metaobject, but failed to compile SBCL in target-2 stage (SEGV). I don't understand SBCL internals very well. Could someone implement my idea or give me an advice? Regards, Tomohiro Matsuyama |
From: Christophe R. <cs...@ca...> - 2012-09-03 11:36:00
|
Tomohiro Matsuyama <to...@cx...> writes: > Hi all, > > I've recently came up with a new idea: user extensible packages. The > mechanism is similar to user extensible sequences (SB-SEQUENCE). ANSI > packages would be instances of STANDARD-PACKAGE class which is a > subclass of PACKAGE class. > > MAKE-PACKAGE, FIND-PACKAGE, INTERN, and others functions around > packages will be still functions, but they rather delegate to their > generic functions like MAKE-PACKAGE-UNDER, FIND-PACKAGE-UNDER, > INTERN-UNDER. Such generic functions will take another argument of the > context package. > > With user extensible packages, we can implement hierarchical packages > and locally-renamed packages without changing the compiler. Cool! > Isn't it? Yes. (I would like this, and also a similar RANDOM protocol). > So, I tried to make SB!XC:PACKAGE located in src/code/package.lisp a > class metaobject, but failed to compile SBCL in target-2 stage (SEGV). This is basically not going to work straightforwardly, because the cross-compiler has not compiled CLOS (and so it is not available when target-2 starts) -- most of target-2 is in fact compiling CLOS. So everything that happens before has to be expressed in terms of non-CLOS mechanisms. > I don't understand SBCL internals very well. Could someone implement > my idea or give me an advice? The good news is "sort of": a while ago I had a tree which implemented a hack, allowing the user (and the cross-compiler) to define a struct which could later be subclassable by standard objects. This would give a more natural feeling, but it isn't strictly necessary: as long as the functional protocol in the various package functions is in place (with a default implementation for the vanilla standard-package class), it is possible to implement everything using structs without worrying too much about the bootstrapping issues. More concretely, try (defstruct sb!xc:package) (defstruct (standard-package (:constructor internal-make-package) (:inherit sb!xc:package) ...) ;; slots from current sb!xc:package go here ) (defun intern (name package) (etypecase package (standard-package (%standard-intern name package)) (sb!xc:package (sb!package:intern name package)))) then to experiment with different protocols the user can simply define substructs of cl:package (after bootstrapping) and methods on e.g. sb!package:intern for their new structure class. Cheers, Christophe |
From: Tomohiro M. <to...@cx...> - 2012-09-03 15:59:53
|
> Tomohiro Matsuyama <to...@cx...> writes: > > > Hi all, > > > > I've recently came up with a new idea: user extensible packages. The > > mechanism is similar to user extensible sequences (SB-SEQUENCE). ANSI > > packages would be instances of STANDARD-PACKAGE class which is a > > subclass of PACKAGE class. > > > > MAKE-PACKAGE, FIND-PACKAGE, INTERN, and others functions around > > packages will be still functions, but they rather delegate to their > > generic functions like MAKE-PACKAGE-UNDER, FIND-PACKAGE-UNDER, > > INTERN-UNDER. Such generic functions will take another argument of the > > context package. > > > > With user extensible packages, we can implement hierarchical packages > > and locally-renamed packages without changing the compiler. Cool! > > Isn't it? > > Yes. (I would like this, and also a similar RANDOM protocol). > > > So, I tried to make SB!XC:PACKAGE located in src/code/package.lisp a > > class metaobject, but failed to compile SBCL in target-2 stage (SEGV). > > This is basically not going to work straightforwardly, because the > cross-compiler has not compiled CLOS (and so it is not available when > target-2 starts) -- most of target-2 is in fact compiling CLOS. So > everything that happens before has to be expressed in terms of non-CLOS > mechanisms. > > > I don't understand SBCL internals very well. Could someone implement > > my idea or give me an advice? > > The good news is "sort of": a while ago I had a tree which implemented a > hack, allowing the user (and the cross-compiler) to define a struct > which could later be subclassable by standard objects. This would give > a more natural feeling, but it isn't strictly necessary: as long as the > functional protocol in the various package functions is in place (with a > default implementation for the vanilla standard-package class), it is > possible to implement everything using structs without worrying too much > about the bootstrapping issues. Your hack sounds great but at the same time seems to be overkill at least for my purpose. Is it easy to implement the vanilla standard-package class? In my poor understand, I would make a special classoid object for bootstrapping and later CLOS. Could you please explain how if you don't mind? > More concretely, try > > (defstruct sb!xc:package) > (defstruct (standard-package > (:constructor internal-make-package) > (:inherit sb!xc:package) > ...) > ;; slots from current sb!xc:package go here > ) > (defun intern (name package) > (etypecase package > (standard-package (%standard-intern name package)) > (sb!xc:package (sb!package:intern name package)))) > > then to experiment with different protocols the user can simply define > substructs of cl:package (after bootstrapping) and methods on > e.g. sb!package:intern for their new structure class. This is exactly what I want! Tomohiro -- Tomohiro Matsuyama <to...@cx...> |
From: Christophe R. <cs...@ca...> - 2012-09-04 21:59:14
Attachments:
minimal-package-hack.diff
|
Tomohiro Matsuyama <to...@cx...> writes: >> More concretely, try >> >> (defstruct sb!xc:package) >> (defstruct (standard-package >> (:constructor internal-make-package) >> (:inherit sb!xc:package) >> ...) >> ;; slots from current sb!xc:package go here >> ) >> (defun intern (name package) >> (etypecase package >> (standard-package (%standard-intern name package)) >> (sb!xc:package (sb!package:intern name package)))) >> >> then to experiment with different protocols the user can simply define >> substructs of cl:package (after bootstrapping) and methods on >> e.g. sb!package:intern for their new structure class. > This is exactly what I want! OK, so, in the interests of getting my hand back in, here's the absolute minimum patch I could make to allow experimentation to begin. There is almost certainly stuff that is actively broken in this, where there are hard-coded assumptions about the slots of the package structure (you'll see in the diff that I found one or two of those) but I got a very silly package class to work: (defstruct (fooble-package (:include package))) (defmethod sb-package:package-name ((x fooble-package)) "FOO") (defmethod sb-package:find-symbol (x (y fooble-package)) (values nil :external)) and then (find-symbol "WHATEVER" (make-fooble-package)) => NIL, :EXTERNAL. Hopefully that's enough to get you started. Probably the next thing to hook in is something in DEFPACKAGE / MAKE-PACKAGE which can create named packages of different classes. |
From: Tomohiro M. <to...@cx...> - 2012-09-05 04:40:37
|
> OK, so, in the interests of getting my hand back in, here's the absolute > minimum patch I could make to allow experimentation to begin. There is > almost certainly stuff that is actively broken in this, where there are > hard-coded assumptions about the slots of the package structure (you'll > see in the diff that I found one or two of those) but I got a very silly > package class to work: > > (defstruct (fooble-package (:include package))) > (defmethod sb-package:package-name ((x fooble-package)) "FOO") > (defmethod sb-package:find-symbol (x (y fooble-package)) > (values nil :external)) > > and then > > (find-symbol "WHATEVER" (make-fooble-package)) => NIL, :EXTERNAL. > > Hopefully that's enough to get you started. Probably the next thing to > hook in is something in DEFPACKAGE / MAKE-PACKAGE which can create named > packages of different classes. Thank you! I was struggled to make PACKAGE a standard-class metaobject. Now I find out that why I failed. Your approach seems to be not perfect but I think it is a great point to get things started. I made a branch for this project so that other people can experiment the feature. https://github.com/m2ym/sbcl/tree/user-extensible-packages I will continue to work on the branch to make some other functions be able work with your patch. By the way, I'd like to know how difficult it is to change structure classes to standard classes. The following code should work, in the future: (defclass hierarchical-package (standard-package) ()) (defclass locally-renamed-package (standard-package) ()) (defclass my-fancy-package (hierarchical-package locally-renamed-package) ()) Regards, Tomohiro |
From: Christophe R. <cs...@ca...> - 2012-09-05 08:25:34
|
Tomohiro Matsuyama <to...@cx...> writes: > By the way, I'd like to know how difficult it is to change structure classes to standard classes. > The following code should work, in the future: > > (defclass hierarchical-package (standard-package) ()) > (defclass locally-renamed-package (standard-package) ()) > (defclass my-fancy-package (hierarchical-package locally-renamed-package) ()) This is what you'd need the work I have done for allowing a standard class as a subclass of a structure class. It's not impossible, but it's three-year-old work now that was never quite finished. I'd be interested to see your experiments with the protocols. I'd be slightly surprised if an overriding protocol (as implied by inheriting from standard-package rather than package) ends up being the best, but if so then the patch I sent yesterday would need a minor adjustment: rather than testing for standard-package as the class to special-case in the TYPECASEs, you'd need to test for class equality: something like (cond ((eq (class-of package) *the-class-standard-package*) (%standard-find-symbol name package)) (t (sb-package:find-symbol name package))) Best, Christophe |
From: Tomohiro M. <to...@cx...> - 2012-09-15 17:10:51
|
Hi, I'd like to tell you the progress of user-extensible packages project. You can obtain the source code from: https://github.com/m2ym/sbcl/tree/user-extensible-packages Here is the small example of hierarchical packages using user-extensible packages. (in-package :cl-user) (defstruct (hierarchical-package (:include sb-package:standard-package))) (defmethod sb-package:find-package (package-designator (package hierarchical-package)) (flet ((find-package-from-string (string) (when (and (> (length string) 0) (char= (elt string 0) #\.)) (setq string (concatenate 'string (package-name package) string))) (call-next-method string package))) (typecase package-designator (package package-designator) (symbol (find-package-from-string (symbol-name package-designator))) (string (find-package-from-string package-designator)) (character (find-package-from-string (string package-designator))) (t (error 'type-error :datum package-designator :expected-type '(or character package string symbol)))))) (defpackage :foo.bar (:use :cl) (:export #:buzz)) (defun foo.bar:buzz () 42) (make-package :foo :class 'hierarchical-package :use '(:cl)) (in-package :foo) (print (find-package :.bar)) #=> #<SB-IMPL::INTERNAL-STANDARD-PACKAGE "FOO.BAR"> (print (.bar:buzz)) #=> 42 The next step is to make DEFPACKAGE user-extensible packages aware and implement PACKAGE-ITERATOR facility to make DO-SYMBOLS, for instance, user-extensible packages aware. My implementation is straightforwrad: INTERN, for example, just dispatches to %STANDARD-INTERN or SB-PAKCAGE:INTERN with considering of the type of the second argument. However, the protocol might be too generic, i.e. users need to implement an ANSI-compliant package system by their hands if the package system is not inherited from STANDARD-PACKAGE. Plus, even if the package system is inherited from STANDARD-PACKAGE, users sometimes need to implement a whole method like INTERN above. Do you have any idea for the protocol issue? Regards, Tomohiro |
From: Christophe R. <cs...@ca...> - 2012-09-15 17:31:40
|
Tomohiro Matsuyama <to...@cx...> writes: > Here is the small example of hierarchical packages using user-extensible packages. Nice. A minor suggestion: > (in-package :cl-user) > > (defstruct (hierarchical-package (:include sb-package:standard-package))) > > (defmethod sb-package:find-package (package-designator (package hierarchical-package)) I would suggest calling this method "find-package-using-package", and specifying that the effect of cl:find-package is to call find-package-using-package with the current package as its second argument. (The analogy is with mop:ensure-class-using-class -- the current package plays the role in extensible packages of the metaclass in extensible classes. This is interesting, actually, because it's like objects having fluid metaclasses... I want to think about this more.) > My implementation is straightforwrad: INTERN, for example, just > dispatches to %STANDARD-INTERN or SB-PAKCAGE:INTERN with considering > of the type of the second argument. > However, the protocol might be too generic, i.e. users need to > implement an ANSI-compliant package system by their hands if the > package system is not inherited from STANDARD-PACKAGE. Plus, even if > the package system is inherited from STANDARD-PACKAGE, users sometimes > need to implement a whole method like INTERN above. > Do you have any idea for the protocol issue? I think I'd like to see two or three package types and what they look like: what ideas do we have? Something like this, where the current package affects package lookup (maybe also per-package nicknames, and a question as to whether per-package nicknames and hierarchical packages can compose); something like the KEYWORD package, where intern has magical effects... anything else? Cheers, Christophe |
From: Tomohiro M. <to...@cx...> - 2012-09-26 08:56:13
|
> Tomohiro Matsuyama <to...@cx...> writes: > > > Here is the small example of hierarchical packages using user-extensible packages. > > Nice. A minor suggestion: > > > (in-package :cl-user) > > > > (defstruct (hierarchical-package (:include sb-package:standard-package))) > > > > (defmethod sb-package:find-package (package-designator (package hierarchical-package)) > > I would suggest calling this method "find-package-using-package", and > specifying that the effect of cl:find-package is to call > find-package-using-package with the current package as its second > argument. (The analogy is with mop:ensure-class-using-class -- the > current package plays the role in extensible packages of the metaclass > in extensible classes. This is interesting, actually, because it's like > objects having fluid metaclasses... I want to think about this more.) OK. I'll do it. > > My implementation is straightforwrad: INTERN, for example, just > > dispatches to %STANDARD-INTERN or SB-PAKCAGE:INTERN with considering > > of the type of the second argument. > > > However, the protocol might be too generic, i.e. users need to > > implement an ANSI-compliant package system by their hands if the > > package system is not inherited from STANDARD-PACKAGE. Plus, even if > > the package system is inherited from STANDARD-PACKAGE, users sometimes > > need to implement a whole method like INTERN above. > > > Do you have any idea for the protocol issue? > > I think I'd like to see two or three package types and what they look > like: what ideas do we have? Something like this, where the current > package affects package lookup (maybe also per-package nicknames, and a > question as to whether per-package nicknames and hierarchical packages > can compose); something like the KEYWORD package, where intern has > magical effects... anything else? I'd like to have reader-extensible package system like: (defpackage foo (:use :cl) (:set-macro-character #\[ #'my-super-parenthesis)) Using-package this package affects the reader behavior as *readtable* changed. See also named-readtables. Tomohiro |
From: Daniel H. <dhe...@te...> - 2012-09-16 02:13:48
|
On Sat, 15 Sep 2012, Christophe Rhodes wrote: > I think I'd like to see two or three package types and what they look > like: what ideas do we have? Something like this, where the current > package affects package lookup (maybe also per-package nicknames, and a > question as to whether per-package nicknames and hierarchical packages > can compose); something like the KEYWORD package, where intern has > magical effects... anything else? Could this give me the tools needed to specify case-folding/preserving behavior on a per-package basis? For example, I would like CL:defun to be the same as CL:DEFUN, but C++:std.cout to be different than C++:STD.COUT -- all with a single reader. So all symbols in CL will be upcased, and all symbols in C++ will be preserved. Thanks, Daniel |
From: Pascal J. B. <pj...@in...> - 2012-09-16 04:03:10
|
Daniel Herring <dhe...@te...> writes: > On Sat, 15 Sep 2012, Christophe Rhodes wrote: >> I think I'd like to see two or three package types and what they look >> like: what ideas do we have? Something like this, where the current >> package affects package lookup (maybe also per-package nicknames, and a >> question as to whether per-package nicknames and hierarchical packages >> can compose); something like the KEYWORD package, where intern has >> magical effects... anything else? > > Could this give me the tools needed to specify case-folding/preserving > behavior on a per-package basis? > > For example, I would like CL:defun to be the same as CL:DEFUN, but > C++:std.cout to be different than C++:STD.COUT -- all with a single > reader. So all symbols in CL will be upcased, and all symbols in C++ will > be preserved. clisp has the :case-sensitive t option to defpackage and make-package that informs the reader to read symbols in those package case sensitively. http://www.clisp.org/impnotes.html#package-case So: (defpackage :c++ (:use :cl) (:case-sensitive t)) (defvar c++:std.cout nil) (in-package :c++) (setf std.cout (make-cout-stream)) (in-package :cl-user) (find-symbol "STD.COUT" :c++) --> NIL, NIL (find-symbol "std.cout" :c++) --> C++:|std.cout|, :internal -- __Pascal Bourguignon__ http://www.informatimago.com/ A bad day in () is better than a good day in {}. |
From: Tomohiro M. <to...@cx...> - 2012-09-16 14:45:32
|
> On Sat, 15 Sep 2012, Christophe Rhodes wrote: > > I think I'd like to see two or three package types and what they look > > like: what ideas do we have? Something like this, where the current > > package affects package lookup (maybe also per-package nicknames, and a > > question as to whether per-package nicknames and hierarchical packages > > can compose); something like the KEYWORD package, where intern has > > magical effects... anything else? > > Could this give me the tools needed to specify case-folding/preserving > behavior on a per-package basis? > > For example, I would like CL:defun to be the same as CL:DEFUN, but > C++:std.cout to be different than C++:STD.COUT -- all with a single > reader. So all symbols in CL will be upcased, and all symbols in C++ will > be preserved. No, because the symbol characters are normally upcased at read-time. So we can't handle case-folding/preserving behavior in INTERN, for example. Regards, Tomohiro |
From: Philipp M. <ph...@ma...> - 2012-09-16 15:55:42
|
>> For example, I would like CL:defun to be the same as CL:DEFUN, but >> C++:std.cout to be different than C++:STD.COUT -- all with a single >> reader. So all symbols in CL will be upcased, and all symbols in C++ will >> be preserved. > No, because the symbol characters are normally upcased at read-time. So we can't handle > case-folding/preserving behavior in INTERN, for example. Well, if the reader preserves the character-case, INTERN might well choose to do STRING-UPCASE (or whatever's been defined) for a given package. In fact, that might simply be a package option :before-interning - with #'STRING-UPCASE as default. With a case-preserving reader that would give (nearly) the same behaviour as currently, but would allow that feature, too. |
From: Tomohiro M. <to...@cx...> - 2012-09-26 08:45:34
|
> >> For example, I would like CL:defun to be the same as CL:DEFUN, but > >> C++:std.cout to be different than C++:STD.COUT -- all with a single > >> reader. So all symbols in CL will be upcased, and all symbols in C++ will > >> be preserved. > > No, because the symbol characters are normally upcased at read-time. So we can't handle > > case-folding/preserving behavior in INTERN, for example. > Well, if the reader preserves the character-case, INTERN might well choose to do > STRING-UPCASE (or whatever's been defined) for a given package. OK. Now how do you handle a symbol like |foobar| in that method? IMHO, we can't postpone case-folding/preserving dicision without addiing a flag saying the symbol name must be preserved. Tomohiro |
From: Nikodemus S. <nik...@ra...> - 2012-09-17 05:53:51
|
Indeed, a somewhat magical intern would IMO be the easiest way to implement many things people request. Like foo:bar == bar:foo Instead of having two symbols be "the same" in some strange way, simply make the one intern as the other. Cheers, -- nikodemus Sent from my tablet, sorry about top-posting. On Sunday, September 16, 2012, Philipp Marek wrote: > >> For example, I would like CL:defun to be the same as CL:DEFUN, but > >> C++:std.cout to be different than C++:STD.COUT -- all with a single > >> reader. So all symbols in CL will be upcased, and all symbols in C++ > will > >> be preserved. > > No, because the symbol characters are normally upcased at read-time. So > we can't handle > > case-folding/preserving behavior in INTERN, for example. > Well, if the reader preserves the character-case, INTERN might well choose > to do > STRING-UPCASE (or whatever's been defined) for a given package. > > In fact, that might simply be a package option :before-interning - with > #'STRING-UPCASE > as default. With a case-preserving reader that would give (nearly) the > same behaviour as > currently, but would allow that feature, too. > > > > > > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://ad.doubleclick.net/clk;258768047;13503038;j? > http://info.appdynamics.com/FreeJavaPerformanceDownload.html > _______________________________________________ > Sbcl-devel mailing list > Sbc...@li... <javascript:;> > https://lists.sourceforge.net/lists/listinfo/sbcl-devel > -- Cheers, -- Nikodemus |
From: Rudolf S. <ru...@co...> - 2012-09-17 08:31:25
|
On Sep 17, 2012, at 07:53, Nikodemus Siivola <nik...@ra...> wrote: > Indeed, a somewhat magical intern would IMO be the easiest way to implement many things people request. Like > > foo:bar == bar:foo > > Instead of having two symbols be "the same" in some strange way, simply make the one intern as the other. Wouldn't you also need a somewhat magical SYMBOL-NAME (symbol-name-using-package?) to go with it? Rudi |
From: Nikodemus S. <nik...@ra...> - 2012-09-17 12:17:59
|
On Monday, September 17, 2012, Rudolf Schlatte wrote: > > On Sep 17, 2012, at 07:53, Nikodemus Siivola <nik...@ra...<javascript:;>> > wrote: > > > Indeed, a somewhat magical intern would IMO be the easiest way to > implement many things people request. Like > > > > foo:bar == bar:foo > > > > Instead of having two symbols be "the same" in some strange way, simply > make the one intern as the other. > > Wouldn't you also need a somewhat magical SYMBOL-NAME > (symbol-name-using-package?) to go with it? Depends on the spec... INTERN seems like the bare minimum, but SYMBOL-NAME would make a good addition, possibly. - ns > > Rudi > > -- Cheers, -- Nikodemus |