cclan-list — Discussions of all things CCLAN

[cclan-list] asdf patch: clisp has mop

[Sam S. <sd...@gn...>]

clisp has full MOP (since 2005), including change-class (since 2003) & method 
combinations (since 2004).

--- asdf.lisp.~1.130.~	2008-10-15 14:56:08.000000000 -0400
+++ asdf.lisp	2008-12-29 11:10:45.000033000 -0500
@@ -705,8 +705,7 @@ the head of the tree"))
  ;;; So you look at this code and think "why isn't it a bunch of
  ;;; methods".  And the answer is, because standard method combination
  ;;; runs :before methods most->least-specific, which is back to front
-;;; for our purposes.  And CLISP doesn't have non-standard method
-;;; combinations, so let's keep it simple and aspire to portability
+;;; for our purposes.

  (defgeneric traverse (operation component))
  (defmethod traverse ((operation operation) (c component))
@@ -1056,9 +1055,6 @@ method.")
             (cond ((and s (eq (type-of (cdr s)) ',class))
                    (setf (car s) (get-universal-time)))
                   (s
-                  #+clisp
-                  (sysdef-error "Cannot redefine the existing system ~A with a 
different class" s)
-                  #-clisp
                    (change-class (cdr s) ',class))
                   (t
                    (register-system (quote ,name)

Re: [cclan-list] How do I replace my broken ':perform' forms?

[Thom G. <tho...@ma...>]

On Dec 8, 2008, at 11:47 AM, Robert Goldman wrote:

> Thom Goodsell wrote:
>> Sorry if this has been covered, or if it's just really obvious, but I
>> couldn't find anything helpful in the archives.
>>
>> I recently (i.e., about 15 minutes ago) got a new version of  
>> asdf.lisp
>> when I updated my copy of ccl, and it broke a behavior I've been
>> using. I *believe* that I was using the behavior incorrectly, and  
>> that
>> the change was a bugfix, though I could be misunderstanding the svn
>> logs.
>>
>> In my defsystem, I have the following bit of code:
>>
>>   :perform (load-op :before (op c)
>> 		    ;; the before method runs after dependencies are loaded,
>> 		    ;; but before the components are loaded, so clsql-mysql is
>> available
>> 		    (pushnew #p"/usr/local/mysql/lib/"      ; local development
>> 			     (symbol-value (find-symbol "*FOREIGN-LIBRARY-SEARCH-PATHS*"
>> 							:clsql-sys)))
>> 		    (asdf:oos 'asdf:load-op 'clsql-mysql))
>>
>> It appears, now, that my comment is not correct. The idea is that I
>> want to load all the systems from :depends-on, which includes clsql.
>> Then I want to set up clsql-sys:*foreign-library-search-paths* to
>> include the correct version of the MySQL libraries. Finally, I want  
>> to
>> load the mysql components of clsql.
>>
>> Does anyone have advice on the Right Way (or even just a good way) to
>> do this?
>>
>
> If I understand correctly, this won't work because the system itself
> undergoes the perform operation only /after/ all of its components
> undergo that operation.  I.e., if you have a system S with  
> components A
> and B, you'll see something like
>
> (perform <op> A) (perform <op> B) (perform <op> C)
>
> This has been a perpetual source of friction in using ASDF to do
> non-standard things.  For example, this is why I can't write  
> an :around
> method to dynamically bind a variable over the entire set of  
> operations
> on a system.
>
> What would be nice would be if the ASDF plan were to include a
> hierarchical structure something like:
>
> (perform <composite-op> S
>   :children
>   ((perform <op> A)
>   (perform <op> B)
>   (perform <op> C)))
>
> It's not obvious how to do this, though, since I think one could end  
> up
> with complex plans where the loading operations for a single component
> (S) are non-contiguous.
>
> There are various unpleasant kludges around this.  For your case, I
> would suggest making a faux system something like
>
> (defsystem clsql-version-identifier
>   :depends-on (:clsql)
>   :perform ... generalization of your :before method here...
> )
>
> Then make your *real* system depend on clsql-version-identifier.
>
> Note that I have NOT tried this, and I have a vague memory that ASDF  
> may
> behave oddly when given a system that has no components.  You'll  
> have to
> investigate this yourself.
>
> Alternatively, can't you just define a method like
>
> (defmethod perform :after ((op load-op) (c (find-system 'clsql- 
> mysql)))
>   ....)

For the record, this was the approach I ended up using (though the  
proper specialization is "(eql (find-system 'clsql-mysql))". After  
considering the other recommendations, this seemed like the best  
option for me. I do not intend to distribute this system (it's the key  
piece of our corporate IP), and my personal preference is to keep  
system startup as simple as possible, which I felt means keeping this  
functionality within the asd file so my start script can just load the  
system and start.

Thanks, all, for the suggestions.

Thom

Re: [cclan-list] How do I replace my broken ':perform' forms?

[Nikodemus S. <nik...@ra...>]

On Mon, Dec 8, 2008 at 6:13 PM, Thom Goodsell <tho...@ma...> wrote:

>   :perform (load-op :before (op c)
>                    ;; the before method runs after dependencies are loaded,
>                    ;; but before the components are loaded, so clsql-mysql is
> available
>                    (pushnew #p"/usr/local/mysql/lib/"      ; local development
>                             (symbol-value (find-symbol "*FOREIGN-LIBRARY-SEARCH-PATHS*"
>                                                        :clsql-sys)))
>                    (asdf:oos 'asdf:load-op 'clsql-mysql))

I would probably just use

  (defsystem ...
    :depends-on (:clsql-mysql)
    ...)

and leave search path configuration outside the system definition
file. (It's not going to be portable between different installations
anyways.)

Just document the fact that mysql libraries need to be set up right
(in LD_LIBRARY_PATH), or by user requiring :clsql-sys and pushing the
path onto *F-L-S-P* prior to trying to use your system.

(How do CLSQL users generally configure it? Is there maybe a
.clsql.lisp you can stick in $HOME, or something like that?)

Cheers,

 -- Nikodemus

Re: [cclan-list] How do I replace my broken ':perform' forms?

[Robert G. <rpg...@si...>]

Thom Goodsell wrote:
> Sorry if this has been covered, or if it's just really obvious, but I  
> couldn't find anything helpful in the archives.
> 
> I recently (i.e., about 15 minutes ago) got a new version of asdf.lisp  
> when I updated my copy of ccl, and it broke a behavior I've been  
> using. I *believe* that I was using the behavior incorrectly, and that  
> the change was a bugfix, though I could be misunderstanding the svn  
> logs.
> 
> In my defsystem, I have the following bit of code:
> 
>    :perform (load-op :before (op c)
> 		    ;; the before method runs after dependencies are loaded,
> 		    ;; but before the components are loaded, so clsql-mysql is  
> available
> 		    (pushnew #p"/usr/local/mysql/lib/"      ; local development
> 			     (symbol-value (find-symbol "*FOREIGN-LIBRARY-SEARCH-PATHS*"
> 							:clsql-sys)))
> 		    (asdf:oos 'asdf:load-op 'clsql-mysql))
> 
> It appears, now, that my comment is not correct. The idea is that I  
> want to load all the systems from :depends-on, which includes clsql.  
> Then I want to set up clsql-sys:*foreign-library-search-paths* to  
> include the correct version of the MySQL libraries. Finally, I want to  
> load the mysql components of clsql.
> 
> Does anyone have advice on the Right Way (or even just a good way) to  
> do this?
> 

If I understand correctly, this won't work because the system itself
undergoes the perform operation only /after/ all of its components
undergo that operation.  I.e., if you have a system S with components A
and B, you'll see something like

(perform <op> A) (perform <op> B) (perform <op> C)

This has been a perpetual source of friction in using ASDF to do
non-standard things.  For example, this is why I can't write an :around
method to dynamically bind a variable over the entire set of operations
on a system.

What would be nice would be if the ASDF plan were to include a
hierarchical structure something like:

(perform <composite-op> S
   :children
   ((perform <op> A)
   (perform <op> B)
   (perform <op> C)))

It's not obvious how to do this, though, since I think one could end up
with complex plans where the loading operations for a single component
(S) are non-contiguous.

There are various unpleasant kludges around this.  For your case, I
would suggest making a faux system something like

(defsystem clsql-version-identifier
   :depends-on (:clsql)
   :perform ... generalization of your :before method here...
)

Then make your *real* system depend on clsql-version-identifier.

Note that I have NOT tried this, and I have a vague memory that ASDF may
behave oddly when given a system that has no components.  You'll have to
investigate this yourself.

Alternatively, can't you just define a method like

(defmethod perform :after ((op load-op) (c (find-system 'clsql-mysql)))
   ....)

This may be tricky, though, to get the find-system to work properly when
this perform method definition is read...

Best,
r

[cclan-list] How do I replace my broken ':perform' forms?

[Thom G. <tho...@ma...>]

Sorry if this has been covered, or if it's just really obvious, but I  
couldn't find anything helpful in the archives.

I recently (i.e., about 15 minutes ago) got a new version of asdf.lisp  
when I updated my copy of ccl, and it broke a behavior I've been  
using. I *believe* that I was using the behavior incorrectly, and that  
the change was a bugfix, though I could be misunderstanding the svn  
logs.

In my defsystem, I have the following bit of code:

   :perform (load-op :before (op c)
		    ;; the before method runs after dependencies are loaded,
		    ;; but before the components are loaded, so clsql-mysql is  
available
		    (pushnew #p"/usr/local/mysql/lib/"      ; local development
			     (symbol-value (find-symbol "*FOREIGN-LIBRARY-SEARCH-PATHS*"
							:clsql-sys)))
		    (asdf:oos 'asdf:load-op 'clsql-mysql))

It appears, now, that my comment is not correct. The idea is that I  
want to load all the systems from :depends-on, which includes clsql.  
Then I want to set up clsql-sys:*foreign-library-search-paths* to  
include the correct version of the MySQL libraries. Finally, I want to  
load the mysql components of clsql.

Does anyone have advice on the Right Way (or even just a good way) to  
do this?

Thanks,
Thom

[cclan-list] asdf replacement

[F. <fa...@gm...>]

Note that if you want to develop something that's incompatible with
ASDF yet has a migration path from ASDF, you might be interested in
hacking XCVB. It's not production-ready yet (and it currently depends
on ASDF to be installed itself), but it does its job and brings
interesting new features like separate compilation.

[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
Be liberal in what you accept and conservative in what you send.
        -- Jon Postel

[cclan-list] adding system-loaded-p to ASDF

[Gary K. <gw...@me...>]

AFAIK, there is no _easy_ way to tell if a system has been loaded in  
ASDF. #'find-system lets me know if a system is defined but I have to  
look at operation-done-p to know whether or not it has been loaded.  
The following adds #'system-loaded-p and two methods on #'operation- 
done-p to make it easier to use by converting symbols into instances  
of reasonable classes.

(in-package #:asdf)

(defmethod operation-done-p ((o symbol) (c t))
  (operation-done-p (make-instance o) c))

(defmethod operation-done-p ((o t) (c symbol))
  (operation-done-p o (find-system c)))

(defun system-loaded-p (system)
  (operation-done-p 'load-op (find-system system)))

If there is no disagreement, I'll add some documentation and tests and  
commit.
--
Gary Warren King, metabang.com
Cell: (413) 559 8738
Fax: (206) 338-4052
gwkkwg on Skype * garethsan on AIM

[cclan-list] a non-contentious patch for ASDF... <smile>

[Gary K. <gw...@me...>]

I made a mistake when I applied Richard Kreuter's fix for ASDF's  
broken circularity detection a while back: I kept in the old ASDF code  
and added Richard's fix. The patch below removes the code that should  
have been removed then. With this in, ASDF passes all 14-tests.

Index: asdf.lisp
===================================================================
RCS file: /cvsroot/cclan/asdf/asdf.lisp,v
retrieving revision 1.129
diff -u -w -r1.129 asdf.lisp
--- asdf.lisp	4 Oct 2008 22:41:04 -0000	1.129
+++ asdf.lisp	15 Oct 2008 18:52:03 -0000
@@ -751,8 +751,6 @@
        (if (component-visiting-p operation c)
            (error 'circular-dependency :components (list c)))
        (setf (visiting-component operation c) t)
-      (loop for (required-op . deps) in (component-depends-on  
operation c)
-	 do (do-dep required-op deps))
        (unwind-protect
  	   (progn
  	     (loop for (required-op . deps) in

I've just committed this.
--
Gary Warren King, metabang.com
Cell: (413) 559 8738
Fax: (206) 338-4052
gwkkwg on Skype * garethsan on AIM

Re: [cclan-list] A plurality of defsystems [was Re: rfc - delay the parsing of system components ]

[GP l. <fp...@cl...>]

   From: Gary King <gw...@me...>
   Date: Tue, 14 Oct 2008 17:59:59 -0400

   Lisp tends to make things much harder because we keep inventing
   things. Each new defsystem adds another barrier to entry because it
   requires more setup and thinking. Choice is good; too much choice
   is paralyzing.

I must agee with this, having felt this way many times.  However, I
can also point at UFFI and CFFI as a somewhat similar situation to
what is going on in ASDF.  The names are similar, the functions are
similar, and the latter one improved upon the former.

I also know the confusion factor from 'same-name but incompatible with
the past'.  Pick a rather similar name, and give the ideas the test of
time.  People do beat pathways to code that moves them in a needed
direction.  Frankly worrying about the name, rather than the code
doesn't seem productive.

r

Re: [cclan-list] A plurality of defsystems [was Re: rfc - delay the parsing of system components ]

[Gary K. <gw...@me...>]

Hi Richard,

How meta! (pun intended)

On it's face, I think that your idea is great.  I do, however, have  
two concerns that revolve around your statement that:

> For my part, I don't see any reason why the Lisp community should
> constrain itself to having exactly one defsystem facility.  ASDF is
> useful, but it has shortcomings, and since it's never clear which of
> ASDF's details are intended or accident, and in any case which details
> are important to anybody, ISTM that it would probably be a good  
> thing if
> alternative defsystems could co-exist.

Sometimes, I think Lisp is nothing but a plethora of ways to do the  
same thing. Maybe it's only because I'm not familiar with the  
communities, but AFAIK, if one wants to install something in Ruby, you  
look for a GEM; if you want to get something for Python, you go to  
CPAN, etc. Lisp tends to make things much harder because we keep  
inventing things. Each new defsystem adds another barrier to entry  
because it requires more setup and thinking. Choice is good; too much  
choice is paralyzing.

To my mind, ASDF has _won_. It is the default and most Lisps are  
bundled with a version of it. I think that we should be improving it,  
not replacing it. The last thing we (as a Lisp community) need is  
another defsystem -- even a 99% compatible one.

My concerns therefore are:

1. that this additional layer adds another thing to break and to  
confuse people.
2. that building a new defsystem adds another thing for people to do  
if they want to use system _foo_ that uses it.

That said, I agree that incompatible changes should not be introduced  
without a lot of thought (that's why noone commits to ASDF unless  
there is a discussion here first, right?).

--
Gary Warren King, metabang.com
Cell: (413) 559 8738
Fax: (206) 338-4052
gwkkwg on Skype * garethsan on AIM

[cclan-list] A plurality of defsystems [was Re: rfc - delay the parsing of system components ]

[Richard M K. <kr...@pr...>]

Gary King writes:

> * I don't want to replace ASDF; it;s good, it works, it's in use;
> adding a new one just muddies the waters.

For my part, I don't see any reason why the Lisp community should
constrain itself to having exactly one defsystem facility.  ASDF is
useful, but it has shortcomings, and since it's never clear which of
ASDF's details are intended or accident, and in any case which details
are important to anybody, ISTM that it would probably be a good thing if
alternative defsystems could co-exist.

So I wonder how hard it would be to introduce an interface that
insulated users and programs from the details of how a system's
construction and loading happens to be implemented.  Something like the
following seems as though it would suffice:

--
BUILD-SYSTEM <NAME> &key &allow-other-keys                   [Function]

Call successive functions in *SYSTEM-BUILDER-HOOKS* using <NAME>, the
list of keywords, and :ALLOW-OTHER-KEYS T, until one returns true, and
signals an error if no function in *SYSTEM-BUILDER-HOOKS* returns true.

*SYSTEM-BUILDER-HOOKS*                                       [Variable]

A list of designators for functions that take a string designator and a
list of keywords.  When called, each function will attempt to build a
system identified by the string designator and possibly some of the
keyword arguments, and return true if such a system been built, or NIL
otherwise.  Building a system might have arbitrary side-effects on the
Lisp instance, but may not cause it to terminate.

LOAD-SYSTEM <NAME> &key &allow-other-keys                    [Function]

Call successive functions in *SYSTEM-LOADER-HOOKS* using <NAME>, the
list of keywords, and :ALLOW-OTHER-KEYS T until one returns true, and
signals an error if no function in *SYSTEM-LOADER-HOOKS* returns true.

*SYSTEM-LOADER-HOOKS*                                        [Function]

A list of designators for functions that take a string designator and a
list of keywords.  When called, each function will attempt to load a
system identified by the string designator and possibly some of the
keyword arguments.
--

So, for example, ASDF could install a hook for each of the two variables
that called OPERATE with suitable arguments, and other defsystems could
do analogous things.  Library management tools like asdf-install could
indirect through BUILD-SYSTEM and LOAD-SYSTEM.  Inter-system
dependencies could translate into to calls to LOAD-SYSTEM (possibly
after trying a defsystem-specific mechanism like what ASDF does now).
Since both of these functions takes keywords, things like system
versions or arguments to the construction or loading machinery can be
supplied and their interpretation is left to an underlying defsystem
tool.

Thoughts?

--
Richard

Re: [cclan-list] rfc - delay the parsing of system components

[Richard M K. <kr...@pr...>]

Gary King writes:

> 1. add keyword
> 
> * As in #0, Delay the parsing of system components (so that these can  
> come later)
> * Add a new keyword (as RIchard Kreuter suggested) that loads  
> extensions and which is processed "outside" of and before traverse/ 
> perform. Let's call it :requires
> * An operation will first process the requires form, then expand the  
> component forms, then run the existing traverse/perform

I didn't think my suggestion through.  One way to use an ASDF extension
is to include package-qualified symbols in the DEFSYSTEM form, and so
it's not generally possible to read a DEFSYSTEM form before the relevant
extensions are loaded.  Example:

(defsystem foo
   :requires "my-asdf-stuff"
   :components ((:file "foo" :class my-asdf-stuff:whizbang-source-file)))

So my suggestion won't work for all cases.  Sorry about that.

--
Richard

Re: [cclan-list] rfc - delay the parsing of system components

[Robert G. <rpg...@si...>]

Gary King wrote:
> OK. I hear what everyone is saying and appreciate the feedback. I'd  
> like to back up (or out!) a bit and reconsider my goals:
> 
> * I don't want to replace ASDF; it;s good, it works, it's in use;  
> adding a new one just muddies the waters.
> * I do want to make it easier to use extensions (new operations and  
> file types) in a system file.
> 
> My first plan was the proposal to delay parsing and muck with  
> traverse / perform. Here is two others:
> 
> 1. add keyword
> 
> * As in #0, Delay the parsing of system components (so that these can  
> come later)
> * Add a new keyword (as RIchard Kreuter suggested) that loads  
> extensions and which is processed "outside" of and before traverse/ 
> perform. Let's call it :requires

Nitpicky suggestion --- let's call it something else to avoid confusing
with the (deprecated) CL notion of require.

> * An operation will first process the requires form, then expand the  
> component forms, then run the existing traverse/perform

While we're messing with this, any chance of getting something which
wraps around the whole operation on a component so that we can, e.g.,
impose dynamic bindings around the entire load-op as applied to a
system?  I suppose that's off-topic, but the current overhaul seems like
a good opportunity to get that done as well....

This seems like the preferable solution.

Re: [cclan-list] rfc - delay the parsing of system components

[Gary K. <gw...@me...>]

OK. I hear what everyone is saying and appreciate the feedback. I'd  
like to back up (or out!) a bit and reconsider my goals:

* I don't want to replace ASDF; it;s good, it works, it's in use;  
adding a new one just muddies the waters.
* I do want to make it easier to use extensions (new operations and  
file types) in a system file.

My first plan was the proposal to delay parsing and muck with  
traverse / perform. Here is two others:

1. add keyword

* As in #0, Delay the parsing of system components (so that these can  
come later)
* Add a new keyword (as RIchard Kreuter suggested) that loads  
extensions and which is processed "outside" of and before traverse/ 
perform. Let's call it :requires
* An operation will first process the requires form, then expand the  
component forms, then run the existing traverse/perform

2. Like #1 only without the new keyword.

* ASDF behaves as it does now except that a systems :depends-on is  
processed before and separately from the processing of its components.

As I see it, #2 this keeps the syntax the same but runs a higher risk  
of breaking things whereas #1 adds syntax but can cause no breakage.

Both of these lose the ability to (easily) wrap restarts around  
modules or systems.
--
Gary Warren King, metabang.com
Cell: (413) 559 8738
Fax: (206) 338-4052
gwkkwg on Skype * garethsan on AIM

Re: [cclan-list] rfc - delay the parsing of system components

[Daniel B. <da...@te...>]

"Nikodemus Siivola" <nik...@ra...> writes:

> On Mon, Oct 13, 2008 at 4:08 PM, Christophe Rhodes <cs...@ca...> wrote:
>
>> I don't see why you even _want_ to keep the same name, frankly.
>
> I agree. "plan, then execute" is one of the major (mis?)features of
> ASDF. While I have often wanted it, calling a system that behaves like
> that ASDF is just confusing.

Excuse me if I'm about to rehash old ground.

I cannot entirely remember the reasons for plan-then-execute, but asdf
was changed to work that way very early in its history and there must
have been some kind of justification at the time.  One point is that if
you want to write operation-done-p methods which compare file dates
then it's much easier to think about if you're comparing the dates
from *before* you started work than from some arbitrary point halfway
through the operation - it's also safer in the face of clock drift.

KMP does the the same thing in his "Description of Large Systems"
paper, though doesn't give much justification for it beyond "Because a
system can be asked to produce a plan for an operation such as
compilation without actually performing the operation, it is possible
to write programs which inspect the plan, possibly optimizing it or
presenting it for for interactive approval, before executing it. "

http://www.nhplace.com/kent/Papers/Large-Systems.html

A lot of this was driven by IRC discussion with Kevin Rosenberg - I
wish I'd kept notes in some more easily-searchable format :-( I don't
have a strong opinion these days on which approach is "better", but I
do agree that the change will break code in the wild, and should imply
a name change if made.


-dan

Re: [cclan-list] rfc - delay the parsing of system components

[Nikodemus S. <nik...@ra...>]

On Mon, Oct 13, 2008 at 4:08 PM, Christophe Rhodes <cs...@ca...> wrote:

> I don't see why you even _want_ to keep the same name, frankly.

I agree. "plan, then execute" is one of the major (mis?)features of
ASDF. While I have often wanted it, calling a system that behaves like
that ASDF is just confusing.

While the change may be practically backwards-incompatible, it is a
major headache to deal with when people start actually using it --
since system depending on it will *not* work on older ASDF versions.

YASDF, ASDF2, ASDF.REC, GKSDF, whatever -- and I *like* the idea, but
not calling it ASDF.

Cheers,

 -- Nikodemus

Re: [cclan-list] rfc - delay the parsing of system components

[Christophe R. <cs...@ca...>]

Gary King <gw...@me...> writes:

> The question reduces to whether or not this is a different species. I
> don't think it is. Richard correctly points out that my proposal might
> cause problems in the "wild" but there is no evidence for this
> (yet...). To my mind, I've kept the same syntax and ASDF does the same
> operations in the same order (at least in as much as ASDF ever did the
> same things in the same order...). The only change is that instead of
> planning everything before executing, ASDF would now execute as it
> went. I don't see this as incompatible. Do you?

Yes, I do, because (as I understand it) it means that performing
operations on components now wraps operations on subcomponents, rather
than happening afterwards.

If this causes no difficulties in the wild, then it also costs you
very little to call this "GHJK", provide an in-the-wild-compatible
"ASDF" package, and advertise 99.9% (or whatever) ASDF-compatibility,
without the horrible broken bits and with additional capabilities.

The cost of not doing so is, in my experience, huge: over time,
references to "ASDF"-the-software accumulate, and suddenly no-one
knows whether a particular reference applies to before or after your
change, or both.

I don't see why you even _want_ to keep the same name, frankly.

Best,

Christophe

Re: [cclan-list] rfc - delay the parsing of system components

[Gary K. <gw...@me...>]

>> Hmmm, evolve or die <smile>.
>
> No problem with that, but biological taxonomists have learnt that it's
> useful to give mutually-incompatible species different names.  Some
> day, computer programmers might see the point of that, too.
>
> (In other words: please do make code easier to write, but please also
> give the resulting software a different name.)


The question reduces to whether or not this is a different species. I  
don't think it is. Richard correctly points out that my proposal might  
cause problems in the "wild" but there is no evidence for this  
(yet...). To my mind, I've kept the same syntax and ASDF does the same  
operations in the same order (at least in as much as ASDF ever did the  
same things in the same order...). The only change is that instead of  
planning everything before executing, ASDF would now execute as it  
went. I don't see this as incompatible. Do you?

thanks,
--
Gary Warren King, metabang.com
Cell: (413) 559 8738
Fax: (206) 338-4052
gwkkwg on Skype * garethsan on AIM

Re: [cclan-list] rfc - delay the parsing of system components

[Christophe R. <cs...@ca...>]

Gary King <gw...@me...> writes:

>> I think #2 is an at-least-in-theory incompatible change.  It might not
>> break anything in the wild, but that can't be known in advance.  It
>> might also lead to a better defsystem than ASDF, but I'm not sure that
>> ASDF should be breaking compatibility with itself at this point.
>
> Hmmm, evolve or die <smile>.

No problem with that, but biological taxonomists have learnt that it's
useful to give mutually-incompatible species different names.  Some
day, computer programmers might see the point of that, too.

(In other words: please do make code easier to write, but please also
give the resulting software a different name.)

Cheers,

Christophe

Re: [cclan-list] rfc - delay the parsing of system components

[Gary K. <gw...@me...>]

Hi Richard, et. al.

Thanks for your comments and thoughts.

> In the plan-before-execute design that ASDF's always had, methods that
> implement pieces of the plan construction protocol (OPERATION-DONE-P,
> COMPONENT-DEPENDS-ON, et al.) can rely on being executed before any
> steps are carried out in the current run.

This is true but I have my doubts that it will matter. Methods like  
COMPONENT-DEPENDS-ON shouldn't really have anything to do with the  
environment (either X depends on Y or it doesn't). Methods like  
operation-done-p should depend on a component and its children (not  
its parents) and thus would not be altered by the proposed change.

> The idiomatic solution for this is to load ASDF extensions in an
> EVAL-WHEN at the beginning of a .asd file.  But maybe what you're  
> after
> is representing the fact that system FOO needs ASDF extension BAR,  
> which
> isn't explicit with the EVAL-WHEN solution.  If that's what you  
> need, it
> might suffice to add a slot for this to SYSTEM and to modify
> PARSE-COMPONENT-FORM to hoist out the slot's initarg and do the LOAD- 
> OP
> on those extensions before looping over the components.

It's true that my proposal isn't required to make ASDF jump through  
the hoops I want it to. IMHO, the eval-when solution is ugly and while  
adding another slot is alright, it seems to me that I should be able  
to use operations and components and such defined by a system's  
dependencies. I.e., that what I'm proposing is the way things should  
be. Obviously, reasonable people can disagree!

> I think #2 is an at-least-in-theory incompatible change.  It might not
> break anything in the wild, but that can't be known in advance.  It
> might also lead to a better defsystem than ASDF, but I'm not sure that
> ASDF should be breaking compatibility with itself at this point.

Hmmm, evolve or die <smile>. There is always more code to write in the  
future than has been written in the past. This proposed change makes  
ASDF simpler in a few ways and, arguably, a bit easier to understand.  
It also allows for restarts like "recompile module 'foo'" or  
"recompile system 'bar'" that the plan-before-execute design makes  
very difficult (because the plan more or less obliterates the  
dependency structure).

There is still more cleanup to do but I think I'll post the source  
somewhere that people can download and try and see if it breaks  
anything. So far, the changed version is liking all the systems I've  
tossed its way.

thanks again,
--
Gary Warren King, metabang.com
Cell: (413) 559 8738
Fax: (206) 338-4052
gwkkwg on Skype * garethsan on AIM

Re: [cclan-list] rfc - delay the parsing of system components

[Richard M K. <kr...@pr...>]

Gary King writes:
> I just finished a rough cut of an ASDF overall that makes three  
> largish changes:
> 
> 1. delays the parsing of system components until needed by traverse.  
> In particular, this means that the :components clause of defsystem  
> form happens after that system's dependencies have been loaded rather  
> than when the ASD file itself is loaded.
>
> 2. rewrites traverse as a set of methods that immediately call perform  
> (rather than building up a list of perform actions that  are then  
> called in operate).

In the plan-before-execute design that ASDF's always had, methods that
implement pieces of the plan construction protocol (OPERATION-DONE-P,
COMPONENT-DEPENDS-ON, et al.) can rely on being executed before any
steps are carried out in the current run.  Since the protocol doesn't
have any solid semantics, extensions' methods can do anything, really;
executing them mid-way through execution can expose them to conditions
that are arbitrarily unlike what such methods have been written to
reason about.

> 3. does away with :do-first and moves what it did into :in-order-to
>
> The main motivation for change #1 is to be able to define new  
> operations and source file classes in a system's dependencies and have  
> them used in that systems definitions; 

The idiomatic solution for this is to load ASDF extensions in an
EVAL-WHEN at the beginning of a .asd file.  But maybe what you're after
is representing the fact that system FOO needs ASDF extension BAR, which
isn't explicit with the EVAL-WHEN solution.  If that's what you need, it
might suffice to add a slot for this to SYSTEM and to modify
PARSE-COMPONENT-FORM to hoist out the slot's initarg and do the LOAD-OP
on those extensions before looping over the components.

> ... the main motivation#2 was #1: it does no good to delay parsing of
> the defsystem form if traverse has to complete the parse before
> anything else happens!. As for #3, while doing three things at once is
> just very human (and besides, it made #2 easier).
> 
> I've got several hours of work left cleaning up loose ends and moving  
> restarts and with-compilation-units around into better places.
> 
> What I'd appreciate is any high-level feedback about the idea of these  
> changes and whether or not I've missed some horrible interaction that  
> nullifies the whole direction. I hope to post the suggested changes  
> later in the week.

I think #2 is an at-least-in-theory incompatible change.  It might not
break anything in the wild, but that can't be known in advance.  It
might also lead to a better defsystem than ASDF, but I'm not sure that
ASDF should be breaking compatibility with itself at this point.

--
Richard

[cclan-list] rfc - delay the parsing of system components

[Gary K. <gw...@me...>]

I just finished a rough cut of an ASDF overall that makes three  
largish changes:

1. delays the parsing of system components until needed by traverse.  
In particular, this means that the :components clause of defsystem  
form happens after that system's dependencies have been loaded rather  
than when the ASD file itself is loaded.

2. rewrites traverse as a set of methods that immediately call perform  
(rather than building up a list of perform actions that  are then  
called in operate).

3. does away with :do-first and moves what it did into :in-order-to

The main motivation for change #1 is to be able to define new  
operations and source file classes in a system's dependencies and have  
them used in that systems definitions; the main motivation#2 was #1:  
it does no good to delay parsing of the defsystem form if traverse has  
to complete the parse before anything else happens!. As for #3, while  
doing three things at once is just very human (and besides, it made #2  
easier).

I've got several hours of work left cleaning up loose ends and moving  
restarts and with-compilation-units around into better places.

What I'd appreciate is any high-level feedback about the idea of these  
changes and whether or not I've missed some horrible interaction that  
nullifies the whole direction. I hope to post the suggested changes  
later in the week.

Until then, happy coding.
--
Gary Warren King, metabang.com
Cell: (413) 559 8738
Fax: (206) 338-4052
gwkkwg on Skype * garethsan on AIM

Re: [cclan-list] in-order-to and/or do-first

[Andreas F. <as...@bo...>]

On Oct 8, 2008, at 2:22, Richard M Kreuter wrote:
> Gary King writes:
>> Can anyone recall why we have both in-order-to and do-first? Doesn't
>> in-order-to subsume do-first?
>
> This came up a little more than a year ago:
>
> http://article.gmane.org/gmane.lisp.cclan.general/674


The separation between the two as it is right now is used by what I'll  
call compilation dependencies: Compilation of file A will trigger  
compilation of dependent file B if load-op is performed on the system  
containing them. I tried to pry in-order-to and do-first apart once,  
then just gave up and left things as they were, because I could not  
figure out how to guarantee that compilation of dependent files  
happens before loading in this case.

HTH,
-- 
Andreas Fuchs, (http://|im:asf@|mailto:asf@)boinkor.net, antifuchs

Re: [cclan-list] in-order-to and/or do-first

[Richard M K. <kr...@pr...>]

Gary King writes:
> Can anyone recall why we have both in-order-to and do-first? Doesn't  
> in-order-to subsume do-first?

This came up a little more than a year ago:

http://article.gmane.org/gmane.lisp.cclan.general/674

--
Richard

[cclan-list] in-order-to and/or do-first

[Gary K. <gw...@me...>]

Can anyone recall why we have both in-order-to and do-first? Doesn't  
in-order-to subsume do-first?

thanks,
--
Gary Warren King, metabang.com
Cell: (413) 559 8738
Fax: (206) 338-4052
gwkkwg on Skype * garethsan on AIM