Re: Regarding defsystem-4

[Marco A. <ma...@cs...>]

Hi Sunil,

let me forward this to clocc-devel too.  It is too interesting.

> Date: Wed, 10 Jan 2001 18:03:45 -0800
> From: Sunil Mishra <sun...@ev...>
> User-Agent: Mozilla/5.0 (X11; U; Linux 2.2.14-5.0.14smp i686; en-US; m18) Gecko/20010109
> X-Accept-Language: en
> Content-Type: multipart/mixed;
>  boundary="------------070503040100060307050007"
> Content-Length: 20892
> 
> Marco Antoniotti wrote:
> 
>  > I will selectively respond.  Sorry.
> 
> Me too :-)

>  >  (much deleted)
>
> 
> OK, let me ask some more specific questions on where you want to go with
> dependency handling.
> 
> 1. Do you want to keep all the current component types-- :module,
> :subsystem, etc.? My impression is that you might not be interested in
> keeping all these distinctions, so I won't say very much about them.

I think we want to maintain

  :system

  :module
  
  :file
	:private-file

I do not believe people use anything different.  Maybe :subsystem can
be maintained as well.  The hierarchy displayed in the spec document
is just a start.

What is the reason to maintain these three "roots"?  I want to
understand the problem as follows.

:system components have two characteristics

	1 - they are mostly standalone (inasmuch as they do not depend
	    on other systems).  They may be compared to "applications"
	    (and in this sense maybe it would be better to distinguish
	    systems into two kinds).

	2 - they are "closed" (or "sealed") in the sense that they can
	    be accessed only via a well defined "published" interface
	    (in a sense the only thing that systems could make available as
	    a piece of testable information is their "version").

	    From this it follows that none of a system innards
	    (i.e. modules and files) is "visible" from the outside.
	    In this sense a "system" is a C/C++ "library".  It also
	    implies that a system may depend only on the systems it
	    declared to depend on in its definition form.

:module components are not standalone.

	They may depend on other parts of the system definition.
	Other components can depend on modules as a whole, but not on their
	"inner" definitions.  However, each piece of a module may depend on
	other parts of the system definition, barring the previous limitation.
	One such dependency will make the whole module depend on it, although
	this the actual "operation" order may not strictly follow the
	module-assumed dependency.

:file components are the building block of systems.
       (apart from the :private-file definition) their dependency
       rules simply follow from the previous definitions.

> 2. Do you want to use topological sort to turn the list of components
> into a serial dependency, as you have called it? Or do you want to use
> topological sort to track inter-file dependencies and come up with a
> minimal set of operations?

Doesn't (1) imply (2) to a certain extent?  I want the topological
sort to alleviate the burden to specify the exact "global" order of
"operation"s.  As such (1) is good and (2) is good too.

> 3. Do you wish to apply topological sort to components, between system
> dependencies, or both? I'm assuming both, in which case I shall try to
> change your mind :-)

I want to apply the topsort to components.  The dependencies among
systems are of a different nature.  I do not have a clear idea about
what to do when a dependency from system A to system B is in place.

Setting aside the issue of "system versions", you can still ask the
question?

Suppose I am manipulating two systems A and B (it happens to me: I use the
priority queue package as a building block in my application) with
system A depending on system B.  I also have the dependency rule in
the definition of system A (note that IMHO this ruls is not quite
well addressed in MK-DEFSYSTEM 3.2i).  Now, I find a bug in system B
and fix it.  However, I recompile/reload system A instead of
recompiling/reloading system B.  What should happen?
One way to resolve the dependency (i.e. to propagate the operation to
system B) could be to keep a "system wide" modified-flag in the system
definition which could be tested in this case.

This is just a thought.

> There is a very natural place where topological sort can be introduced
> into the current order of events. A gf, operate-on-next-component, walks
> through the components list of a system. Components may be arbitrarily
> ordered at this stage, which is what you would want to do for a
> topological sort. If all you are interested in is turning a list of
> dependencies into a nicer serial ordering, that would be trivial to
> accomplish. (AFAIK, that is what CLOS does as it dynamically calculates
> the specificity of methods for some set of arguments.) The more complex
> operation of calculating a minimal set I believe could also be done in
> the existing framework. But that might take a little more creative
> work.

I am assuming, one component = one operation.  If you can serialize
the components, why bother with "minimality" of operations?

> If you want to support all the different component types that currently
> exist in defsystem, I don't think it will be nearly as easy to decompose
> the make problem to fit into the scheme that I have now. And as I have
> already said, I don't think they add all that much to the expressiveness
> of defsystem.

I am not so stubborn.  However, back compatibility is very important
to me (and I assume to many other people).

> As for the issue of what it is that you want to topologically sort... I
> can see the benefit a topological sort might bring to handling
> dependencies that exist within a system. But inter-system dependencies
> have turned out to be a bit different. In my experience, you want to
> check all the systems that comprise the application for updates, for
> every operation. (Unless the user explicitly specifies otherwise, of
> course.) Otherwise it is very easy to miss updates to subsystems, etc. A
> topological sort at this level, therefore, would not do all that much
> good. A simpler DFS is sufficient to capture the dependencies.

Come on, implementing a DFS or a Topsort has the same complexity.
What is trickier is to *define* in advance all (most of) the possible
cases that can tweak the underlying dependency graph one way or the
other.  That is wy I insist on writing out the specs first.

> I have put in some language support in the code I'm attaching, but it is 
> untested. All I can say with certainty is that it compiles and shows 
> signs of proper functionality under lispworks 4.1. The system keywords 
> and functions have been modified to closely match those of mk:defsystem. 
> MK has also been added as a package nickname. I have nearly all the 
> functionality I had wanted from a defsystem, so there is a chance I 
> might not be an active participant in the further development of 
> defsystem4. If this serves as a useful starting point for the rest of 
> you, that would make for a great outcome for my efforts.

I am sure it will.  I think you should at least subscribe to the
clocc-devel mailing list.  It is pretty low volume so far.

(As long as the #$!!@ CVS messages get rerouted to the proper mailing
list 'clocc-cvs' :) ).

Cheers

-- 
Marco Antoniotti =============================================================
NYU Bioinformatics Group			 tel. +1 - 212 - 998 3488
719 Broadway 12th Floor                          fax  +1 - 212 - 995 4122
New York, NY 10003, USA				 http://galt.mrl.nyu.edu/valis
             Like DNA, such a language [Lisp] does not go out of style.
			      Paul Graham, ANSI Common Lisp