clocc-devel

Re: Problem with cvs

[Stefan <st...@hi...>]

Hello Stig,

On Mon, 31 Jan 2000, Stig Erik Sandoe wrote:
> stig@palomba(13:24)[~/Projects] 141> cvs
> -d:pserver:ano...@cv...:/cvsroot/clocc login
> (Logging in to ano...@cv...)
> CVS password:

You maybe have hit a key inadvertently. Just press <return> and it should
work.

Bye,

Stefan

Problem with cvs

[Stig E. S. <st...@ii...>]

hi,

I tried to do a cvs login as anonymous to get the clocc source so
far, but got:

stig@palomba(13:24)[~/Projects] 141> cvs
-d:pserver:ano...@cv...:/cvsroot/clocc login
(Logging in to ano...@cv...)
CVS password:
cvs [login aborted]: authorization failed: server
cvs.clocc.sourceforge.net rejected access



------------------------------------------------------------------
Stig Erik Sandoe     st...@ii...    http://www.ii.uib.no/~stig/

Re: Correct phrasing of copyright notices.

[Bruno H. <ha...@il...>]

Marco Antoniotti writes:

> suppose I want to release something under the LGPL.  Is the following
> phraseology correct/acceptable
> 
> ;;; Copyright (c) 2000 Marco Antoniotti, all rights reserved.
> ;;; This software is released under the terms of the LGPL (see file
> ;;; COPYRIGHT for details).

I think it would be better to spell out its name:

;;; This software is released under the terms of the GNU Lesser General
;;; Public License (LGPL, see file COPYRIGHT for details).

Bruno

Re: what sort of modules should be in clocc

[<do...@co...>]

  > >   >  >> There are plenty of large projects with their own make programs.
  > >   > this is BAD.
  > > I disagree.  I think it's fine, for instance, that cl-http comes with
  > > its own build-cl-http function.  I don't know or care how it's
  > > implemented.
  > 
  > The question you should ask is "why does CL-HTTP comes with its own
  > build utilitiy?"
  > 
  > I believe the answer is "because up to now there was no acceptable and
  > docemented and stand-alone 'defsystem' running on all implementations.
  > 
  > I bet that the CL-HTTP developers would have chosen MK:DEFSYSTEM
  > (which does not have the pretense of "being better") had it been
  > "available" when they started (where "available" is to be intended in
  > a wide, license-encompassing, way).

This is certainly one possibility, and my view is that if this is the
case, nobody should mind that they don't rush to replace whatever they
have with a standard defsystem.  

  > > Do you imagine that there is no cost to replacing something that
  > > already works?  Generally it takes a considerable effort even to
  > > determine that the replacement actually has all the capabilities that
  > > you need.  I don't know much about mk defsystem (other than what I've
  > > read in the last hour), but I'm willing to bet right now that it does
  > > not have all that I'd need to replace the homegrown defsystems that
  > > I've been maintaining.
  > 
  > Well, try MK:DEFSYSTEM out and make a list of missing features.  Then,
  > if you are still not satisfied advocate the use of your defsystem and
  > make it available as a standalone module.

You imagine there is one ideal defsystem for all uses.  I don't buy
that.  A system that supports lots of complex features is not worth
the trouble for simple uses.  Just like defsystem is not worth the
trouble for very small facilities.

    [[my tangled dependency hierarchy nightmare]]
  > Allow me to say that this is the situation you run in C/C++

I agree, but that doesn't make it any more attractive.

  > programming as well.  So you should not be terrified by it.  As far as
  > the CLOCC is concerned I'd hope that the code appearing here would be
  > tested as much as possible on as many platforms as possible.  So, the
  > possibility to run into errors should be minimized.

My point is that we can minimize the problem by minimizing
dependencies to those that are essential.

    [[examples of such problems]] 
  > What are you supposed to do in this case?  Write code that does not
  > use these fetures because it may cause problems to somebody wanting to
  > use your code? Or do you work around them?  I choose to work around
  > them and hope the the full code will be therefore more useful.

I agree this is better.  It's impossible to avoid features
that don't work in implementations you don't even know about.

  > Logically "a system" should be standalone.  If your system depends on
  > another system, then this should mean that you are using the
  > "advertised interface" provied to you by the designer of the
  > system/library/whatever.

Yes, that is correct.  If your system is stand alone then I am happy
(in the sense of this discussion) since there are NO dependencies.

I begin to see that part of the question here is what constitutes a
system.  One of my desires is that I should be able to get not much
more code than I need. That argues that Sam's cllib should not be one
system, take it or leave it.  I should be able to get the url stuff
without getting the animals or gnuplot stuff (I don't really know
whether these are realistic examples).
On the other hand, if he breaks cllib into 10 systems, there will be
a lot of dependencies between them.  I don't mind that, as long as
those are really essential dependencies, e.g., getting stock quotes
off the web relies on web stuff.  

What the downloading user then needs (I don't know what we have now)
is (1) a tar/gzip bundle of all the files that make up one system so
he doesn't have to find and download them separately, and (2) some
facility for tracing the dependencies among systems that helps him
to download the transitive closure of those systems needed by the one
he wants.  Since we do seem to have versioning, it would be nice if
these things were version sensitive.  That is, the "database" I have
in mind should indicate that system get-quote version 3.2 relies on
system web-interface version 5.3.  If I want to load get-quote 3.2
and also animals 4.5, and one of these relies on base 2.1 while the
other relies on base 2.2, there's an incompatibility.  
(BTW, this is the kinds of thing that is supported by on of the build
facilities I mentioned earlier.)

  > > More to the point, though, even if you could do this, the dependencies
  > > are only among files, right?  Or is there a model of dependency among
  > > individual top level forms?  (We used to have something like this in
  > > a project I worked on, but I've never seen it anywhere else.)
  > 
  > You may ahev dependencies among "modules" (i.e. collections of
  > components, e.g. files).  It seems to me that you are wanting a very
  > fine grained dependency definition.  What is the "semantic" ground you
  > base this request?

This was not actually a request.  The point is that without such a
fine grained dependency model you have to put some effort into
organizing files so as to minimize dependencies.

Re: sourceforge CLOCC modules

[Stefan <st...@hi...>]

Hello again, :-)

On Sat, 29 Jan 2000, Stefan wrote:
> With 'modules' one can structure the 'file release' page,
> so for each module you can see a list of recent releases on that
> page.

I think (but do not know 100%) that adding a module via the 
'add module' link will change the CVS-structure like this:

---
CVSROOT
clocc
<new module>
---

This means, that defsystem would have to be checked out off the repository
seperately all the time:

cvs <lots of options...:-)> co clocc

and

cvs <lots of options...:-)> co defsystem

I do not like that, actually, but maybe I am wrong.

I will ask the admins...

Bye,

Stefan

Re: sourceforge CLOCC modules

[Stefan <st...@hi...>]

Hi Marco,

On Sat, 29 Jan 2000, Marco Antoniotti wrote:
> The 'File releases' section in the clocc page
> shows only the CLOCC module.  Is this correct?
> 

I have just read the sourceforge-module docu.

CVS-modules and project-modules are different.
So defining defsystem in CVSROOT/modules 
affects only CVS and not the 'File release'-page.

With 'modules' one can structure the 'file release' page,
so for each module you can see a list of recent releases on that
page.

If you'd like to provide seperate releases of defsystem apart from
clocc, we should add defsystem as another module to the 'File release'
page.


Bye,

Stefan

sourceforge CLOCC modules

[Marco A. <ma...@pa...>]

Hi

just to be annoying.  The 'File releases' section in the clocc page
shows only the CLOCC module.  Is this correct?

Cheers

-- 
Marco Antoniotti ===========================================
PARADES, Via San Pantaleo 66, I-00186 Rome, ITALY
tel. +39 - 06 68 10 03 17, fax. +39 - 06 68 80 79 26
http://www.parades.rm.cnr.it/~marcoxa

Correct phrasing of copyright notices.

[Marco A. <ma...@pa...>]

Hi,

suppose I want to release something under the LGPL.  Is the following
phraseology correct/acceptable

;;; Copyright (c) 2000 Marco Antoniotti, all rights reserved.
;;; This software is released under the terms of the LGPL (see file
;;; COPYRIGHT for details).

Cheers

-- 
Marco Antoniotti ===========================================
PARADES, Via San Pantaleo 66, I-00186 Rome, ITALY
tel. +39 - 06 68 10 03 17, fax. +39 - 06 68 80 79 26
http://www.parades.rm.cnr.it/~marcoxa

Re: what sort of modules should be in clocc

[Marco A. <ma...@pa...>]

> Delivery-Date: Fri, 28 Jan 2000 21:23:42 +0100
> Date: Fri, 28 Jan 2000 12:23:56 -0800
> From: do...@co... (Don Cohen)
> Sender: clo...@li...
> X-Mailman-Version: 1.1
> Precedence: bulk
> List-Id:  <clocc-devel.lists.sourceforge.net>
> X-BeenThere: clo...@li...
> 
> I thought I saw defsystem doc in clocc before but now I don't.
> The source mentions
> ;;; For more information, see the documentation and examples in 
> ;;; lisp-utilities.ps.
> 
>   >  >> There are plenty of large projects with their own make programs.
>   > this is BAD.
> I disagree.  I think it's fine, for instance, that cl-http comes with
> its own build-cl-http function.  I don't know or care how it's
> implemented.

The question you should ask is "why does CL-HTTP comes with its own
build utilitiy?"

I believe the answer is "because up to now there was no acceptable and
docemented and stand-alone 'defsystem' running on all implementations.

I bet that the CL-HTTP developers would have chosen MK:DEFSYSTEM
(which does not have the pretense of "being better") had it been
"available" when they started (where "available" is to be intended in
a wide, license-encompassing, way).

> 
>   > there certainly IS a reason to drop your own home-grown tools when there
>   > is a standard one available which does the job better.
>   > do you make your car yourself or buy it from a well-established vendor?
> 
> Do you imagine that there is no cost to replacing something that
> already works?  Generally it takes a considerable effort even to
> determine that the replacement actually has all the capabilities that
> you need.  I don't know much about mk defsystem (other than what I've
> read in the last hour), but I'm willing to bet right now that it does
> not have all that I'd need to replace the homegrown defsystems that
> I've been maintaining.

Well, try MK:DEFSYSTEM out and make a list of missing features.  Then,
if you are still not satisfied advocate the use of your defsystem and
make it available as a standalone module.

     ...

> I think we can agree that this is a matter of degree.  The extreme
> that I fear is that I want a few functions from a few files, but these
> use other files and those use other files ...  I eventually get them
> all and then run into errors, e.g., maybe there are incompatibilities
> in the versions I got, or maybe there are incompatibilities between
> two different things I want to put together, or one of those things
> doesn't work cause of a bug in the lisp implementation I'm using.  Now
> I have to understand all that code.  Or more likely, I start from what
> I want and trace the dependencies and throw out all that I don't need
> in hopes that the errors come from stuff I didn't need anyway.

Allow me to say that this is the situation you run in C/C++
programming as well.  So you should not be terrified by it.  As far as
the CLOCC is concerned I'd hope that the code appearing here would be
tested as much as possible on as many platforms as possible.  So, the
possibility to run into errors should be minimized.

Just to make an example.  I was fooling around with DEFSYSTEM on LWPC
and ACLPC (the free editions).  I ran in the following problems:

1 - COMPILE-FILE does not work on ACLPC Free (as it should be - and
    yes, I know of LOAD-COMPILED).
2 - EXCL:CHANGE-DIRECTORY does not change
    *DEFAULT-PATHNAMES-DEFAULTS*, so, doing (LOAD "filename.lisp")
    does not work, even if you used EXCL:CHANGE-DIRECTORY correctly.
3 - LWPC chokes on files containing the idiom

         (format t "this is a line ~
                    and this is another")

    when the original file was modified on a UN*X system.

What are you supposed to do in this case?  Write code that does not
use these fetures because it may cause problems to somebody wanting to
use your code? Or do you work around them?  I choose to work around
them and hope the the full code will be therefore more useful.

> You're right that I do not know about MK defsystem.  However, the
> comment below from the source suggests that this ability is not even 
> available.  I suspect this refers to modules within one system, and
> that such dependencies between parts of different systems is even
> further out of the question.

Logically "a system" should be standalone.  If your system depends on
another system, then this should mean that you are using the
"advertised interface" provied to you by the designer of the
system/library/whatever.

It simply a matter of granularity.  Somewhere/sometime, interfaces do
get setup.  At that point it is an all or nothing situation.  Or
better, it is not up to you to decide what comes inn or not.

When in Java you do

     import java.util.StringTokenizer;

you have no guarantee that you will not load much more than you
bargained for.  The same goes for systems define with mk:defsystem.

Having siad that, I am all for providing more complex dependencies
within the implementations.

> More to the point, though, even if you could do this, the dependencies
> are only among files, right?  Or is there a model of dependency among
> individual top level forms?  (We used to have something like this in
> a project I worked on, but I've never seen it anywhere else.)

You may ahev dependencies among "modules" (i.e. collections of
components, e.g. files).  It seems to me that you are wanting a very
fine grained dependency definition.  What is the "semantic" ground you
base this request?

> I worry that something I don't even use in bar.lsp will need something
> in another file, and so on, so that I end up getting way too much.

See my comment above about Java.

> ;;; Current dependencies are limited to siblings. Maybe we should allow
> ;;; nephews and uncles? So long as it is still a DAG, we can sort it.
> ;;; Answer: No. The current setup enforces a structure on the modularity.
> ;;; Otherwise, why should we have modules if we're going to ignore
> ;;; it?

This is a design choiche I am aware of.  As I said, more complex
dependencies may be worked in.  The only requirement I'd set up, is
that the semantic imposed should be "at higher level" than "let's have
dependencies among single functions".

Cheers


-- 
Marco Antoniotti ===========================================
PARADES, Via San Pantaleo 66, I-00186 Rome, ITALY
tel. +39 - 06 68 10 03 17, fax. +39 - 06 68 80 79 26
http://www.parades.rm.cnr.it/~marcoxa

Re: what sort of modules should be in clocc

[<do...@co...>]

  >  >> The big difference is that my homegrown tool is part of the thing
  >  >> I'm distributing.  If your macro were defined in each of the
  >  >> packages you distribute I'd say that's just how you decided to
  >  >> implement it.
  > 
  > so you are suggetsing that I should have the same code in each file
  > instead of in one file which I load anyway?!
  > 
  > am I the only one who is not particularly elated by this?

At this point I expect your base file contains only a few macros.  If
there were more in it that would be a better argument for sharing it.
At least it doesn't require anything else, so I could just consider
that file a small overhead for using your modules.  I just want to
make sure that overhead does not snowball.  The general principle is
that each transitively required module has a cost, and you should only
do it if you get enough benefit to justify that cost.

I happen to think the user is better off seeing a declaration and a
normal lisp defining form, but I'm not going to start a religious war
out of that.

Just so I don't feel like we've fallen into a rut and are getting
nowhere, here's the next piece of code to peruse.
Again I justify leaving it in user package on the grounds that it's
not meant to be called by other code or saved into an image, just 
used during development.  (I could still be persuaded to make a new
package for it.)

Let me know if you have any suggestions:

================
;; -*- Mode: LISP; Package: USER; Syntax: Common-lisp -*-
(lisp::in-package "USER")

#|  Record the calls to given functions 
Unlike most function monitoring packages that tell you the number of
calls and total time spent in a function, this records the inputs,
outputs and times of individual calls.  It's meant for helping you to
see WHICH PARTICULAR calls are expensive.  It's also useful for debugging 
in that you can see a history of calls, when they enter and exit, etc.

Notes: 
- This works only on functions.
- It's best to avoid trace, advice, etc. on the functions you want to
  record, since these facilities redefine the function (as does this).
  Just untrace functions before recording and turn off recording before
  you trace them.

 How to use it

  In order to monitor a function one first prepares it for monitoring, then
  one can turn monitoring on and off at high frequency.  One can also reset
  or read the monitoring data for a function.  Finally one can forget about
  monitoring a function.

  *monitored-fns* is a list of functions currently prepared for monitoring.

  (prepare-record-calls '(f1 f2 f3)) prepares the functions named.
    additional keyword arguments: entryforms, exitforms, test
    The entryforms are evaluated at function entry, the exitforms at function
    exit.  This is where you can add things like the current process, 
    a cons counter, or your own application data.
    The results are recorded along with inputs, outputs, entry time,
    exit time and entry sequence number and exit sequence number.
    Test is a form (default is T) that determines whether this particular
    call will be recorded.  It runs in an environment where USER::ARGS
    is bound to the argument list of the function.

    Note that if you redefine a function after it is prepared for recording 
    the preparation is, in effect, undone but this package does not know
    about it.  However you can redo the prepare for it and then all will be
    well.
    
  (record-on '(f1 f2 f3)) turns on recording for these functions.

  (record-off '(f1 f2 f3)) turns it off.

  (initialize-records '(f1 f2 f3)) discards all monitoring data for the
    functions (but does not turn recording off or on and does not forget
    preparation).
    
  (recorded-calls 'f1) returns a list of the call records for f1.
    This is a list of call records (see call-rec below) containing
     inputs outputs start-time end-time entry-sequence# exit-sequence#
	<values of entry forms> <values of exit forms>
    Times are represented as integers - microseconds of run time.
    The sequence numbers are integers that increase with each call to be
    recorded.  The entry (exit) sequence number of one record is less than
    that of another iff the first call was entered (exited) earlier than
    the second.

  (forget-record-calls '(f1 f2 f3))
    discards all monitoring data and preparation

  The expected mode of usage is that you write your own programs to
  do the sorts of analysis you need.  However I do include here a
  few functions that provide a start.  
    
  (summarize-calls '(f1 f2 f3)) prints a summary of the calls.
    This is the traditional number of calls, total time, average.
    The argument defaults to *monitored-fns*.
    Additional optional argument: name-alist
    Name-alist is something like ((f1 . "updating database") (f2 . "waiting"))
    and is used to translate function names into something more meaningful.

  (longest-n-calls 'f2 3) returns the 3 longest recorded calls of f2
   additional keyword arguments: start end filterfn
   filterfn - a function of 1 arg (a call record)
    should return T if the call is "interesting"
   start/end are special cases - filter out anything that starts before start
    or ends after end

  (time-line '(f1 f2 f3) produces an ascii time line of activity
   additional keyword arguments: (width 80) filterfn start end name-alist
   This is handy for getting a pictorial view of when functions start and
   finish.

   Since calls are recorded by pushing onto a list at exit, they are ordered
   by decreasing exit time.  This is handy for finding the outermost calls
   in the case where the calls all come from the same process (and must thus
   be properly nested).
  (outermost (recorded-calls 'foo))
   returns the subset of the calls to foo that are outermost.

  (outline calls ...) prints an outline of a set of calls (presumed to
   be from the same process - who knows what will happen if they don't nest)
   See definition for more arguments.
  
  More sample code is in the comment at the end.

  ** this section de-implemented **
    Both symbolics and TI have a fast short clock and a slow long one.
    We use the fast one on symbolics, slow one on TI.
		      time before wrap around / #usec to read clock
		      --------------------------------------------
		      symbolics 3600           TI explorer II
	       fast   >.5 hour / 67   *        16 sec. / 260
	       slow   >100 yrs / 218           >1 hour / 260   *

    Actually we notice wrap around and record it - whenever a clock access
    returns a smaller value than the previous one we increment a counter.
    Therefore all events are ordered correctly, but if you fail to read the
    clock for an hour or so, it's as if that time never passed.  This is bad
    if you time things on such a coarse scale, but good if you time one thing
    for a minute today and something else for a minute tomorrow - the time
    line between such events never separates them by much more than an hour.
    In practice I don't think this will matter much.
  **

  2/17/89 - try to avoid advice, not so much because it's not commonlisp
  as because it's not compiled!  In fact, I want to be able to turn on and
  off recording at high frequency and encapsulations seem to get in the way
  of this.  For now I'll assume that one does not encapsulate and record the
  same functions.

  1/27/2000 - remove use of pp and update doc in preparation for export
|#

(defvar *monitored-fns* nil)
(defvar *enter-seq-num* 0)
(defvar *exit-seq-num* 0)
;(defvar *clock-cycle* 0)
;(defvar *last-time* 0)

(defstruct call-rec inputs outputs start end enter# exit# entryvals exitvals)

(defun prepare-record-calls (fns &key entryforms exitforms (test t))
  (loop for fn in fns do (prepare-record-call fn entryforms exitforms test)))

; record-calls-fn prop is cons substitute and original fns
(defun prepare-record-call (fn entryforms exitforms test &aux prop)
  (cond ((not (fboundp fn)) (error "no such function as ~A" fn))
	#+zetalisp
	((and (si:function-encapsulated-p fn)
	      (warn "~A is an encapsulation") nil))
	((and (setf prop (get fn 'record-calls-fn))
	      (or (eq (cdr prop) (symbol-function fn))
		  (eq (car prop) (symbol-function fn))))
	 (warn "~A already recorded - doing nothing" fn))
	;; note - if you redefine a function after preparing it
	;; the preparation goes away without us knowing about it
	;; but you can redo the prepare and then all is ok
	(t ; not cached ...
	 (setf (get fn 'record-calls-fn)
	       (cons (make-record-fn fn entryforms exitforms test)
		     (symbol-function fn)))
	 (pushnew fn *monitored-fns*))))

(defun make-record-fn (fn entryforms exitforms test)
  (compile nil
      `(lambda (&rest args &aux start values finish entryvals sn)
	 (if ,test
	     (unwind-protect
		 (progn (setq entryvals (list ,@entryforms)
			      sn (incf *enter-seq-num*)
			      start (microsec-time)
			      ; start1 *clock-cycle*
			      values (multiple-value-list
				       (apply ',(symbol-function fn) args))
			      finish (microsec-time) ; finish1 *clock-cycle*
			      )
			(values-list values))
	       (record-1-call ',fn (copy-list args)
			      (if finish values :abnormal-exit)
			      start ; start1
			      (or finish (microsec-time))
			      ; (or finish1 *clock-cycle*)
			      sn
			      entryvals
			      (list ,@exitforms)))
	     (apply ',(symbol-function fn) args)))))
; perhaps we should try to correct for the time spent in the new function?

(defun forget-record-calls (fns)
  (record-off fns)
  (loop for fn in fns do
    (setq *monitored-fns* (delete fn *monitored-fns*))
    (setf (get fn 'record-calls-fn) nil)
    (setf (get fn 'recorded-calls) nil)))

(defun record-on (fns)
  (loop for fn in fns do
        (let ((prop (get fn 'record-calls-fn)))
	  (cond ((not prop) (cerror "skip turning on recording"
				    "~A not prepared for recording" fn))
		((eq (cdr prop) (symbol-function fn))
		 (setf (symbol-function fn) (car prop)))
		;; if already on then it's a noop
		((eq (car prop) (symbol-function fn)))
		(t (cerror "skip turning on recording"
			   "~A has changed since last prepared for recording"
			   fn))))))

(defun record-off (fns)
  (loop for fn in fns do
        (let ((prop (get fn 'record-calls-fn)))
	  (cond ((not prop)
		 (cerror "continue" "~A not prepared for recording" fn))
		((eq (car prop) (symbol-function fn))
		 (setf (symbol-function fn) (cdr prop)))
		;; if already off then it's a noop
		((eq (cdr prop) (symbol-function fn)))
		(t (cerror "continue"
			   "~A has changed since recording last turned on"
			   fn))))))

(defun microsec-time ()
  ;; hmm - might end up using rational arithmetic if internal...second
  ;; does not divide a million ...
  (* (get-internal-run-time) (/ 1000000 internal-time-units-per-second)))

#+ignore ;; old version for lisp machine microsecond clocks
(defun microsec-time (&aux time)  
  (setq time
        #-(or symbolics ti)
	(* (get-internal-run-time) (/ 1000000 internal-time-units-per-second))
	#+symbolics (time:fixnum-microsecond-time)
	#+TI (time:microsecond-time))
  (when (< time *last-time*) (incf *clock-cycle*))
  (setf *last-time* time))

(defun record-1-call (fn inputs results t1 t2 seqno entryvals exitvals)
  (push (make-call-rec :inputs inputs :outputs results :start t1 :end t2
		       :enter# seqno :exit# (incf *exit-seq-num*)
		       :entryvals entryvals :exitvals exitvals)
	(get fn 'recorded-calls)))

(defun initialize-records (fns)
  (loop for fn in fns do (setf (get fn 'recorded-calls) nil)))

(defun recorded-calls (fn) (get fn 'recorded-calls))

(defun summarize-calls (&optional (fns *monitored-fns*) name-alist)
  (loop for fn in fns do
    (summarize-record fn (get fn 'recorded-calls) name-alist)))

(defun summarize-record (fn calls name-alist)
  (when calls (loop for x in calls sum 1 into ncalls
		    sum (elapsed (call-rec-start x) (call-rec-end x))
		    into time finally
		    (print-summarize-record fn ncalls time name-alist))))

(defun print-summarize-record (fn ncalls time name-alist)
  (multiple-value-bind (total tunits)
      (standardized-time-units time)
    (multiple-value-bind (avg aunits)
      (standardized-time-units (float (/ time ncalls)))
      (format *standard-output* "~%~A: ~A calls, ~A ~A (avg. ~A~:[ ~a~; ~])"
	  (or (cdr (assoc fn name-alist)) fn)
	  ncalls total tunits avg (eq aunits tunits) aunits))))

(defun standardized-time-units (usec)
  (cond ((> usec 999999) (values (float (/ usec 1000000)) "sec."))
	((> usec 999) (values (float (/ usec 1000))  "msec."))
	(t (values usec "usec."))))

(defun elapsed (t1 t2) (- t2 t1))

#+ignore ;; again, lisp machine version 
(defun elapsed (t1 t11 t2 t21)
  (+ (- t2 t1) (* (- t21 t11) (* 1024 1024 2048 #+TI 2))))

(defun longest-n-calls (fn n &key start end filterfn
			&aux next time current
			(candidates (recorded-calls fn)) (i 0))
  ; filterfn decides whether a record is "interesting"
  ; special cases: start/end filters out anything that starts before start
  ; or ends after end
  (flet ((filter (e) (and (or (null start)
			      (plusp (elapsed start (call-rec-start e))))
			  (or (null end)
			      (plusp (elapsed (call-rec-end e) end)))
			  (or (null filterfn) (funcall filterfn e)))))
    (loop while (and (< i n) (setq next (pop candidates)))
	  when (filter next)
	  do (incf i)
	     (push (cons (elapsed (call-rec-start next) (call-rec-end next))
			 next) current))
    (setq current (sort current #'<= :key #'car))
    (loop while (setq next (pop candidates))
	  when (filter next)
	  when (< (caar current)
		  (setq time (elapsed (call-rec-start next)
				      (call-rec-end next))))
	  do (setq current (merge 'list (cdr current)
				  (list (cons time next))
				  #'<= :key #'car)))
    (nreverse current)))

(defmacro totalt (x) x)
#+ignore 
; get the time represented by the two numbers x (low order) and y (high order)
(defun totalt (x y) (elapsed 0 0 x y))

(defvar *time-line-key*
  "Start time = ~A, End time = ~A, Width = ~A, ~
  ~& each column represents ~A ~A~
  ~& Key: ( = 1 entry, ) = 1 exit, * = more than one entry/exit~
  ~&      if no entry/exit, a digit indicates number of active calls,~
  ~&         blank indicates no change, + indicates >9 ~% ")

(defun time-line (fns &key (width 80) filterfn start end len name-alist
		    &aux events)
  (flet ((filter (e) (and (or (null start)
			      (plusp (elapsed start (call-rec-start e))))
			  (or (null end)
			      (plusp (elapsed (call-rec-end e) end)))
			  (or (null filterfn) (funcall filterfn e)))))
    (setq events (loop for f in fns collect
	 	       (cons f (loop for e in (recorded-calls f)
				     when (filter e) collect e))))
    (unless (and start end)
      (loop for e in events do
	    (loop for r in (cdr e) do
	          (when (or (null start)
			    (minusp (elapsed start (call-rec-start r))))
		    (setq start (totalt (call-rec-start r))))
		  (when (or (null end)
			    (minusp (elapsed (call-rec-end r) end)))
		    (setq end (totalt (call-rec-end r)))))))
    (when (and start end) (setq len (- end start)))
    (unless (and len (> len 0)) (return-from time-line "empty interval"))
    (multiple-value-bind (number unit)
	(when (and start end width)
	  (standardized-time-units (/ (- end start 0.0) width)))
      (apply #'concatenate 'string
	     (format nil *time-line-key* start end width number unit)
	     (loop for f in events collect
		   (concatenate 'string
		       (let ((string (make-string width
						  :initial-element #\space))
			     index
			     (countstart
			       (make-array (list width)
					   :initial-element 0
					   :element-type 'integer))
			     (countend
			       (make-array (list width) :initial-element 0
					   :element-type 'integer)))
			 (loop for e in (cdr f) do
			   (setq index
				 (min (1- width)
				      (floor (* width (/ (- (totalt
							     (call-rec-start e))
							    start)
							 len)))))
			   (incf (aref countstart index))
			   (setf (aref string index)
				 (if (char= #\space (aref string index))
				     #\( #\*))
			   (setq index
				 (min (1- width)
				      (floor (* width (/ (- (totalt
							     (call-rec-end e))
							    start)
							 len)))))
			   (decf (aref countend index))
			   (setf (aref string index)
				 (if (char= #\space (aref string index))
				     #\) #\*)))
			 (loop for i below width with sum = 0 do
			   (setf sum (+ sum (aref countstart i)
					(aref countend i)))
			   (when (and (/= i 0)
				      (/= (aref countstart (1- i)) 0)
				      (/= (aref countend (1- i)) 0)
				      (char= #\space (aref string i))
				      (> sum 0))
			     (setf (aref string i)
				   (if (> sum 9) #\+ (aref "0123456789" sum)))))
			 string)
		       (format nil "  ~A~& "
			       (symbol-name (or (cdr (assoc (car f) name-alist))
						(car f))))))))))

; relies on being sorted by decreasing exit time
(defun outermost (calls &aux outer)
  ; change to use sequence numbers instead of time
  (loop for c in calls
	unless (and outer
		    (<= (call-rec-enter# outer) (call-rec-enter# c)))
	collect (setf outer c)))


#| While I'm at it, perhaps a call tree would be nice:
((call1 (call2 (call3 call4) call5) call6) call7 call8)
or even printing an outline:
call1
  call2
    call3
      call4
    call5
  call6
call7
call8

In such a call outline, we want to print records in entry order, with
indentation proportional to the number of other records that both
entered earlier and exited later.
|#

; this seems to come up often
(defun call-time (x)
  (/ (- (call-rec-end x) (call-rec-start x)) 1000000.0))

(defun default-show-record (x depth)
  ; list structure that shows what we think we'll want to see
 (list (call-time x)
       :depth depth ; useful for searching the output
       ; useful for limiting the scope of later outlines
       :enter# (call-rec-enter# x) :exit# (call-rec-exit# x)
       (call-rec-inputs x)))

;; (require :pp) ;; just use the printer of your choice
(defun outline (calls &key
		      (stream *standard-output*) (rmargin 80)
		      (printer 'princ)
		      (indentation 3) (showfn #'default-show-record)
		      (max-depth 15) ; don't go deeper than this
		      &aux (sorted (sort (copy-list calls) #'<=
					 :key #'call-rec-enter#))
		      open depth nspaces)
  (loop for c in sorted do
	(setf open
	      (loop for o in open when
		    (and (< (call-rec-enter# o) (call-rec-enter# c))
			 (< (call-rec-exit# c) (call-rec-exit# o)))
		    collect o))
	(setf depth (length open))
	(if (> depth max-depth)
	  (princ "."stream) ; just to indicate that something's there
	  (progn
	   (terpri stream)
	   (setf nspaces (mod (* indentation depth) (/ rmargin 2)))
	   ; don't space out too far
	   (loop for i below nspaces do (princ " " stream))
	   ;;(moveto stream 1 nspaces) ;; from pp
	   (funcall printer (funcall showfn c depth) stream)))
	(push c open)))

(defun call-inside (c1 c2) ; c1 is inside c2
  (and (< (call-rec-enter# c2) (call-rec-enter# c1))
       (< (call-rec-exit# c1)  (call-rec-exit# c2))))

; useful for filtering sets of records
(defun calls-between (calls &key (enter 0) (exit 999999))
  (loop for c in calls
	when (and (<= enter (call-rec-enter# c)) (<= (call-rec-exit# c) exit))
	collect c))

#|
 sample code for finding the "internal" time for each call, i.e., the
 time not included in any call made by that one:

 ; all calls
 (length (setf all (loop for x in *monitored-fns* append
			 (recorded-calls x))))
 ; sorted so outermost works on it
 (length (setf all (sort (copy-list all) #'>= :key #'call-rec-exit#)))
 ; compute internal time (stored in exitvals)
 (loop for a in all do
       (setf (call-rec-exitvals a)
	     (- (call-time a)
		(loop for c in
		      (outermost
		       (cdr (calls-between all
				 :enter (call-rec-enter# a)
				 :exit (call-rec-exit# a))))
		      sum (call-time c)))))
 ; sorted by internal time
 (length (setf z (sort (copy-list all) #'>= :key #'call-rec-exitvals)))

 ; while I'm at it, store the function name in the entryvals
 (loop for f in *monitored-fns* do
       (loop for c in (recorded-calls f) do
	     (setf (call-rec-entryvals c) f)))
 ; show function to highlight internal time
 (defun show-call (x &optional depth)
  (list (call-rec-exitvals x) ; internal time
	(call-rec-entryvals x) ; fn
	; useful for limiting the scope of later outlines
	:enter# (call-rec-enter# x) :exit# (call-rec-exit# x)
	:start (/ (call-rec-start x) 1000000.0)
	:end (/ (call-rec-end x) 1000000.0)
	:total-time (call-time x)
	(call-rec-inputs x)))
 ; show top 10
 (loop for i below 10 as a in z do (terpri)(princ (show-call a)))
|#

Re: what sort of modules should be in clocc

[Sam S. <sd...@gn...>]

>>>> In message <200...@is....>
>>>> On the subject of "Re: what sort of modules should be in clocc"
>>>> Sent on Fri Jan 28 16:04:48 EST 2000
>>>> Honorable Don Cohen <do...@co...> writes:
 >> 
 >> The big difference is that my homegrown tool is part of the thing
 >> I'm distributing.  If your macro were defined in each of the
 >> packages you distribute I'd say that's just how you decided to
 >> implement it.

so you are suggetsing that I should have the same code in each file
instead of in one file which I load anyway?!

am I the only one who is not particularly elated by this?

-- 
Sam Steingold (http://www.podval.org/~sds)
Micros**t is not the answer.  Micros**t is a question, and the answer is Linux,
(http://www.linux.org) the choice of the GNU (http://www.gnu.org) generation.
Parachute for sale, once used, never opened, small stain.

Re: what sort of modules should be in clocc

[Stefan <st...@hi...>]

Hi everybody,

Although the fact holds true that if I talk about DEFSYSTEM
is like as if a  blind person talks about color ...
I would like to give you some stimulus on how we could take
advantage out of the current situation:

On Fri, 28 Jan 2000, Don Cohen wrote:
> you need.  I don't know much about mk defsystem (other than what I've
> read in the last hour), but I'm willing to bet right now that it does
> not have all that I'd need to replace the homegrown defsystems that
> I've been maintaining.

You could mail the maintainer of mk:defsystem and ask him, if the features
that you depend on are implemented in mk:defsystem.

Ask him repeatedly what features are present in defsystem until he gives
you all the details you need to know. Along the way one of you could write
a documentation on the features, that are present and which should be
included on a wish - list ( feature-request).
Marco is currently working on the next version
of defsystem. You could ask him if some of your wishes could be supported
by the next version. He could tell you how your wishes fit into the
general capabilities of mk:defsystem

He could also explain you if your feature-requests are orthogonal
extensions of defsystem or if they interfere with his project development
plan in a unpleasant way ( duplication; overlapping goals but
different design philosophies)

You should describe the requirements of your code on a building system in
a formal way ( which means, bare of any coding details in your source)

We seem to have a really hard interface question here that needs to be
settled.

The first step should be to acquire information on the view point of  the
other developer, so that everyone knows what is going on.

Bye,

Stefan

Re: what sort of modules should be in clocc

[<do...@co...>]

  >  >> Do you imagine that there is no cost to replacing something that
  >  >> already works?  Generally it takes a considerable effort even to
  > 
  > compare this to the effort of maintaining a separate facility.

That's just what I'm doing.  At the moment that cost is zero, and it's
likely to remain zero if the code is not changing much, which is the
case for most of the code I expect to put into clocc.
If some change becomes necessary that requires an expensive change to
this separate facility, that's the time to consider replacing it.

  > I find your position inconsistent: you worry about an extra macro in my
  > code (which I use throughout and which is generally useful) and you are
  > willing to hoist upon your users your own homegrown tool which
  > duplicates a well-established and widely known facility.

The big difference is that my homegrown tool is part of the thing I'm
distributing.  If your macro were defined in each of the packages 
you distribute I'd say that's just how you decided to implement it.
(I still prefer to see defvar all over the place instead of defcustom,
but that's another argument.)

So where's the doc for mk:defsystem ?

Re: what sort of modules should be in clocc

[Sam S. <sd...@gn...>]

>>>> In message <200...@is....>
>>>> On the subject of "Re: what sort of modules should be in clocc"
>>>> Sent on Fri Jan 28 15:22:59 EST 2000
>>>> Honorable Don Cohen <do...@co...> writes:
 >> 
 >>   >  >> There are plenty of large projects with their own make programs.
 >>   > this is BAD.
 >> I disagree.  I think it's fine, for instance, that cl-http comes with
 >> its own build-cl-http function.  I don't know or care how it's implemented.

yeah, but then you have to figure out how to call the function, what the
args are &c.  when a unix package comes with ./configure, I know what to
do right away.

 >> Do you imagine that there is no cost to replacing something that
 >> already works?  Generally it takes a considerable effort even to

compare this to the effort of maintaining a separate facility.

 >> determine that the replacement actually has all the capabilities
 >> that you need.  I don't know much about mk defsystem (other than
 >> what I've read in the last hour), but I'm willing to bet right now
 >> that it does not have all that I'd need to replace the homegrown
 >> defsystems that I've been maintaining.

I find your position inconsistent: you worry about an extra macro in my
code (which I use throughout and which is generally useful) and you are
willing to hoist upon your users your own homegrown tool which
duplicates a well-established and widely known facility.

-- 
Sam Steingold (http://www.podval.org/~sds)
Micros**t is not the answer.  Micros**t is a question, and the answer is Linux,
(http://www.linux.org) the choice of the GNU (http://www.gnu.org) generation.
My other CAR is a CDR.

Re: what sort of modules should be in clocc

[<do...@co...>]

I thought I saw defsystem doc in clocc before but now I don't.
The source mentions
;;; For more information, see the documentation and examples in 
;;; lisp-utilities.ps.

  >  >> There are plenty of large projects with their own make programs.
  > this is BAD.
I disagree.  I think it's fine, for instance, that cl-http comes with
its own build-cl-http function.  I don't know or care how it's implemented.

  > there certainly IS a reason to drop your own home-grown tools when there
  > is a standard one available which does the job better.
  > do you make your car yourself or buy it from a well-established vendor?

Do you imagine that there is no cost to replacing something that
already works?  Generally it takes a considerable effort even to
determine that the replacement actually has all the capabilities that
you need.  I don't know much about mk defsystem (other than what I've
read in the last hour), but I'm willing to bet right now that it does
not have all that I'd need to replace the homegrown defsystems that
I've been maintaining.

  >  >> Everyone who wants to use any code from clocc has to download all of
  >  >> it?  I hope not!  Even if this were a trivial and cheap operation,
  >  >> I think it's best to give people a way to grab only what they need.
  > 
  > fine - you can d/l any part you want, just like with CPAN (BTW, CLAN is
  > another good name for CLOCC), but you should be prepared to be told that
  > package X needs Y.

  >  >> I don't follow this at all.  I'm only suggesting that the code you
  >  >> offer for public libraries not make use of that extensibility
  >  >> unnecessarily.  As much as possible each thing should stand alone.
  > 
  > I wrote `defcustom' because I wanted this extension to write other
  > code.  if you don't like it, you don't have to use it.
  > But I wrote it ***NOT*** for the other to use (although I welcome others
  > who would like to use it), but to make ***MY*** development easier.
  > IMO, your "stand alone" thing is either trivial (when taken moderately)
  > or contrary to the idea of code reuse (when taken as you seem to imply). 

There are things that are reasonable in your own code that you ought
to not put in code for export.  Getting rid of those is an extra cost
of preparing code for export.  In the case of defcustom you should ask
yourself whether this is helping the user (who downloads your code) or
not.  I think it's easier for him to see the macroexpansion than to
see the defcustom.  And it's easier for him to see defcustom if the
definition is in the same file/module/system than if it is in another.

I think we can agree that this is a matter of degree.  The extreme
that I fear is that I want a few functions from a few files, but these
use other files and those use other files ...  I eventually get them
all and then run into errors, e.g., maybe there are incompatibilities
in the versions I got, or maybe there are incompatibilities between
two different things I want to put together, or one of those things
doesn't work cause of a bug in the lisp implementation I'm using.  Now
I have to understand all that code.  Or more likely, I start from what
I want and trace the dependencies and throw out all that I don't need
in hopes that the errors come from stuff I didn't need anyway.

  > You seem to be unaware of the power of defsystem.
  > You can have 50 files in the library X and the package Y might need
  > the function BAR which in is in the file X/bar.lsp.
  > When you tell defsystem that Y depends on X/bar.lsp, it can figure out
  > which files in X it needs to load and it will not load all 50 files if
  > it can help it.

You're right that I do not know about MK defsystem.  However, the
comment below from the source suggests that this ability is not even 
available.  I suspect this refers to modules within one system, and
that such dependencies between parts of different systems is even
further out of the question.

More to the point, though, even if you could do this, the dependencies
are only among files, right?  Or is there a model of dependency among
individual top level forms?  (We used to have something like this in
a project I worked on, but I've never seen it anywhere else.)
I worry that something I don't even use in bar.lsp will need something
in another file, and so on, so that I end up getting way too much.

;;; Current dependencies are limited to siblings. Maybe we should allow
;;; nephews and uncles? So long as it is still a DAG, we can sort it.
;;; Answer: No. The current setup enforces a structure on the modularity.
;;; Otherwise, why should we have modules if we're going to ignore it?

Re: what sort of modules should be in clocc

[Sam S. <sd...@gn...>]

>>>> In message <200...@is....>
>>>> On the subject of "Re: what sort of modules should be in clocc"
>>>> Sent on Fri Jan 28 12:20:11 EST 2000
>>>> Honorable Don Cohen <do...@co...> writes:
 >> 
 >> This is the first I ever heard that lack of >'s made mail hard to

you see, I don't want to read my own mail to which you are replying.
indentation alone provides too poor medium of separation of what you
wrote from what I wrote, because. e.g., indentation is sometimes used to
separate paragraphs (and even if you don't use it like this, my eye is
still trained that it might happen, so it spends time to figure out
whether it is the case).

 >> read.  Is this better?  I personally prefer just the indentation,
 >> but of course, I'm not writing this for ME to read.

yes, this is much better.  thanks.

 >> There are plenty of large projects with their own make programs.

this is BAD.

on UNIX, there have been plenty of mechanism for platform detection,
including xmkmf, user editing makefiles, custon scripts &c.  now that
there is GNU Autoconf, each and every UNIX program of reasonable size
should use it.

same with lisp: now that mk:defsystem comes with a reasonable copyright,
there is absolutely no excuse for using home-grown make.

 >> Surely there's no reason to rewrite those.

there certainly IS a reason to drop your own home-grown tools when there
is a standard one available which does the job better.
do you make your car yourself or buy it from a well-established vendor?

 >> Surely you wouldn't reject a package just cause it came with its own
 >> build procedure.

sure, but we will encourage the author to use mk:defsystem, since this
improves interoperability and reduces code duplication.

 >> Also, I can't believe that any particular build procedure is best
 >> for all purposes.  It seems to me that the strongest statement
 >> justified here is that you should at least consider this one before
 >> you go off and write your own.

of course.  and if you already have your own, but mk:defsystem is
adequate for your purposes, you should use it.

 >> Everyone who wants to use any code from clocc has to download all of
 >> it?  I hope not!  Even if this were a trivial and cheap operation,
 >> I think it's best to give people a way to grab only what they need.

fine - you can d/l any part you want, just like with CPAN (BTW, CLAN is
another good name for CLOCC), but you should be prepared to be told that
package X needs Y.

 >> I don't follow this at all.  I'm only suggesting that the code you
 >> offer for public libraries not make use of that extensibility
 >> unnecessarily.  As much as possible each thing should stand alone.

I wrote `defcustom' because I wanted this extension to write other
code.  if you don't like it, you don't have to use it.
But I wrote it ***NOT*** for the other to use (although I welcome others
who would like to use it), but to make ***MY*** development easier.
IMO, your "stand alone" thing is either trivial (when taken moderately)
or contrary to the idea of code reuse (when taken as you seem to imply). 

 >> I gather just putting a file in a public place with no copyright 
 >> or proprietary claims is not sufficient to make it PD.

absolutely true.  [IANAL!]

 >> What should I do if I do want to make a file PD?

Place something like this in the header:
"I, Don Cohen <do...@co...>, the author of this code, am hereby
placing it in the Public Domain."

I suggest that you at least consider a more restrictive license, like
LGPL.

 >> To me this is a matter of degree.
 >> If defsystem needs only a small amount of stuff from the string
 >> library and the string library is large (and even more so if the
 >> string library in turn requires other libraries!) I'd rather just copy
 >> what I need into defsystem.  My assumption is that this is not a big
 >> maintenance problem - the function in the string library is not likely
 >> to change often, and if it does, the old version will continue to work
 >> fine for defsystem.  A small amount of duplicated code (in a different
 >> package) is preferred over a package dependency.

You seem to be unaware of the power of defsystem.
You can have 50 files in the library X and the package Y might need
the function BAR which in is in the file X/bar.lsp.
When you tell defsystem that Y depends on X/bar.lsp, it can figure out
which files in X it needs to load and it will not load all 50 files if
it can help it.

-- 
Sam Steingold (http://www.podval.org/~sds)
Micros**t is not the answer.  Micros**t is a question, and the answer is Linux,
(http://www.linux.org) the choice of the GNU (http://www.gnu.org) generation.
Marriage is the sole cause of divorce.

Re: what sort of modules should be in clocc

[Raymond T. <to...@rt...>]

>>>>> "Don" == Don Cohen <do...@co...> writes:

First, welcome aboard!

    Don> There are plenty of large projects with their own make programs.
    Don> Surely there's no reason to rewrite those.  Surely you wouldn't
    Don> reject a package just cause it came with its own build procedure.

I'm pretty sure such a package wouldn't be rejected.  It was just
strongly suggested to use defsystem if possible.

    Don> Everyone who wants to use any code from clocc has to download all of
    Don> it?  I hope not!  Even if this were a trivial and cheap operation, 
    Don> I think it's best to give people a way to grab only what they need.

I used to care about these things but not anymore.  There are better
things to do than to try to figure out the minimal set of things I
need to install something.  I just get it all.

    Don> I gather just putting a file in a public place with no copyright or
    Don> proprietary claims is not sufficient to make it PD.  What should I
    Don> do if I do want to make a file PD?

Well you can do what CMUCL does.  Every file says:

;;; This code was written as part of the CMU Common Lisp project at
;;; Carnegie Mellon University, and has been placed in the public domain.

Don't know if this is legally binding or not, but it seems pretty
clear cut.

    Don> To me this is a matter of degree.  
    Don> If defsystem needs only a small amount of stuff from the string
    Don> library and the string library is large (and even more so if the
    Don> string library in turn requires other libraries!) I'd rather just copy
    Don> what I need into defsystem.  My assumption is that this is not a big

I think defsystem should be as self-contained as possible since it's
intended to be used to build "everything".

Ray

Re: what sort of modules should be in clocc

[<do...@co...>]

replies to several messages combined ...

  > Please use something in addition to indentation to separate your replies
  > from the original article.  Your posts are hard to read.
  > If you use Emacs, you might want to investigate `mail-yank-prefix'.

This is the first I ever heard that lack of >'s made mail hard to read.
Is this better?  I personally prefer just the indentation, but of
course, I'm not writing this for ME to read.

  > MK:DEFSYSTEM is a 'make' for Common Lisp.  It is one key piece of
  > technology for the whole CL community and therefore it should be used
  > by CL projects of a given size.
  > 
  > Of course, "size does matter". :)  If you have a single file utility,
  > there is no real problem to provided it "as is", but for projects of
  > larger size you need to have separate files, which means that you
  > quickly end up needing a 'make', hence MK:DEFSYSTEM.
  >
  > For the CLOCC, I believe there is a consensus that your code should
  > not come with a 'defsystem' different from MK:DEFSYSTEM, that's all.
  > Not because MK:DEFSYSTEM is better than other ones, but because it
  > reasonably runs on *most* CL implementations.

There are plenty of large projects with their own make programs.
Surely there's no reason to rewrite those.  Surely you wouldn't
reject a package just cause it came with its own build procedure.
Also, I can't believe that any particular build procedure is best
for all purposes.  It seems to me that the strongest statement 
justified here is that you should at least consider this one before
you go off and write your own.

  >  >> I'd prefer all your packages not to use things like your defcustom
  >  >> since it just makes the reader go back and read a bunch of
  >  >> additional definitions.  If there are just a few small things then
  >  >> I'd rather have them repeated than have to get an additional file.
  > 
  > defsystem solves the problem for you - it loads everything you need.
  > if by "get" you mean "download", then forget it - the user is expected
  > to download the whole thing anyway.

Everyone who wants to use any code from clocc has to download all of
it?  I hope not!  Even if this were a trivial and cheap operation, 
I think it's best to give people a way to grab only what they need.

  > "Lisp grows to meet your needs".  If we heed your recommendations, we
  > will be giving up the most important strength of Lisp - extensibility.

I don't follow this at all.  I'm only suggesting that the code you
offer for public libraries not make use of that extensibility
unnecessarily.  As much as possible each thing should stand alone.


  >  >>   and you will never be able to prevent me from doing it (I am not sure
  >  >>   I can (c) what you wrote, but I can certainly base a proprietary
  >  >>   system on it).
  >  >> 
  >  >> What's wrong with that?
  > 
  > Some people do not like the idea of others using their work and not
  > giving anything back.

I gather just putting a file in a public place with no copyright or
proprietary claims is not sufficient to make it PD.  What should I
do if I do want to make a file PD?

  >  >> Let's take one egregious example: SPLIT-STRING or STRING-SPLIT, or
  >  >> whatever.
  >  >> Unfortunately this is not in CL and many packages (MK:DEFSYSTEM
  >  >> included) provide a special version of it, hence reinventing the
  >  >> wheel.
  >  >> 
  >  >> Now the question is: how should CLOCC address this problem?  What
  >  >> about MK:DEFSYSTEM.
  >  >> 
  >  >> I'll tell you how I'd proceeed.  I'd set up a package CL.EXT.STRINGS
  >  >> which contained extensions to the string handling stuff of CL, and put
  >  >> SPLIT-STRING in it.  Then and only then, I would and could give up the
  >  >> special SPILT-STRING in MK:DEFSYSTEM.

To me this is a matter of degree.  
If defsystem needs only a small amount of stuff from the string
library and the string library is large (and even more so if the
string library in turn requires other libraries!) I'd rather just copy
what I need into defsystem.  My assumption is that this is not a big
maintenance problem - the function in the string library is not likely
to change often, and if it does, the old version will continue to work
fine for defsystem.  A small amount of duplicated code (in a different
package) is preferred over a package dependency.  

  > Apart from that there is another thing I would like to stress out.
  > Whatever ends up in CLOCC should strive to run in as many CL
  > implementations as possible.

Certainly everyone will agree with that.
One good way to achieve that is to restrict your contributed code to ansi
CL wherever possible. 

Re: what sort of modules should be in clocc

[Sam S. <sd...@gn...>]

>>>> In message <200...@co...>
>>>> On the subject of "Re: what sort of modules should be in clocc"
>>>> Sent on Fri Jan 28 05:34:32 EST 2000
>>>> Honorable Marco Antoniotti <ma...@pa...> writes:
 >> 
 >> Let's take one egregious example: SPLIT-STRING or STRING-SPLIT, or
 >> whatever.
 >> Unfortunately this is not in CL and many packages (MK:DEFSYSTEM
 >> included) provide a special version of it, hence reinventing the
 >> wheel.
 >> 
 >> Now the question is: how should CLOCC address this problem?  What
 >> about MK:DEFSYSTEM.
 >> 
 >> I'll tell you how I'd proceeed.  I'd set up a package CL.EXT.STRINGS
 >> which contained extensions to the string handling stuff of CL, and put
 >> SPLIT-STRING in it.  Then and only then, I would and could give up the
 >> special SPILT-STRING in MK:DEFSYSTEM.

perfect!  great!  thanks!
I have plenty of string handling stuff in cllib/util.lsp - please merge
yours with mine when you create CL.EXT.STRINGS!
then I'll drop it from cllib/util.lsp!

-- 
Sam Steingold (http://www.podval.org/~sds)
Micros**t is not the answer.  Micros**t is a question, and the answer is Linux,
(http://www.linux.org) the choice of the GNU (http://www.gnu.org) generation.
non-smoking section in a restaurant == non-peeing section in a swimming pool

Re: Cannot cvs to sourceforge....

[Marco A. <ma...@pa...>]

> Delivery-Date: Fri, 28 Jan 2000 15:32:00 +0100
> From: Bruno Haible <ha...@il...>
> Date: Fri, 28 Jan 2000 15:31:19 +0100 (MET)
> Cc: clo...@li...
>
> I think you must give your ssh key (contents of identity.pub) into the
> appropriate web form, and then wait 6 hours so the thing gets propagated
> to the correct machine.

Oooops.  I missed the "Keys" section in the "Login Maintainance" page.
Thanks

Cheers


-- 
Marco Antoniotti ===========================================
PARADES, Via San Pantaleo 66, I-00186 Rome, ITALY
tel. +39 - 06 68 10 03 17, fax. +39 - 06 68 80 79 26
http://www.parades.rm.cnr.it/~marcoxa

Re: what sort of modules should be in clocc

[Sam S. <sd...@gn...>]

Please use something in addition to indentation to separate your replies
from the original article.  Your posts are hard to read.
If you use Emacs, you might want to investigate `mail-yank-prefix'.

>>>> In message <200...@is....>
>>>> On the subject of "what sort of modules should be in clocc"
>>>> Sent on Thu Jan 27 17:55:17 EST 2000
>>>> Honorable Don Cohen <do...@co...> writes:
 >> 
 >> I'd prefer all your packages not to use things like your defcustom
 >> since it just makes the reader go back and read a bunch of
 >> additional definitions.  If there are just a few small things then
 >> I'd rather have them repeated than have to get an additional file.

defsystem solves the problem for you - it loads everything you need.
if by "get" you mean "download", then forget it - the user is expected
to download the whole thing anyway.

"Lisp grows to meet your needs".  If we heed your recommendations, we
will be giving up the most important strength of Lisp - extensibility.

 >>      and you will never be able to prevent me from doing it (I am not sure
 >>      I can (c) what you wrote, but I can certainly base a proprietary
 >>      system on it).
 >> 
 >> What's wrong with that?

Some people do not like the idea of others using their work and not
giving anything back.

-- 
Sam Steingold (http://www.podval.org/~sds)
Micros**t is not the answer.  Micros**t is a question, and the answer is Linux,
(http://www.linux.org) the choice of the GNU (http://www.gnu.org) generation.
"Syntactic sugar causes cancer of the semicolon."	-Alan Perlis

Re: Cannot cvs to sourceforge....

[Bruno H. <ha...@il...>]

Marco Antoniotti writes:
> 
> Ok...  here is what I have
> 
> 1> cvs -d ma...@cv...:/cvsroot/clocc checkout clocc
> Host key not found from the list of known hosts.
> Are you sure you want to continue connecting (yes/no)? yes
> Host 'cvs.clocc.sourceforge.net' added to the list of known hosts.
> ma...@cv...'s password: <my sourcefoge passwd here>
> 
> Permission denied.
> cvs [checkout aborted]: end of file from server (consult above messages if any)
> 2> 

I think you must give your ssh key (contents of identity.pub) into the
appropriate web form, and then wait 6 hours so the thing gets propagated
to the correct machine.

Bruno

Cannot cvs to sourceforge....

[Marco A. <ma...@pa...>]

Ok...  here is what I have

1> cvs -d ma...@cv...:/cvsroot/clocc checkout clocc
Host key not found from the list of known hosts.
Are you sure you want to continue connecting (yes/no)? yes
Host 'cvs.clocc.sourceforge.net' added to the list of known hosts.
ma...@cv...'s password: <my sourcefoge passwd here>

Permission denied.
cvs [checkout aborted]: end of file from server (consult above messages if any)
2> 

Is my public key in place?

Any idea about what I may be doing wrong?

Cheers

-- 
Marco Antoniotti ===========================================
PARADES, Via San Pantaleo 66, I-00186 Rome, ITALY
tel. +39 - 06 68 10 03 17, fax. +39 - 06 68 80 79 26
http://www.parades.rm.cnr.it/~marcoxa

Re: what sort of modules should be in clocc

[Marco A. <ma...@pa...>]

> Delivery-Date: Thu, 27 Jan 2000 23:47:25 +0100
> Date: Thu, 27 Jan 2000 14:47:07 -0800
> From: do...@co... (Don Cohen)
> CC: st...@hi..., clo...@li...
> Sender: clo...@li...
> X-Mailman-Version: 1.1
> Precedence: bulk
> List-Id:  <clocc-devel.lists.sourceforge.net>
> X-BeenThere: clo...@li...
> 
>    >> A comment on one of the requirements for code in this repository:
>    >>     Self-contained, i.e. does not require packages not in this
>    >>     repository,
>    >> I suggest that even reliance on this repository be minimized.
> 
>   NO WAY!!!!
>   then each and every package in CLOCC will reimplement the same stuff
>   over and over again.
>   I suggest mandatory support of defsystem by each package - this way we
>   can more or less be sure that people can get what they want to compile. 
> 
> I'm not arguing that there should be NO dependency.  For instance,
> anything that uses resources outside of lisp will probably use the
> cross-implementation stuff.  Also, cases where one facility clearly
> depends on another are fine.  I just don't want a tangle of packages
> where in order to get one simple thing you end up getting 10 more that
> seem to be unrelated.  For instance, the record-calls package that I
> just looked at contained a call to my pretty printer.  I've removed
> that and replaced it with a print-function parameter.  I'd prefer all
> your packages not to use things like your defcustom since it just
> makes the reader go back and read a bunch of additional definitions.
> If there are just a few small things then I'd rather have them
> repeated than have to get an additional file.
> 
> What does mandatory support of defsystem mean?  You don't want any
> file that can simply be loaded?

MK:DEFSYSTEM is a 'make' for Common Lisp.  It is one key piece of
technology for the whole CL community and therefore it should be used
by CL projects of a given size.

Of course, "size does matter". :)  If you have a single file utility,
there is no real problem to provided it "as is", but for projects of
larger size you need to have separate files, which means that you
quickly end up needing a 'make', hence MK:DEFSYSTEM.

For the CLOCC, I believe there is a consensus that your code should
not come with a 'defsystem' different from MK:DEFSYSTEM, that's all.
Not because MK:DEFSYSTEM is better than other ones, but because it
reasonably runs on *most* CL implementations.

> What's wrong with that?
> 
>   4. I suggest GPL for "end user code" and LGP for what other programmers
>      will want to use.
> will look ...
> 
>    >> A few other suggestions:
>    >> - Each piece of code, as much as possible should do one thing, so you
>    >> can take what you need without having to get a lot of extra stuff.
> 
>   do you want 100000 different files? :-)
>   memory is cheap.
> 
> This is not a matter of memory.  It's a matter of what the user of
> this archive has to do.  I want him to be able to read and understand 
> the code.  If there's extra code that he doesn't need, well, ok, maybe
> that doesn't do much harm, but if that code requires other packages he
> has to either realize that he doesn't need those other packages and
> modify the source to get rid of them, or he has to get these other
> packages.  That's what I really want to avoid.

Let's take one egregious example: SPLIT-STRING or STRING-SPLIT, or
whatever.
Unfortunately this is not in CL and many packages (MK:DEFSYSTEM
included) provide a special version of it, hence reinventing the
wheel.

Now the question is: how should CLOCC address this problem?  What
about MK:DEFSYSTEM.

I'll tell you how I'd proceeed.  I'd set up a package CL.EXT.STRINGS
which contained extensions to the string handling stuff of CL, and put
SPLIT-STRING in it.  Then and only then, I would and could give up the
special SPILT-STRING in MK:DEFSYSTEM.

I.e. I am for Software Buddhism. The middle way (between Cathedral and
Bazaar :) ) is best.  The Romans also had the say "in medium stat
virtus" (if my Latin is not failing me). :)

Apart from that there is another thing I would like to stress out.
Whatever ends up in CLOCC should strive to run in as many CL
implementations as possible.

Cheers


-- 
Marco Antoniotti ===========================================
PARADES, Via San Pantaleo 66, I-00186 Rome, ITALY
tel. +39 - 06 68 10 03 17, fax. +39 - 06 68 80 79 26
http://www.parades.rm.cnr.it/~marcoxa

what sort of modules should be in clocc

[<do...@co...>]

   >> I suggest maintaining one readme file in which the entire contents
   >> of the archive are listed.  You should be able to download that and
   >> then search for what you want, rather than having to download many
   >> different subdirectories.

  the archiving provided by sourceforge is quite adequate, so I suggest we
  use it.

I'm not actually talking about archiving but about how you search
the archive.  Is there some way of getting a table of contents?
Is there a brief description of each module?  I think there should be.

   >> A comment on one of the requirements for code in this repository:
   >>     Self-contained, i.e. does not require packages not in this
   >>     repository,
   >> I suggest that even reliance on this repository be minimized.

  NO WAY!!!!
  then each and every package in CLOCC will reimplement the same stuff
  over and over again.
  I suggest mandatory support of defsystem by each package - this way we
  can more or less be sure that people can get what they want to compile. 

I'm not arguing that there should be NO dependency.  For instance,
anything that uses resources outside of lisp will probably use the
cross-implementation stuff.  Also, cases where one facility clearly
depends on another are fine.  I just don't want a tangle of packages
where in order to get one simple thing you end up getting 10 more that
seem to be unrelated.  For instance, the record-calls package that I
just looked at contained a call to my pretty printer.  I've removed
that and replaced it with a print-function parameter.  I'd prefer all
your packages not to use things like your defcustom since it just
makes the reader go back and read a bunch of additional definitions.
If there are just a few small things then I'd rather have them
repeated than have to get an additional file.

What does mandatory support of defsystem mean?  You don't want any
file that can simply be loaded?

   >> I like the others, but is there such a thing as totally unlicensed
   >> code?  Do you have to put something on the top in order to make it
   >> qualify as that?  What's "public domain"?  I'm open to suggestions.

  *IANAL*
(Sorry, what's that mean?)

  3. PD means that you explicitly declared that anyone can take it.  e.g.,
     if you state in your header "this code is placed in the public domain
     by the author", I can take the code and do whatever I want with it,
     and you will never be able to prevent me from doing it (I am not sure
     I can (c) what you wrote, but I can certainly base a proprietary
     system on it).

What's wrong with that?

  4. I suggest GPL for "end user code" and LGP for what other programmers
     will want to use.
will look ...

   >> A few other suggestions:
   >> - Each piece of code, as much as possible should do one thing, so you
   >> can take what you need without having to get a lot of extra stuff.

  do you want 100000 different files? :-)
  memory is cheap.

This is not a matter of memory.  It's a matter of what the user of
this archive has to do.  I want him to be able to read and understand 
the code.  If there's extra code that he doesn't need, well, ok, maybe
that doesn't do much harm, but if that code requires other packages he
has to either realize that he doesn't need those other packages and
modify the source to get rid of them, or he has to get these other
packages.  That's what I really want to avoid.

Anyone out there who wants to look at the next piece of code let me
know when you're ready.