CC Mode

> After writing cc-learn.el, http://www.gyve.org/~jet/ChangeLog,
> I've found cc-guess.el in cc-mode CVS repository.
> I am surprised that none has worked on such useful feature
> after Barry A. Warsaw's work.
> 
> I have merged my effort on cc-learn.el into cc-guess.el.
> Comments and docstring are still poor. But I'd like to hear
> your comments.
> 
> The weakness of original cc-guess.el is that in some case the original
> cc-guess.el guesses clearly wrong indentation if a target c file's not
> consistent.
> 
> With my patch, cc-guess.el guesses with taking frequency of 
> indentation into account. Therefore if the target c file's not 
> consistent enough, cc-guess.el guesses better indentation than
> the original.
> 
> I think it is worth to define a key bind and a menu item for cc-guess.

I've updated my patch for cc-guess.el.
It seems that my last post was ignored. If you like this patch, give a 
feedback here. 

Masatake YAMATO

Index: cc-guess.el
===================================================================
RCS file: /cvsroot/cc-mode/cc-mode/cc-guess.el,v
retrieving revision 5.34
diff -u -r5.34 cc-guess.el
--- cc-guess.el	17 Jun 2003 14:01:43 -0000	5.34
+++ cc-guess.el	9 Mar 2005 07:01:47 -0000
@@ -28,7 +28,7 @@
 ;;; Commentary:
 ;;
 ;; This file contains routines that help guess the cc-mode style in a
-;; particular region of C, C++, or Objective-C code.  It is provided
+;; particular region/buffer of C, C++, or Objective-C code.  It is provided
 ;; for example and experimentation only.  It is not supported in
 ;; anyway.  Some folks have asked for a style guesser and the best way
 ;; to show my thoughts on the subject is with this sample code.  Feel
@@ -36,7 +36,9 @@
 ;; results.  Note that style guessing is lossy!
 ;;
 ;; The way this is intended to be run is for you to mark a region of
-;; code to guess the style of, then run the command, cc-guess-region.
+;; code to guess the style of, then run the command, `cc-guess-region'.
+;;
+;; In addition...
 
 ;;; Code:
 
@@ -54,6 +56,20 @@
 
 (defvar cc-guessed-style nil
   "Currently guessed style.")
+(defvar cc-guess-delta-accumulator nil
+  "A variable accumulated sampled indent information.
+Information is represented in a list. Each element of
+list has following strcture:
+
+ (syntactic-symbol ((indentation-delta1 . number-of-times1)
+                    (indentation-delta2 . number-of-times2)
+                    ...))
+
+This structure is built by `cc-guess-accumulate-delta'.
+
+Here we call the pair (indentation-delta1 . number-of-times1),
+a counter.`cc-guess-sort-delta-accumulator' sorts the order of
+counters by number-of-times.")
 
 (defvar cc-guess-conversions
   '((c . c-lineup-C-comments)
@@ -64,51 +80,138 @@
     (cpp-macro . -1000)))
   
 
+(defun cc-guess (&optional accumulate)
+  "Apply `cc-guess-region' on the whole current buffer; 
+and set the guessed results to `cc-offsets-alist'.
+If optional argument ACCUMULATE is non-nil, use the value
+of `cc-guess-delta-accumulator' as the base syntactic information
+for guessing."
+  (interactive "P")
+  (cc-guess-region (point-min) (point-max) (not accumulate))
+  (cc-guess-install))
+
+(defun cc-guess-install ()
+  "Set guessed style(`cc-guessed-style') to `c-offsets-alist'"
+  (interactive)
+  (message "Set: %S" (pp cc-guessed-style))
+  (setq c-offsets-alist cc-guessed-style))
+
 (defun cc-guess-region (start end &optional reset)
   "Sets `c-offset-alist' indentation values based on region of code.
 Every line of code in the region is examined and the indentation
 values of the various syntactic symbols in `c-offset-alist' is
-guessed.  The first such positively identified indentation is used, so
+guessed. Frequencies in use is taken into account. Therefore,
 if an inconsistent style exists in the C code, the guessed indentation
-may be incorrect.
+may be correct in some case.
 
 Note that the larger the region to guess in, the slower the
-guessing. Previous guesses can be concatenated together, unless the
+guessing. Previous guesses can be take into account, unless the
 optional RESET is provided.
 
 See `cc-guess-write-style' to find out how to save the guessed style,
 and `cc-guess-view-style' for viewing the guessed style."
   (interactive "r\nP")
-  (if (consp reset)
-      (setq cc-guessed-style nil))
-  (save-excursion
-    (goto-char start)
-    (while (< (point) end)
-      (c-save-buffer-state
+  (let ((delta-accumulator (unless reset cc-guess-delta-accumulator))
+	(reporter (when (fboundp 'make-progress-reporter)
+		    (make-progress-reporter "Sampling Indentation " start end))))
+    ;;
+    ;; Sampling stage
+    ;;
+    (save-excursion
+      (goto-char start)
+      (while (< (point) end)
+	(c-save-buffer-state
 	    ((syntax (c-guess-basic-syntax))
-	     (relpos (cdr (car syntax)))
-	     (symbol (car (car syntax)))
-	     point-indent relpos-indent)
-	;; TBD: for now I can't guess indentation when more than 1
-	;; symbol is on the list, nor for symbols without relpos's
-	(if (or (/= 1 (length syntax))
-		(not (numberp relpos))
-		;; also, don't try to reguess an already guessed
-		;; symbol
-		(assq symbol cc-guessed-style))
-	    nil
-	  (back-to-indentation)
-	  (setq point-indent (current-column)
-		relpos-indent (save-excursion
-				(goto-char relpos)
-				(current-column)))
-	  ;; guessed indentation is the difference between point's and
-	  ;; relpos's current-column indentation
-	  (setq cc-guessed-style
-		(cons (cons symbol (- point-indent relpos-indent))
-		      cc-guessed-style))
-	  ))
-      (forward-line 1))))
+	     (relpos (car (cdr (car syntax))))
+	     (symbol (car (car syntax))))
+	  ;; TBD: for now I can't guess indentation when more than 1
+	  ;; symbol is on the list, nor for symbols without relpos's
+	  ;;
+	  ;; I think it is too stricted for ((topmost-intro) (comment-intro)).
+	  ;; -- Masatake
+	  (unless (or ; (/= 1 (length syntax))
+		   (not (numberp relpos))
+		   (eq (line-beginning-position)
+		       (line-end-position)))
+	    (setq delta-accumulator (cc-guess-accumulate-delta
+				     delta-accumulator
+				     symbol
+				     (- (progn (back-to-indentation)
+					       (current-column) )
+					(save-excursion
+					  (goto-char relpos)
+					  (current-column)))))))
+	(when reporter (progress-reporter-update reporter (point)))
+	(forward-line 1)))
+    (when reporter (progress-reporter-done reporter))
+    ;;
+    ;; Guessing stage
+    ;;
+    (setq delta-accumulator (cc-guess-sort-delta-accumulator  
+			     delta-accumulator)
+	  cc-guess-delta-accumulator delta-accumulator)
+    (let* ((typical-style (cc-guess-make-style delta-accumulator))
+	   (merged-style (cc-guess-merge-styles 
+			  (copy-list cc-guess-conversions)
+			  typical-style)))
+      (setq cc-guessed-style merged-style))))
+
+(defun cc-guess-accumulate-delta (accumulator symbol delta)
+  "Added SYMBOL and DELTA to ACCUMULATOR.
+See `cc-guess-delta-accumulator' about the structure of ACCUMULATOR." 
+  (let* ((entry    (assoc symbol accumulator))
+	 (counters (cdr entry))
+	 counter)
+    (if entry
+	(progn
+	  (setq counter (assoc delta counters))
+	  (if counter
+	      (setcdr counter (1+ (cdr counter)))
+	    (setq counters (cons (cons delta 1) counters))
+	    (setcdr entry counters))
+	  accumulator)
+      (cons (cons symbol (cons (cons delta 1) nil)) accumulator))))
+
+(defun cc-guess-sort-delta-accumulator (accumulator)
+  "Sort the each element of ACCUMULATOR by the number-of-times.
+See `cc-guess-delta-accumulator' for more details."
+  (mapcar
+   (lambda (entry)
+     (let ((symbol (car entry))
+	   (counters (cdr entry)))
+       (cons symbol (sort counters 
+			  (lambda (a b)
+			    (if (> (cdr a) (cdr b))
+				t
+			      (and 
+			       (eq (cdr a) (cdr b))
+			       (< (car a) (car b)))))))))
+   accumulator))
+	
+(defun cc-guess-make-style (accumulator)
+  "Throw away the rare cases in accumulator and make a style structure."
+  (mapcar 
+   (lambda (entry)
+     (cons (car entry) 
+	   (car (car (cdr entry)))))
+   accumulator))
+
+(defun cc-guess-merge-styles (strong weak)
+  "Merge two styles into one.
+When two styles has the same symbol entry, give STRONG 
+priority over WEAK."
+  (mapc
+   (lambda (weak-elt)
+     (unless (assoc (car weak-elt) strong)
+       (setq strong (cons weak-elt strong))))
+   weak)
+  strong)
+
+(defun cc-guess-view-style ()
+  "Show `cc-guessed-style'."
+  (interactive)
+  (with-output-to-temp-buffer "*Indentation Guessing Result*"
+    (pp cc-guessed-style)))
 
 
 (cc-provide 'cc-guess)

Masatake YAMATO <jet@...> wrote:

> I've updated my patch for cc-guess.el.

Thank you. It looks interesting, I will take a close look at it.

> It seems that my last post was ignored. If you like this patch, give a 
> feedback here.

It wasn't ignored, but I have a bad habit of not responding until I've
investigated something thoroughly, and it can be a good while before I
get the time to do that. I'm sorry for the lack of response.

Masatake YAMATO <jet@...> wrote:

> I've updated my patch for cc-guess.el.
> It seems that my last post was ignored. If you like this patch, give a 
> feedback here. 

Now I've taken a look at this. It's clearly an improvement, so I've
installed it - thanks for the patch.

I've made some linguistic fixes and converted some docstrings to
comments to follow the convention in CC Mode that internal functions
and variables shouldn't have docstrings.

I've also changed cc-guess-region a bit to be more in line with
cc-guess:

o  Firstly, I inverted the reset flag since I think it's more sane to
   make resetting the default. I believe you do too since you inverted
   it in your new cc-guess wrapper. I guess you didn't want to change
   the calling convention for cc-guess-region, but since the whole
   package is labeled as experimental and since I doubt it's used very
   much, I don't think it'll be a problem.

o  Secondly, while I were messing up cc-guess-region anyway, I also
   made it install the guessed style on c-offsets-alist using a merge.
   I think that makes it more convenient.

I haven't tested it much, but I noted one case where it doesn't do a
good job: arglist-close is commonly lined up under the opening paren
using c-lineup-close-paren, and there's therefore no common
indentation at all for it. The guessing code doesn't account for cases
like that, so one gets a fixed but arbitrary indentation instead.

From the looks of it, this case is intended to be solved by adding an
entry for arglist-close (and other similar symbols) on
cc-guess-conversions. That isn't very good however since it
effectively disables guessing for those symbols.

An idea is to instead try the lineups on cc-guess-conversions during
guessing and collect the frequency where they agree. If the
cc-guess-conversions variant works more often than a plain offset then
it's used instead. This could also be extended so that several
different alternatives to try for each symbol can be listed on
cc-guess-conversions.

Another improvement that occurred to me is to add a threshold so that
a certain amount of occurrences are required to set a symbol at all.
If there's only one or two places in the guessed region where some
fairly uncommon symbol occurs, the guessing is likely to be wrong
(especially if it's indented using some lineup function). In that case
it's probably better not to set anything at all on c-offsets-alist so
that the defaults remain intact.

What do you say?

> > I've updated my patch for cc-guess.el.
> > It seems that my last post was ignored. If you like this patch, give a 
> > feedback here. 
> 
> Now I've taken a look at this. It's clearly an improvement, so I've
> installed it - thanks for the patch.
> 
> I've made some linguistic fixes and converted some docstrings to
> comments to follow the convention in CC Mode that internal functions
> and variables shouldn't have docstrings.
> 
> I've also changed cc-guess-region a bit to be more in line with
> cc-guess:
> 
> o  Firstly, I inverted the reset flag since I think it's more sane to
>    make resetting the default. I believe you do too since you inverted
>    it in your new cc-guess wrapper. I guess you didn't want to change
>    the calling convention for cc-guess-region, but since the whole
>    package is labeled as experimental and since I doubt it's used very
>    much, I don't think it'll be a problem.
> 
> o  Secondly, while I were messing up cc-guess-region anyway, I also
>    made it install the guessed style on c-offsets-alist using a merge.
>    I think that makes it more convenient.

I agree with you. However, cc-guess-install in my patch is still
useful when adding a new source file to the target project directory;
you may want to import the indentation to the new file from the other 
source files in the directory.

> I haven't tested it much, but I noted one case where it doesn't do a
> good job: arglist-close is commonly lined up under the opening paren
> using c-lineup-close-paren, and there's therefore no common
> indentation at all for it. The guessing code doesn't account for cases
> like that, so one gets a fixed but arbitrary indentation instead.
> 
> From the looks of it, this case is intended to be solved by adding an
> entry for arglist-close (and other similar symbols) on
> cc-guess-conversions. That isn't very good however since it
> effectively disables guessing for those symbols.

I think (arglist-close . c-lineup-close-paren) is good enough because
it is very simple. I

> Another improvement that occurred to me is to add a threshold so that
> a certain amount of occurrences are required to set a symbol at all.

In that case the user should run cc-guess with the accumulate arg.
Remeber `cc-guess-delta-accumulator' doesn't hold only indentations
but also sampling count:

    ;;  (syntactic-symbol ((indentation-delta1 . number-of-times1)
    ;; 		           (indentation-delta2 . number-of-times2)
    ;; 		       ...))

> If there's only one or two places in the guessed region where some
> fairly uncommon symbol occurs, the guessing is likely to be wrong
> (especially if it's indented using some lineup function). In that case
> it's probably better not to set anything at all on c-offsets-alist so
> that the defaults remain intact.

I don't have a strong idea now.  How do you think adding a function
`cc-guess-reset-guessing-item-about-point'? It is better to get feed backs
from users. Until then, I think it is better to make the guessing rule 
simpler.

In MANIFEST

    cc-guess.el: Experiments in style guessing.  This is provided for
    your convenience only, and is unguaranteed and unsupported.

This situation won't change?
I feel cc-guess is worth to be used widely.
I think key definitions for cc-guess and cc-guess-install are needed.
My idea is:

    C-c >  for cc-guess
    C-c <  for cc-guess-install.

Regards,
Masatake YAMATO

Masatake YAMATO <jet@...> wrote:

> /.../ However, cc-guess-install in my patch is still useful when
> adding a new source file to the target project directory; you may
> want to import the indentation to the new file from the other source
> files in the directory.

Very well. I've added it now, but with a merge instead of a plain
assignment.

>> Another improvement that occurred to me is to add a threshold so that
>> a certain amount of occurrences are required to set a symbol at all.
>
> In that case the user should run cc-guess with the accumulate arg.

That isn't the same thing. That would be a way to make settings for
rare symbols go over the threshold, but what I suggested is adding a
threshold in the first place.

>> If there's only one or two places in the guessed region where some
>> fairly uncommon symbol occurs, the guessing is likely to be wrong
>> (especially if it's indented using some lineup function). In that case
>> it's probably better not to set anything at all on c-offsets-alist so
>> that the defaults remain intact.
>
> I don't have a strong idea now.  How do you think adding a function
> `cc-guess-reset-guessing-item-about-point'? /.../

This would be invoked by the user when (s)he discovers that the
indentation of the current line is wrong due to a wrong guess, I
presume? Then I personally would prefer the threshold since it would
avoid the wrong guess from ever occurring. It would instead use my
preexisting style which at least would indent the line as I like.
Since the syntactic context in question hardly occurs in the sampled
region, my personal preference wouldn't deviate much from the existing
style anyway.

> In MANIFEST
>
>     cc-guess.el: Experiments in style guessing.  This is provided for
>     your convenience only, and is unguaranteed and unsupported.
>
> This situation won't change?

Maybe it can. It certainly would if you're willing to support it.

(Personally I would however like another lighter variant of guessing:
A function that completely automatically on file load analyzed a bit
of the buffer to establish a setting for c-basic-offset only. It would
otherwise leave my configured style intact. But this lighter guessing
function can very well coexist with the one we're discussing here.)

> I think key definitions for cc-guess and cc-guess-install are needed.
> My idea is:
>
>     C-c >  for cc-guess
>     C-c <  for cc-guess-install.

I don't think those two key sequences associate particularly well with
these functions. They seem more suited for some kind of forward and
backward movement (c.f. M-< and M->), so I'd prefer to keep them free
if something like that comes along.

I think C-c C-g would work nicely for cc-guess, or maybe
cc-guess-region - isn't cc-guess a bit too susceptible to getting
annoyingly slow if the buffer gets large?

I can't think of any obvious binding for cc-guess-install, though. C-c
C-i would perhaps be closest but that one will probably get bound to a
toggle for electric indentation.

> > /.../ However, cc-guess-install in my patch is still useful when
> > adding a new source file to the target project directory; you may
> > want to import the indentation to the new file from the other source
> > files in the directory.
> 
> Very well. I've added it now, but with a merge instead of a plain
> assignment.
> 
> >> Another improvement that occurred to me is to add a threshold so that
> >> a certain amount of occurrences are required to set a symbol at all.
> >
> > In that case the user should run cc-guess with the accumulate arg.
> 
> That isn't the same thing. That would be a way to make settings for
> rare symbols go over the threshold, but what I suggested is adding a
> threshold in the first place.
> 
> >> If there's only one or two places in the guessed region where some
> >> fairly uncommon symbol occurs, the guessing is likely to be wrong
> >> (especially if it's indented using some lineup function). In that case
> >> it's probably better not to set anything at all on c-offsets-alist so
> >> that the defaults remain intact.
> >
> > I don't have a strong idea now.  How do you think adding a function
> > `cc-guess-reset-guessing-item-about-point'? /.../
> 
> This would be invoked by the user when (s)he discovers that the
> indentation of the current line is wrong due to a wrong guess, I
> presume? Then I personally would prefer the threshold since it would
> avoid the wrong guess from ever occurring. It would instead use my
> preexisting style which at least would indent the line as I like.
> Since the syntactic context in question hardly occurs in the sampled
> region, my personal preference wouldn't deviate much from the existing
> style anyway.

I may understand. I'll introduce cc-guess-thresholds. 

> > In MANIFEST
> >
> >     cc-guess.el: Experiments in style guessing.  This is provided for
> >     your convenience only, and is unguaranteed and unsupported.
> >
> > This situation won't change?
> 
> Maybe it can. It certainly would if you're willing to support it.

I'll support it.

> (Personally I would however like another lighter variant of guessing:
> A function that completely automatically on file load analyzed a bit
> of the buffer to establish a setting for c-basic-offset only. It would
> otherwise leave my configured style intact. But this lighter guessing
> function can very well coexist with the one we're discussing here.)

Based on the cc-guess.el, it may not be difficult to implementat the lighter
one.

> > I think key definitions for cc-guess and cc-guess-install are needed.
> > My idea is:
> >
> >     C-c >  for cc-guess
> >     C-c <  for cc-guess-install.
> 
> I don't think those two key sequences associate particularly well with
> these functions. They seem more suited for some kind of forward and
> backward movement (c.f. M-< and M->), so I'd prefer to keep them free
> if something like that comes along.

I agree with you.

> I think C-c C-g would work nicely for cc-guess, or maybe
> cc-guess-region - isn't cc-guess a bit too susceptible to getting
> annoyingly slow if the buffer gets large?

About binding see the bottom of this mail.

I strongly believe that it is much more annoying than the slowness of
guessing to write a style and/or to set c-basic-offset by my hand for
strangely indented source code.

Furtherer more, once guessing on a large buffer, we can install the guessing
result to the other buffer with cc-guess-install.

> I can't think of any obvious binding for cc-guess-install, though. C-c
> C-i would perhaps be closest but that one will probably get bound to a
> toggle for electric indentation.

My renewal ideas are that 

    C-c C-g g cc-guess
    C-c C-g r cc-guess-region
    C-c C-g i cc-guess-install

Regards,
Masatake YAMATO