CEDET

Hi,

  I found one of the symbols (^=) broke all matches after it in the
list of operator syms in the file bovine/c.by.  I've checked in a new
version that fixes the problem for some symbols by moving the ^= match
to the end.  I hadn't debugged why ^= is a problem though.

Eric

>>> Arne Schmitz <arne.schmitz@...> seems to think that:
  [ ... ]
>I noticed that Semantic has a problem parsing something like:
>
>	T& operator+=3D(const &T) ...
>
>It seems that two-character operators are not supported.
  [ ... ]

-- 
          Eric Ludlam:                 zappo@..., eric@...
   Home: http://www.ludlam.net            Siege: http://www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: http://www.gnu.org

Am Freitag, 16. September 2005 20:21 schrieb Eric M. Ludlam:
> =C2=A0I've checked in a new
> version that fixes the problem for some symbols by moving the ^=3D match
> to the end. =C2=A0I hadn't debugged why ^=3D is a problem though.

Maybe because ^ has a special meaning for regexps? Just a guess...

Arne

=2D-=20
[--- PGP key FD05BED7 --- http://www.root42.de/ ---]

Hmmm, sounds possible.  The match works out like this:

%token <punctuation>   HAT        "\\`[^]\\'"

I thought brackets [ ] disabled hatness, but C-M-s indicates
otherwise.  I'll change it to \^ instead and test it out.

Eric

>>> Arne Schmitz <arne.schmitz@...> seems to think that:
>--nextPart1802665.ai2d4qH0gn
>Content-Type: text/plain;
>  charset="utf-8"
>Content-Transfer-Encoding: quoted-printable
>Content-Disposition: inline
>
>Am Freitag, 16. September 2005 20:21 schrieb Eric M. Ludlam:
>> =C2=A0I've checked in a new
>> version that fixes the problem for some symbols by moving the ^=3D match
>> to the end. =C2=A0I hadn't debugged why ^=3D is a problem though.
>
>Maybe because ^ has a special meaning for regexps? Just a guess...
>
>Arne
>
>=2D-=20
>[--- PGP key FD05BED7 --- http://www.root42.de/ ---]
>
>--nextPart1802665.ai2d4qH0gn
>Content-Type: application/pgp-signature
>
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1.4.0 (GNU/Linux)
>
>iD8DBQBDOlSIzpP3gv0FvtcRAmqWAKDrxZCMmOzJkiRR4okE6Wo4/WLj6gCggItf
>IHIRRKS6TmABf+0blA8PPKM=
>=ts/m
>-----END PGP SIGNATURE-----
>
>--nextPart1802665.ai2d4qH0gn--
>
>
>
>-------------------------------------------------------
>This SF.Net email is sponsored by:
>Power Architecture Resource Center: Free content, downloads, discussions,
>and more. http://solutions.newsforge.com/ibmarch.tmpl
>_______________________________________________
>cedet-semantic mailing list
>cedet-semantic@...
>https://lists.sourceforge.net/lists/listinfo/cedet-semantic
>

>>> Kevin Dwyer <kevin@...> seems to think that:
>Hello list,
>
>I'm currently running cedet-1.0pre3 coupled with ecb-2.32 and I'm
>experiencing some really slow behavior whenever I load or save a decent
>sized source file (1500 line python file).  What occurs is the
>minibuffer shows the LALR progress bar and I have to wait a good 15-20
>seconds (which for a 1 line change is equal to about eternity - 1
>seconds) before it will finish and relinquish control to emacs.  Emacs
>is frozen during this period (except for the progress bar) so I'm unable
>to do anything else.
>
>I've tried scouring the FAQ and other documentation looking for help in
>speeding this up but there is no mention of speed issues.  Can anyone
>offer a clue?
  [ ... ]

Hi,

  As I'm sure you can imagine, the larger a file, the longer it takes
to parse.  Python is interesting because it doesn't have convenient
scoping characters that Emacs likes as C does, such as { and }.  That
makes it take a little longer to go through the lexical step.

  Edits should be fast though, as only a small section of the file
would be parsed.  If you have over 1000 lines of one thing, that could
explain a bit.

  The interrupt time is taken up by the idle timer, Idle parsing
should be an interruptible process, such that when you press a key,
parsing stops on the incremental parser.  I don't think a full parse
doesn't have check points in it.  (I don't remember.)

  You can change `semantic-idle-scheduler-idle-time' to something
larger, which will remove the annoying typing interrupts.  If you
don't use or like the summary or completions mode, you could turn the
idle scheduler off completely via the startup configuration.  The idle
scheduler is considered part of the 'minimum' feature set for smooth
operations, but could be turned off if needed.  That time would then
be spent when using senator, ECB, or other functions.

Eric

-- 
          Eric Ludlam:                 zappo@..., eric@...
   Home: http://www.ludlam.net            Siege: http://www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: http://www.gnu.org

On Thu, Sep 22, 2005 at 05:59:17PM -0400, Eric M. Ludlam wrote:
> 
>   As I'm sure you can imagine, the larger a file, the longer it takes
> to parse.  Python is interesting because it doesn't have convenient
> scoping characters that Emacs likes as C does, such as { and }.  That
> makes it take a little longer to go through the lexical step.

Right, I understand it's a different kind of parsing.

>   Edits should be fast though, as only a small section of the file
> would be parsed.  If you have over 1000 lines of one thing, that could
> explain a bit.

Edits are fine on these large files.

>   The interrupt time is taken up by the idle timer, Idle parsing
> should be an interruptible process, such that when you press a key,
> parsing stops on the incremental parser.  I don't think a full parse
> doesn't have check points in it.  (I don't remember.)
> 
>   You can change `semantic-idle-scheduler-idle-time' to something
> larger, which will remove the annoying typing interrupts.  If you
> don't use or like the summary or completions mode, you could turn the
> idle scheduler off completely via the startup configuration.  The idle
> scheduler is considered part of the 'minimum' feature set for smooth
> operations, but could be turned off if needed.  That time would then
> be spent when using senator, ECB, or other functions.

I don't think I'm running into the idle scheduler, unless that's the one
that happens whenever you save a file.  My issue is that when you save a
large file, you're locked out of using emacs entirely until the parsing
is finished, which takes a while.  Frequently I will save a file and
switch to another buffer to make corresponding edits in a different
file, but this disrupts my work.

Thanks,
-kpd

>>> Kevin Dwyer <kevin@...> seems to think that:
  [ ... ]
>>   You can change `semantic-idle-scheduler-idle-time' to something
>> larger, which will remove the annoying typing interrupts.  If you
>> don't use or like the summary or completions mode, you could turn the
>> idle scheduler off completely via the startup configuration.  The idle
>> scheduler is considered part of the 'minimum' feature set for smooth
>> operations, but could be turned off if needed.  That time would then
>> be spent when using senator, ECB, or other functions.
>
>I don't think I'm running into the idle scheduler, unless that's the one
>that happens whenever you save a file.  My issue is that when you save a
>large file, you're locked out of using emacs entirely until the parsing
>is finished, which takes a while.  Frequently I will save a file and
>switch to another buffer to make corresponding edits in a different
>file, but this disrupts my work.
  [ ... ]

I do not know what it would be doing during a save file.  Semantic has
no save or write file hooks.  You would need to get a stack trace when
it is being slow with "M-x toggle-debug-on-quit RET".  If C-g is
disabled, it is likely in one of the special hooks, like a timer.

A likely scenario is that some other save hook value is calling into
some semantic function which, in turn, needs to get a fresh set of
tags.

Eric

-- 
          Eric Ludlam:                 zappo@..., eric@...
   Home: http://www.ludlam.net            Siege: http://www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: http://www.gnu.org

On Tue, Sep 27, 2005 at 07:24:48AM -0400, Eric M. Ludlam wrote:
> 
> I do not know what it would be doing during a save file.  Semantic has
> no save or write file hooks.  You would need to get a stack trace when
> it is being slow with "M-x toggle-debug-on-quit RET".  If C-g is
> disabled, it is likely in one of the special hooks, like a timer.
> 
> A likely scenario is that some other save hook value is calling into
> some semantic function which, in turn, needs to get a fresh set of
> tags.


I did a little more testing, and this LALR parsing that takes forever
seems to only happen after I do ecb-activate.  Just loading cedet
doesn't cause it to happen.

The code that does the slow parsing is within semantic though.  I guess
ECB calls out to it on save (and sometimes on load) of a file.

-kpd

Aha, thanks David.

  I had totally mis-read the message.  I typed in 

"M-: (eval-and-compile 'cl-extra)"

 and thought the loading there-of caused the problem.

 Your patch makes good sense.  Please check it in.

Eric

>>> David PONCE <david.ponce@...> seems to think that:
>Hi Eric,
>
>> That seems a bit mysterious.  I was not able to replicate the problem
>> either in Emacs 22.0.
>
>I was able to replicate the problem in Emacs 22.  It happens because
>the `eval-and-compile' and `eval-when-compile' nested form parser
>assumes that the body of those forms only contains parenthesized
>expressions that can be parsed to produce tags.  Here is a patch
>to semantic-el.el that fixes the bug for me. WDYT?
>
>Regards.
>David
>
>P.S.: IMO (eval-and-compile 'cl-extra) does nothing useful, it
>just eval the symbol `cl-extra'! I suppose it should be
>(eval-and-compile (require 'cl-extra)) instead.
>
>
>2005-09-07  David Ponce  <david@...>
>
>	* cedet/semantic/bovine/semantic-el.el
>
>	(eval-and-compile, eval-when-compile): Make the nested form parser
>	more robust.
>
>Index: semantic-el.el
>===================================================================
>RCS file: /cvsroot/cedet/cedet/semantic/bovine/semantic-el.el,v
>retrieving revision 1.35
>diff -c -r1.35 semantic-el.el
>*** semantic-el.el	19 Apr 2005 02:59:03 -0000	1.35
>--- semantic-el.el	7 Sep 2005 15:22:46 -0000
>***************
>*** 210,219 ****
>  
>  (semantic-elisp-setup-form-parser
>      (lambda (form start end)
>!       (condition-case foo
>!           (semantic-parse-region start end nil 1)
>!         (error (message "MUNGE: %S" foo)
>!                nil)))
>    eval-and-compile
>    eval-when-compile
>    )
>--- 210,223 ----
>  
>  (semantic-elisp-setup-form-parser
>      (lambda (form start end)
>!       (let ((tags
>!              (condition-case foo
>!                  (semantic-parse-region start end nil 1)
>!                (error (message "MUNGE: %S" foo)
>!                       nil))))
>!         (if (semantic-tag-p (car-safe tags))
>!             tags
>!           (semantic-tag-new-code (format "%S" (car form)) nil))))
>    eval-and-compile
>    eval-when-compile
>    )
>
>
>

-- 
          Eric Ludlam:                 zappo@..., eric@...
   Home: http://www.ludlam.net            Siege: http://www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: http://www.gnu.org

>  Your patch makes good sense.  Please check it in.

Done. Thanks.

David

That seems a bit mysterious.  I was not able to replicate the problem
either in Emacs 22.0.

If you were editing a lisp file, you may need to load the
semantic-el.el library (not the .elc) and try again to get a better
stack on the problem.

Eric

>>> Stefan =3D?iso-8859-1?Q?Reich=3DF6r?=3D <stefan@...> seems to t=
hink that:
>Hi!
>
>I use the following statement in a lisp file:
>(eval-and-compile 'cl-extra)
>
>
>When I have open that file, I get always annoying error message popups
>from the idle scheduler:
>
>
>Debugger entered: ((1 29))
>  byte-code("=C3=81=C3=82!.=C3=83=08!." [tag message "A Rule must return=
 a single tag-line list!" debug] 2)
>  semantic--tag-expand((1 29))
>  semantic-repeat-parse-whole-stream(((semantic-list 1 . 29)) nil nil)
>  semantic-parse-region-default(1 30 nil nil nil)
>  semantic-parse-region(1 30)
>  semantic-fetch-tags()
>  byte-code("..=C3=80 .*=C3=81." [semantic-fetch-tags nil] 1)
>  semantic-idle-scheduler-refresh-tags()
>  byte-code("=C3=86.=C3=87p=C3=87=C3=86=C3=88=C3=89=C3=8A \"\"\".=C3=86.=
..=C3=86...=C3=9D....=C3=9D....=C3=9D...=C3=8BX.1.=C3=8C ..W.=C3=9D.....=C3=
=8D .	=C3=86..........@... .W.=C3=8F..=C3=90\".r..q...\f=3D....=
...............=C3=8BX...=C3=8C ..W...=C3=8D ....p\nB.)..A....B.*\n.)...=C3=
=86....=1F.=C3=9C..=1F@... .=C2=B9.=C3=8F..=C3=91\".. .=C3=
=84.=C3=92=C3=93..\"... .. .=C3=93.=C3=92=C3=94..\"..=1FA..=1F.=C2=A4.+.	=
=C3=86....!.,..!@.....=C3=BD.=C3=8E .=C3=BD.=C3=8F..=C3=95\".r..q...\"...=
.\"....\"...=C3=8BX.=1F.=C3=8C ..W.\".=C3=8D .).!A..!.=C3=A8..=08=C3=86."=
 [inhibit-quit buffers others safe mode semantic-idle-scheduler-mode nil =
delq mapcar #[(b) "=C3=81=08!.=07.=08." [b buffer-file-name] 2] buffer-li=
st 0 buffer-size semantic-idle-scheduler-refresh-tags input-pending-p thr=
ow parsing-mode-buffers idle-queue working-temp-message "IDLE: execture s=
ervice %s..." "IDLE: execture service %s...done" parsing-other-buffers se=
mantic-debug-enabled semantic-lex-debug semantic-idle-scheduler-max-buffe=
r-size major-mode b --cl-dolist-temp-- semantic-current-input-throw-symbo=
l semantic-idle-scheduler-queue service --cl-dolist-temp-- semantic-idle-=
-scheduler-verbose-flag --cl-dolist-temp--] 8)
>  semantic-idle-core-handler()
>  byte-code("=C3=81 .=C3=82 .=C3=83.=C3=84 *." [save-match-data-internal=
 semantic-idle-scheduler-kill-timer match-data ((byte-code "=C3=81=08=C3=82=
\"." [save-match-data-internal set-match-data evaporate] 3)) semantic-idl=
e-core-handler] 1)
>  semantic-idle-scheduler-function()
>  apply(semantic-idle-scheduler-function nil)
>  byte-code("=C3=81=08=C3=82H=08=C3=83H\"." [timer apply 5 6] 4)
>  timer-event-handler([t 0 2 0 t semantic-idle-scheduler-function nil t]=
)
>
>
>What is wrong here?
>
>
>Stefan.
>
>
>
>-------------------------------------------------------
>SF.Net email is Sponsored by the Better Software Conference & EXPO
>September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practi=
ces
>Agile & Plan-Driven Development * Managing Projects & Teams * Testing & =
QA
>Security * Process Improvement & Measurement * http://www.sqe.com/bsce5s=
f
>_______________________________________________
>cedet-semantic mailing list
>cedet-semantic@...
>https://lists.sourceforge.net/lists/listinfo/cedet-semantic
>

--=20
          Eric Ludlam:                 zappo@..., eric@...=
om
   Home: http://www.ludlam.net            Siege: http://www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: http://www.gnu.org

Hi Eric!

> That seems a bit mysterious.  I was not able to replicate the problem
> either in Emacs 22.0.
>
> If you were editing a lisp file, you may need to load the
> semantic-el.el library (not the .elc) and try again to get a better
> stack on the problem.

Here is a better trace from M-x bovinate:
But I don't know, how I can get some more information instead of ...


Debugger entered: ((1 29))
  (condition-case nil (let (...) (semantic--tag-set-overlay tag ...) (setcdr ocdr nil)) (error (message "A Rule must return a single tag-line list!") (debug tag) nil))
  (if (semantic--tag-expanded-p tag) tag (condition-case nil (let ... ... ...) (error ... ... nil)) (unless semantic-tag-expand-function (setq semantic-tag-expand-function semantic-expand-nonterminal)) (if semantic-tag-expand-function (or ... ...) (list tag)))
  semantic--tag-expand((1 29))
  (mapcar (function (lambda ... ... tag)) (semantic--tag-expand tag))
  (setq tag (mapcar (function ...) (semantic--tag-expand tag)) result (append tag result))
  (if (car tag) (setq tag (mapcar ... ...) result (append tag result)))
  (if tag (if (car tag) (setq tag ... result ...)) (if returnonerror (setq stream nil) (setq semantic-unmatched-syntax-cache ...)))
  (while stream (setq nontermsym (semantic-parse-stream stream nonterm) tag (car ...)) (if (not nontermsym) (error "Parse error @ %d" ...)) (if (eq ... stream) (error "Parser error: Infinite loop?")) (if tag (if ... ...) (if returnonerror ... ...)) (setq stream (car nontermsym)) (if stream (if ... ... ...)))
  (let ((result nil) (case-fold-search semantic-case-fold) nontermsym tag) (while stream (setq nontermsym ... tag ...) (if ... ...) (if ... ...) (if tag ... ...) (setq stream ...) (if stream ...)) result)
  semantic-repeat-parse-whole-stream(((semantic-list 1 . 29)) nil nil)
  (nreverse (semantic-repeat-parse-whole-stream (or ... ...) nonterminal returnonerror))
  semantic-parse-region-default(1 30 nil nil nil)
  (if override (funcall override start end nonterminal depth returnonerror) (semantic-parse-region-default start end nonterminal depth returnonerror))
  (let ((override ...)) (if override (funcall override start end nonterminal depth returnonerror) (semantic-parse-region-default start end nonterminal depth returnonerror)))
  semantic-parse-region(1 30)
  (setq res (semantic-parse-region (point-min) (point-max)))
  (progn (setq res (semantic-parse-region ... ...)) (working-status t))
  (unwind-protect (progn (setq res ...) (working-status t)) (setq working-mode-line-message nil) (if working-use-echo-area-p (message working-current-message) (working-mode-line-update) (sit-for 0)))
  (let ((working-current-message ...) (working-message ...) (working-donestring "done") (working-ref1 0) (working-last-percent 0)) (unwind-protect (progn ... ...) (setq working-mode-line-message nil) (if working-use-echo-area-p ... ... ...)))
  (working-status-forms (semantic-parser-working-message (buffer-name)) "done" (setq res (semantic-parse-region ... ...)) (working-status t))
  (cond ((semantic-parse-tree-needs-update-p) (setq res ...) (if ... ... ... ...)) ((semantic-parse-tree-needs-rebuild-p) (working-status-forms ... "done" ... ...) (let ... ...) (semantic--tag-link-list-to-buffer res) (semantic--set-buffer-cache res)))
  (let* ((gc-cons-threshold ...) (semantic-lex-block-streams nil) (res nil)) (garbage-collect) (cond (... ... ...) (... ... ... ... ...)))
  (and (semantic-active-p) (run-hook-with-args-until-failure (quote semantic-before-toplevel-bovination-hook)) (run-hook-with-args-until-failure (quote semantic--before-fetch-tags-hook)) (not (semantic-parse-tree-unparseable-p)) (not (semantic-parse-tree-up-to-date-p)) (let* (... ... ...) (garbage-collect) (cond ... ...)))
  semantic-fetch-tags()
  (let* ((start ...) (out ...) (end ...)) (message "Retrieving tags took %.2f seconds." (semantic-elapsed-time start end)) (when (or ... ...) (pop-to-buffer "*Parser Output*") (require ...) (erase-buffer) (insert ...) (goto-char ...)))
  bovinate(nil)
  call-interactively(bovinate)
  execute-extended-command(nil)
  call-interactively(execute-extended-command)