Thread: [CEDET-devel] CEDET and Matlab
Brought to you by:
zappo
From: David <de...@ar...> - 2008-05-14 20:15:08
|
After roughly 2 years I'm currently trying CEDET again (from CVS) and must say that I'm impressed how stable and fast it has become. Great job! Since I also do lots of Matlab coding I'm really interested in this little file semantic-matlab.el in the contrib directory. I tried to get it to work but with little success. It obviously tags function definitions but I couldn't really get it to work across several files with completion. Is it supposed to be usable and is somebody working with this? |
From: Eric M. L. <er...@si...> - 2008-05-14 23:32:32
|
>>> David <de...@ar...> seems to think that: >After roughly 2 years I'm currently trying CEDET again (from CVS) and >must say that I'm impressed how stable and fast it has become. Great >job! Since I also do lots of Matlab coding I'm really interested in this >little file semantic-matlab.el in the contrib directory. I tried to get >it to work but with little success. It obviously tags function >definitions but I couldn't really get it to work across several files >with completion. Is it supposed to be usable and is somebody working >with this? [ ... ] Hi, Thanks for reminding me that I had fixes for that. I checked them in. The semantic-matlab parsing stuff will likely move out of the CEDET distribution (it is not a part of the distribution) to the http://matlab-emacs.sf.net project, which is where I keep my Matlab specific stuff. I haven't worked on the CEDET/Semantic support for Matlab much. It needs some help as the file layout is a bit different from a lot of other languages. I have quite a bit I need to do for the Matlab support in Emacs in general, such as dealing with the new object system, but that is tied to my work, as opposed to my home project which is CEDET, and I'm pretty busy when I'm at work these days. (I work at The MathWorks in Natick.) Eric -- Eric Ludlam: er...@si... Siege: www.siege-engine.com Emacs: http://cedet.sourceforge.net |
From: David <de...@ar...> - 2008-05-15 09:55:06
|
"Eric M. Ludlam" <er...@si...> writes: > Thanks for reminding me that I had fixes for that. I checked them > in. Thanks. It still didn't really work for me since semantic-matlab-match-function-re assumes that return values are in square brackets, which I usually omit if there's only one return value. I changed it to "\\(^\\s-*function\\b[ \t\n.]*\\)\\(.*=\\|\\)\\s-*\\(\\sw+\\)\\>" which seems to cover at least the following function definitions: function [a,b] = test1(foo,bar) function [a] = test2(foo) function a = test3(foo) function a = test4 function test5(foo) function test6 I still cannot get completion to work across different files though, but I guess this is somehow a configuration error on my part? I don't use EDE for now but simply set semanticdb-project-roots to the directory containing my Matlab files. I can complete functions which are defined in the currently visited buffer, but not from functions in the other files. BTW, I also get the following error due to the default value of semanticdb-default-save-directory: (file-error "Opening output file" "no such file or directory" "/home/david/.semanticdb/!home!david!testcedet!semantic.cache"): /home/david/.semanticdb/!home!david!testcedet!semantic.cache Not sure if this is just to let me know that the tags are saved elsewhere, but I created ~/.semanticdb now, just to be sure. > The semantic-matlab parsing stuff will likely move out of the CEDET > distribution (it is not a part of the distribution) to the > http://matlab-emacs.sf.net project, which is where I keep my Matlab > specific stuff. Yes, I'm using matlab.el from CVS. > I have quite a bit I need to do for the Matlab support in Emacs in > general, such as dealing with the new object system, but that is tied > to my work, as opposed to my home project which is CEDET, and I'm > pretty busy when I'm at work these days. (I work at The MathWorks in > Natick.) I haven't looked at the new object system yet since I'm still using Matlab 7.4. I'm working a lot with the current object system with the awkward @-directories, which I guess isn't really parseable? However, I was working with etags so far for my Matlab need (see http://www.physik3.gwdg.de/~engster/matlab-tags.html) and it would be great if I could replace it with CEDET. -David |
From: Eric M. L. <er...@si...> - 2008-05-16 16:17:06
|
>>> David <de...@ar...> seems to think that: >"Eric M. Ludlam" <er...@si...> writes: >> Thanks for reminding me that I had fixes for that. I checked them >> in. > >Thanks. It still didn't really work for me since >semantic-matlab-match-function-re assumes that return values are in >square brackets, which I usually omit if there's only one return >value. I changed it to > > "\\(^\\s-*function\\b[ \t\n.]*\\)\\(.*=\\|\\)\\s-*\\(\\sw+\\)\\>" > >which seems to cover at least the following function definitions: > >function [a,b] = test1(foo,bar) >function [a] = test2(foo) >function a = test3(foo) >function a = test4 >function test5(foo) >function test6 Hi, Thats for the test. I updated the regexp to handle your listed cases, but to also disallow some items, where .* was too general. >I still cannot get completion to work across different files though, but >I guess this is somehow a configuration error on my part? I don't use >EDE for now but simply set semanticdb-project-roots to the directory >containing my Matlab files. I can complete functions which are defined >in the currently visited buffer, but not from functions in the other >files. The MATLAB support right now is only for basic parsing for tags. Setting the semanticdb-project-roots to point at your top-level function is the right thing to do. I tried jumping around (with C-c , J .. which is capital J) and it found tags across files for me. You won't be able to get any of the smart completion to work, because MATLAB is not a typed language, and the MATLAB language support excludes local context parsing. If you look in semantic/bovine/semantic-el.el, you will find it overrides a bunch of methods for local context parsing. It seems likely this will need to be done to start getting the smart completion to work for MATLAB. If you'd like to tackle this problem, I'd be happy to help you out. >BTW, I also get the following error due to the default value of >semanticdb-default-save-directory: > >(file-error "Opening output file" "no such file or directory" >"/home/david/.semanticdb/!home!david!testcedet!semantic.cache"): >/home/david/.semanticdb/!home!david!testcedet!semantic.cache > >Not sure if this is just to let me know that the tags are saved >elsewhere, but I created ~/.semanticdb now, just to be sure. Hmmm. I was under the impression this would be made for you, but I see now it is not. I'll put that on my todo list. [ ... ] >> I have quite a bit I need to do for the Matlab support in Emacs in >> general, such as dealing with the new object system, but that is tied >> to my work, as opposed to my home project which is CEDET, and I'm >> pretty busy when I'm at work these days. (I work at The MathWorks in >> Natick.) > >I haven't looked at the new object system yet since I'm still using >Matlab 7.4. I'm working a lot with the current object system with the >awkward @-directories, which I guess isn't really parseable? However, I The dir structure isn't parsable, but it is certainly navigable. There are routines that could be overloaded for MATLAB to do a directory search instead of parser output search for that sort of thing. I'd be happy to help you through that as needed. Eric -- Eric Ludlam: er...@si... Siege: www.siege-engine.com Emacs: http://cedet.sourceforge.net |
From: Eric M. L. <er...@si...> - 2008-05-16 22:06:43
|
>>> "Eric M. Ludlam" <er...@si...> seems to think that: >>>> David <de...@ar...> seems to think that: [ ... ] >>function [a,b] = test1(foo,bar) >>function [a] = test2(foo) >>function a = test3(foo) >>function a = test4 >>function test5(foo) >>function test6 > >Hi, > > Thats for the test. [ ... ] ^^^^^ That should read "Thanks" for the test. :) Eric -- Eric Ludlam: er...@si... Siege: www.siege-engine.com Emacs: http://cedet.sourceforge.net |
From: David <de...@ar...> - 2008-05-17 09:16:01
|
"Eric M. Ludlam" <er...@si...> writes: > That should read "Thanks" for the test. :) Well, I figured that one out. :-) > I updated the regexp to handle your listed cases, but to also disallow > some items, where .* was too general. Thanks. The new regexp works flawlessly for me. > I tried jumping around (with C-c , J .. which is capital J) > and it found tags across files for me. Does work for me, too. > You won't be able to get any of the smart completion to work, because > MATLAB is not a typed language, and the MATLAB language support > excludes local context parsing. OK, that means I cannot use anything that depends on semantic-analyze-possible-completions, right? But I'd already be happy if I had some kind of dumb completion, which simply uses the list of all known tags, just like what semantic-complete-jump does. Sorry if there exists something obvious, but I can't seem to find it in semantic-complete.el. > If you look in semantic/bovine/semantic-el.el, you will find it > overrides a bunch of methods for local context parsing. It seems > likely this will need to be done to start getting the smart completion > to work for MATLAB. > > If you'd like to tackle this problem, I'd be happy to help you out. Thanks. I'd love to dig into semantic, but I'm afraid I won't have the time to do it. I really need to get some more stuff done in Matlab first... :-) Thanks for your help, David |
From: Eric M. L. <er...@si...> - 2008-05-17 11:49:20
|
>>> David <de...@ar...> seems to think that: >"Eric M. Ludlam" <er...@si...> writes: [ ... ] >> You won't be able to get any of the smart completion to work, because >> MATLAB is not a typed language, and the MATLAB language support >> excludes local context parsing. > >OK, that means I cannot use anything that depends on >semantic-analyze-possible-completions, right? But I'd already be happy >if I had some kind of dumb completion, which simply uses the list of all >known tags, just like what semantic-complete-jump does. Sorry if there >exists something obvious, but I can't seem to find it in >semantic-complete.el. [ ... ] Hi, The senator complete C-c , TAB has a unsmart completion backup when the analyzer fails. If you use hippie expand (which you have to bind to your own key) you will get the best of many worlds, as that has semantic integration also, along with the other goodness that comes along with hippie-expand. Eric -- Eric Ludlam: er...@si... Siege: www.siege-engine.com Emacs: http://cedet.sourceforge.net |
From: David <de...@ar...> - 2008-05-23 13:25:32
|
"Eric M. Ludlam" <er...@si...> writes: > The senator complete C-c , TAB has a unsmart completion backup when > the analyzer fails. If you use hippie expand (which you have to bind > to your own key) you will get the best of many worlds, as that has > semantic integration also, along with the other goodness that comes > along with hippie-expand. 'senator-complete-symbol' is working nicely, thanks! Another question (for which I guess the answer is quite easy): How do I get CEDET to scan all files in my semantic-db-project-roots directory for tags/symbols? I was under the impression that this would be done through the idle-work-timer, but it somehow doesn't happen - I have to manually load every file once to have its tags scanned and saved in the database. This is also the case for my ELISP projects - while the ELISP files included through 'require' get scanned, semantic doesn't seem to scan the current directory (which is semantic-db-project-roots). Thanks for your help, David |
From: Eric M. L. <er...@si...> - 2008-05-23 18:00:03
|
>>> David <de...@ar...> seems to think that: >"Eric M. Ludlam" <er...@si...> writes: >> The senator complete C-c , TAB has a unsmart completion backup when >> the analyzer fails. If you use hippie expand (which you have to bind >> to your own key) you will get the best of many worlds, as that has >> semantic integration also, along with the other goodness that comes >> along with hippie-expand. > >'senator-complete-symbol' is working nicely, thanks! Another question >(for which I guess the answer is quite easy): How do I get CEDET to scan >all files in my semantic-db-project-roots directory for tags/symbols? I >was under the impression that this would be done through the >idle-work-timer, but it somehow doesn't happen - I have to manually load >every file once to have its tags scanned and saved in the database. This >is also the case for my ELISP projects - while the ELISP files included >through 'require' get scanned, semantic doesn't seem to scan the current >directory (which is semantic-db-project-roots). [ ... ] The work-timer thing only loads in the files referenced by the files currently loaded in. Since matlab doesn't have an "include" statement equivalent, it won't do anything. Adding a feature to do what you are requesting wouldn't be too hard. Someone just needs to add the feature. If you look at semanticdb-mk.el has a command line version (through semanticdb.sh) that could probably be adapted. Eric -- Eric Ludlam: er...@si... Siege: www.siege-engine.com Emacs: http://cedet.sourceforge.net |
From: David <de...@ar...> - 2008-05-27 13:24:44
|
"Eric M. Ludlam" <er...@si...> writes: >I updated the regexp to handle your listed cases, but to also disallow >some items, where .* was too general. I just noticed one little problem: the regexp doesn't properly handle function names with underscores in them. > The work-timer thing only loads in the files referenced by the files > currently loaded in. Since matlab doesn't have an "include" statement > equivalent, it won't do anything. [...] > If you look at semanticdb-mk.el has a command line version (through > semanticdb.sh) that could probably be adapted. OK, I managed to scan all the tags in my Matlab files with semanticdb-mk.el. This works nicely for completing/searching tags for files in the current directory, but how do I get CEDET/semantic to also use the tags in all the subdirectories? As you said, Matlab doesn't have an 'include' statement, so I guess I would somehow have to specify "manually" which subdirectories should be included? Regards, David PS: If you rather want to continue this thread on the matlab-emacs mailing list, just do so (I am subscribed to it). |
From: Eric M. L. <er...@si...> - 2008-05-29 01:36:20
|
>>> David <de...@ar...> seems to think that: >"Eric M. Ludlam" <er...@si...> writes: >>I updated the regexp to handle your listed cases, but to also disallow >>some items, where .* was too general. > >I just noticed one little problem: the regexp doesn't properly handle >function names with underscores in them. I'll get to this eventually. I usually fix this stuff up at work. >> The work-timer thing only loads in the files referenced by the files >> currently loaded in. Since matlab doesn't have an "include" statement >> equivalent, it won't do anything. >[...] >> If you look at semanticdb-mk.el has a command line version (through >> semanticdb.sh) that could probably be adapted. > >OK, I managed to scan all the tags in my Matlab files with >semanticdb-mk.el. This works nicely for completing/searching tags for >files in the current directory, but how do I get CEDET/semantic to also >use the tags in all the subdirectories? As you said, Matlab doesn't have >an 'include' statement, so I guess I would somehow have to specify >"manually" which subdirectories should be included? Since MATLAB doesn't conform to the C like mechanisms, getting this sort of thing to work requires overloading all the functions that are too C centric. For example, `semanticdb-find-tag-by-name' calls something that builds a `path'. That semanticdb-find-translate-path should be overridden to include the local file, anything in private with the specified name, and anything on the MATLAB path with the specified name. (Find-translate-path builds a map of DB tables to search, and is different from a what might be considered an include path.) A second thing is to write a simple EDE project for MATLAB that knows how to look at a MATLAB install, and extract the file path. EDE needs some sort of high speed `locate' function which would then be used for this case. Rooting out all these features wouldn't be too hard, but would be time-consuming. >Regards, >David > >PS: If you rather want to continue this thread on the matlab-emacs mailing >list, just do so (I am subscribed to it). [ ... ] Either is ok. Those on the other list might not be interested in detailed development discussion, but would be interested in intermediate announcements of enhancements. Eric -- Eric Ludlam: er...@si... Siege: www.siege-engine.com Emacs: http://cedet.sourceforge.net |
From: David <de...@ar...> - 2008-07-18 14:11:37
|
Following up on a thread from May '08... "Eric M. Ludlam" <er...@si...> writes: >>>> David <de...@ar...> seems to think that: >>OK, I managed to scan all the tags in my Matlab files with >>semanticdb-mk.el. This works nicely for completing/searching tags for >>files in the current directory, but how do I get CEDET/semantic to also >>use the tags in all the subdirectories? As you said, Matlab doesn't have >>an 'include' statement, so I guess I would somehow have to specify >>"manually" which subdirectories should be included? > > Since MATLAB doesn't conform to the C like mechanisms, getting this > sort of thing to work requires overloading all the functions that are > too C centric. For example, `semanticdb-find-tag-by-name' calls > something that builds a `path'. That semanticdb-find-translate-path > should be overridden to include the local file, anything in private > with the specified name, and anything on the MATLAB path with the > specified name. (Find-translate-path builds a map of DB tables to > search, and is different from a what might be considered an include > path.) OK, after some Emacs Lisp I now got back to Matlab. :-) Matlab doesn't have an include mechanism, instead you could say it simply includes every function it finds in its path. So I overloaded semantidb-find-translate-path for matlab-mode to make semantic believe a M-file just does that. The Matlab path is for now simply specified by a new variable (semantic-matlab-include-paths), but should be better replaced by what you already suggested: > A second thing is to write a simple EDE project for MATLAB that knows > how to look at a MATLAB install, and extract the file path. But since not everyone uses EDE, I guess it also makes sense to have a variable to manually specify the search path. I attached to this message how far I've come. This is all pretty crude yet and needs some serious cleaning up; I just copied most of the stuff from semantidb-find.el, but it already works reasonably well, with the notable exception of caching. First, caching seems to work: If I initiate a completion, the find-translate-path function gets called, the M-files are scanned, and in the subsequent calls to find-translate-path (still for the same completion) the cached values are used. However, every time I initiate a *new* completion, the files get re-scanned. Somehow the include-path in the tag object is not saved between completions. Maybe you immediately know what the problem is here, otherwise I'll further have to dig into it. I don't know if these is even a good way to do this. Maybe one should better use an omniscient databases for this, like in Emacs Lisp mode? Thanks for your help, David Here's the code I put into semantic-matlab.el: (defvar semantic-matlab-include-paths '("/usr/matlab/7.4/toolbox/matlab/general" "/home/david/someproject") "Directories which should be scanned for m-files.") (defun semantic-matlab-generate-include-tags (dirs) "Create include tags with m-files in DIRS. DIRS is a list of directories." (let (tags) (dolist (dir dirs) (dolist (cur (directory-files dir t "\\.m$" t)) (push (semantic-tag-new-include cur nil) tags))) tags)) (define-mode-local-override semanticdb-find-translate-path matlab-mode (path brutish) "Translate PATH into a list of semantic tables." (if (semanticdb-find-results-p path) ;; Perform the search over these results. nil (if brutish (semanticdb-find-translate-path-brutish-default path) (let ((table (cond ((null path) semanticdb-current-table) ((semanticdb-abstract-table-child-p path) path) (t nil)))) (if table ;; If we were passed in something related to a TABLE, ;; do a caching lookup. (let ((index (semanticdb-get-table-index table))) (if (semanticdb-find-need-cache-update-p table) ;; Lets go look up our indicies (let ((ans (semantic-matlab-translate-path-includes--internal path))) (oset index include-path ans) ;; Once we have our new indicies set up, notify those ;; who depend on us if we found something for them to ;; depend on. (when ans (semanticdb-refresh-references table)) ans) ;; ELSE ;; ;; Just return the cache. (oref index include-path))) ;; If we were passed in something like a tag list, or other boring ;; searchable item, then instead do the regular thing without caching. (semantic-matlab-translate-path-includes--internal path)))))) (defun semantic-matlab-translate-path-includes--internal (path) "Include all m-files found in `semantic-matlab-include-paths' in PATH." (let ((includetags nil) (curtable nil) (matchedtables (list semanticdb-current-table)) (matchedincludes nil) (lostincludes nil) (scannedincludes nil) (incfname nil) nexttable) (setq includetags (semantic-matlab-generate-include-tags semantic-matlab-include-paths)) (cond ((null path) (setq curtable semanticdb-current-table incfname (buffer-file-name)) ) ((semanticdb-table-p path) (setq curtable path incfname (semanticdb-full-filename path)) ) ((bufferp path) (setq curtable (save-excursion (set-buffer path) semanticdb-current-table) incfname (buffer-file-name path))) (t (when includetags ;; If we have some tags, derive a table from them. ;; else we will do nothing, so the table is useless. ;; @todo - derive some tables (message "Need to derive tables for %S in translate-path-includes--default." path) ))) ;; Make sure each found include tag has an originating file name associated ;; with it. (when incfname (dolist (it includetags) (semantic--tag-put-property it :filename incfname))) ;; Loop over all include tags adding to matchedtables (while includetags (semantic-throw-on-input 'semantic-find-translate-path-includes-default) ;; If we've seen this include string before, lets skip it. (if (member (semantic-tag-name (car includetags)) matchedincludes) (progn (setq nexttable nil) (push (cons 'duplicate (semantic-tag-clone (car includetags))) scannedincludes) ) (setq nexttable (semanticdb-find-table-for-include (car includetags) curtable)) (when (not nexttable) ;; Save the lost include. (push (car includetags) lostincludes) (push (cons 'lost (semantic-tag-clone (car includetags))) scannedincludes) ) ) ;; Push the include file, so if we can't find it, we only ;; can't find it once. (push (semantic-tag-name (car includetags)) matchedincludes) (message "Scanning %s" (semantic-tag-name (car includetags))) (when (and nexttable (not (memq nexttable matchedtables)) (semanticdb-equivalent-mode-for-search nexttable (current-buffer)) ) ;; Add to list of tables (push nexttable matchedtables) ) (setq includetags (cdr includetags))) (setq semanticdb-find-lost-includes lostincludes) (setq semanticdb-find-scanned-include-tags (reverse scannedincludes)) (nreverse matchedtables))) |
From: Eric M. L. <er...@si...> - 2008-07-19 17:16:32
|
Hi, Thanks for the patch. You're guess is correct that creating a omniscient database for MATLAB is the right way to go. That would give you the structured needed to prevent rescanning over and over. There is semanticdb-skel.el, which you can use to start a new database. Chances are you can just replace SKEL with matlab, copy your code into the right slots, and be done with it. I will likely convert your patch like that before I install it into the matlab-emacs project, though I know I don't have time for that this weekend. For the EDE part, I'm trying to build some infrastructure for simpler projects, where EDE will auto detect some kinds of basic projects w/out the need for full-up code generation. This is an important just as an abstraction layer between file management and semantic, so Semantic doesn't need to know about it. Eric >>> David <de...@ar...> seems to think that: >Following up on a thread from May '08... > >"Eric M. Ludlam" <er...@si...> writes: >>>>> David <de...@ar...> seems to think that: >>>OK, I managed to scan all the tags in my Matlab files with >>>semanticdb-mk.el. This works nicely for completing/searching tags for >>>files in the current directory, but how do I get CEDET/semantic to also >>>use the tags in all the subdirectories? As you said, Matlab doesn't have >>>an 'include' statement, so I guess I would somehow have to specify >>>"manually" which subdirectories should be included? >> >> Since MATLAB doesn't conform to the C like mechanisms, getting this >> sort of thing to work requires overloading all the functions that are >> too C centric. For example, `semanticdb-find-tag-by-name' calls >> something that builds a `path'. That semanticdb-find-translate-path >> should be overridden to include the local file, anything in private >> with the specified name, and anything on the MATLAB path with the >> specified name. (Find-translate-path builds a map of DB tables to >> search, and is different from a what might be considered an include >> path.) > >OK, after some Emacs Lisp I now got back to Matlab. :-) Matlab doesn't >have an include mechanism, instead you could say it simply includes >every function it finds in its path. So I overloaded >semantidb-find-translate-path for matlab-mode to make semantic believe a >M-file just does that. The Matlab path is for now simply specified by a >new variable (semantic-matlab-include-paths), but should be better >replaced by what you already suggested: > >> A second thing is to write a simple EDE project for MATLAB that knows >> how to look at a MATLAB install, and extract the file path. > >But since not everyone uses EDE, I guess it also makes sense to have a >variable to manually specify the search path. > >I attached to this message how far I've come. This is all pretty crude >yet and needs some serious cleaning up; I just copied most of the stuff >from semantidb-find.el, but it already works reasonably well, with the >notable exception of caching. First, caching seems to work: If I >initiate a completion, the find-translate-path function gets called, the >M-files are scanned, and in the subsequent calls to find-translate-path >(still for the same completion) the cached values are used. However, >every time I initiate a *new* completion, the files get >re-scanned. Somehow the include-path in the tag object is not saved >between completions. Maybe you immediately know what the problem is >here, otherwise I'll further have to dig into it. > >I don't know if these is even a good way to do this. Maybe one should >better use an omniscient databases for this, like in Emacs Lisp mode? > >Thanks for your help, >David > [ ... ] -- Eric Ludlam: er...@si... Siege: www.siege-engine.com Emacs: http://cedet.sourceforge.net |
From: David <de...@ar...> - 2008-07-20 20:46:22
|
"Eric M. Ludlam" <er...@si...> writes: > Thanks for the patch. You're guess is correct that creating a > omniscient database for MATLAB is the right way to go. That would > give you the structured needed to prevent rescanning over and over. > There is semanticdb-skel.el, which you can use to start a new > database. Chances are you can just replace SKEL with matlab, copy > your code into the right slots, and be done with it. Though I guessed correctly, I must admit I'm still a bit puzzled why an omniscient database is needed. I thought these omniscient databases are solely for external tag sources, like the one used for Emacs Lisp, where it is used to "wrap" calls to apropos-internal and all-completions. The MATLAB files are scanned and I can see the saved tags in the .semanticdb directory. Also, I can use (semanticdb-file-table-object "somematlabfile.m" t) to get tags from an already scanned file without the need to rescan it, so it seems to me all the necessary tools are already there. This is why I wonder why an omniscient database is needed. > I will likely convert your patch like that before I install it into > the matlab-emacs project, though I know I don't have time for that > this weekend. If you don't find the time, just let me know and I'll try to do it. -David |
From: Eric M. L. <er...@si...> - 2008-07-20 21:39:15
|
>>> David <de...@ar...> seems to think that: >"Eric M. Ludlam" <er...@si...> writes: >> Thanks for the patch. You're guess is correct that creating a >> omniscient database for MATLAB is the right way to go. That would >> give you the structured needed to prevent rescanning over and over. >> There is semanticdb-skel.el, which you can use to start a new >> database. Chances are you can just replace SKEL with matlab, copy >> your code into the right slots, and be done with it. > >Though I guessed correctly, I must admit I'm still a bit puzzled why an >omniscient database is needed. I thought these omniscient databases are >solely for external tag sources, like the one used for Emacs Lisp, where >it is used to "wrap" calls to apropos-internal and all-completions. > >The MATLAB files are scanned and I can see the saved tags in the >.semanticdb directory. Also, I can use > >(semanticdb-file-table-object "somematlabfile.m" t) > >to get tags from an already scanned file without the need to rescan it, >so it seems to me all the necessary tools are already there. This is why >I wonder why an omniscient database is needed. The problem is this: Since Matlab doesn't 'include' a file, any given function must be found out on the path. Since all the functions match up to file names, the data is then `external'. ie - the file system. You could then write something that might use find or whatnot to create a list of all the global files on your path. Then when searching for a function, the omniscient data base would cough up the file by parsing it the normal way. I scanned through your patch to try and understand it better. It looks like you are generating an include tag for each file on the path for the include path. That list is automatically reset whenever a reparse happens. In C, if you reparse a buffer, the list of tags can change. In addition, Semantic builds a crossreference table. All your files become dependant on eachother, so any reparse affacts all the other files. Does that make sense? I think the basic scanning is the right direction. The database would give you a place to then store the info so you don't need to rescan. It could keep directory mod times so it would know if it has to rescan that way instead. >> I will likely convert your patch like that before I install it into >> the matlab-emacs project, though I know I don't have time for that >> this weekend. > >If you don't find the time, just let me know and I'll try to do it. [ ... ] Give it a shot. Let me know if you have questions. Eric -- Eric Ludlam: er...@si... Siege: www.siege-engine.com Emacs: http://cedet.sourceforge.net |
From: David <de...@ar...> - 2008-07-21 09:47:17
|
"Eric M. Ludlam" <er...@si...> writes: > I scanned through your patch to try and understand it better. It > looks like you are generating an include tag for each file on the path > for the include path. Yes. My idea was that a m-file would implicitly include every file on the MATLAB path (without recursion, of course). > That list is automatically reset whenever a reparse happens. In C, if > you reparse a buffer, the list of tags can change. In addition, > Semantic builds a crossreference table. All your files become > dependant on eachother, so any reparse affacts all the other files. > > Does that make sense? Yes. I was hoping that one could somehow disable these "dynamic" features needed for C and other more complex languages. But I understand that an omniscient database is providing exactly that. > Give it a shot. Let me know if you have questions. I'll look into it. -David |
From: David <de...@ar...> - 2008-07-22 14:08:10
Attachments:
semanticdb-matlab.el
semantic-matlab-patch.diff
|
"Eric M. Ludlam" <er...@si...> writes: > You could then write something that might use find or whatnot to > create a list of all the global files on your path. Then when > searching for a function, the omniscient data base would cough up the > file by parsing it the normal way. [...] > Does that make sense? I think the basic scanning is the right > direction. The database would give you a place to then store the info > so you don't need to rescan. It could keep directory mod times so it > would know if it has to rescan that way instead. OK, I've implemented this, based on semanticdb-skel, and it already works really well. I've attached my current version to this mail (semantidb-matlab.el). Reading your above reply, I first thought that I would have to manually care for getting/storing tags and mod times, but I saw that `semanticdb-file-stream' does exactly what I want, including taking care of the mod times and initiate a rescan if necessary. Is this the right approach? I also had to overload `semantic-ctxt-current-class-list' to only care for functions, since this is all we are scanning for at the moment. The variable `semantic-matlab-include-paths' specifies which paths should be scanned for MATLAB files. However, this variable should only contain user specific files, since I also used (defcustom-mode-local-semantic-dependency-system-include-path in semantic-matlab.el to specify directories which contain m-files that ship with MATLAB - I figured those could be seen as "system files". I also changed some other stuff in semantic-matlab.el. The diff is attached. The list of changes: (semantic-matlab-match-function-re): deal with underscores in function names. (semantic-matlab-function-tags): For builtin MATLAB functions, where the m-file only contains documentation, the function name is determined from the doc string. Function arguments and return values are currently set to nil. Additionally, for all functions, the lookfor-part of the doc string is parsed and returned. (semantic-matlab-sort-raw-tags): Include docstring in the tag. (semantic-matlab-root-directory): New variable which specifies the root directory of the MATLAB installation. (semantic-matlab-dependency-system-include-path): Overloaded to specify MATLAB system include paths, which may contain builtin functions. (semantic-matlab-display-docstring): New variable if documentation should be displayed in the modeline when completion was done. (semantic-ia-insert-tag): Overloaded to better insert tags for completion and to display function information in the modeline. Builtin functions from MATLAB are really a problem, since we cannot easily determine the arguments and return values. One could try to parse the doc string, but it can't really see a standard that is followed there, so I guess parsing would be hit and miss. For determining at least the number of arguments, one could use nargin/nargout in the Matlab shell (if available), but without meaningful variable names, this won't really help much. -David |
From: Eric M. L. <er...@si...> - 2008-07-27 18:28:58
|
>>> David <de...@ar...> seems to think that: >"Eric M. Ludlam" <er...@si...> writes: >> You could then write something that might use find or whatnot to >> create a list of all the global files on your path. Then when >> searching for a function, the omniscient data base would cough up the >> file by parsing it the normal way. >[...] >> Does that make sense? I think the basic scanning is the right >> direction. The database would give you a place to then store the info >> so you don't need to rescan. It could keep directory mod times so it >> would know if it has to rescan that way instead. > >OK, I've implemented this, based on semanticdb-skel, and it already >works really well. I've attached my current version to this mail >(semantidb-matlab.el). Hi, Thanks for the great effort! I've been away on vacation. I will try out your code when I get to work (where I do Matlab stuff) in merge it into CVS when I can. Work keeps me pretty busy so this can be slow sometimes. Do you have a sourceforge account? Eric -- Eric Ludlam: er...@si... Siege: www.siege-engine.com Emacs: http://cedet.sourceforge.net |
From: David <de...@ar...> - 2008-07-28 18:30:37
|
"Eric M. Ludlam" <er...@si...> writes: > Do you have a sourceforge account? Yes. Regarding semantic-matlab, I wonder how one would best implement dealing with MATLAB objects. I've only skimmed through the documentation of the new object system in MATLAB 7.6, but I guess this can be parsed quite nicely. However, I'm still working with lots of code which uses the old object system, based the @-directories, so this would currently be my priority. I figure there is not really a "nice" way of parsing this. My starting point would be to simply search for assignments like A = anObject; and look if there exists an @anObject directory in the search path. This directory could then be scanned and analyzed for methods and attributes. I guess this shouldn't be too difficult, but maybe I'm missing something here? -David |
From: Eric M. L. <er...@si...> - 2008-08-19 01:07:53
|
>>> David <de...@ar...> seems to think that: >"Eric M. Ludlam" <er...@si...> writes: >> Do you have a sourceforge account? > >Yes. My apologies for being slow. I've been pretty busy at work and home. The good news is that I successfully threw some Zucchini and got some good distance with my catapult. http://www.siege-engine.com/Zukapult2008.shtml Anyway, I'm finally getting to this (not at work). If you tell me what your sourceforge account is, I can add you as a contributor. To submit your database file into CVS, it would be good for you to add a copyright notice to the top. This is important so that it has correct attribution and copyright information. Also, your root directory variable in the semantic-matlab.el patch is specific to your machine. You probably want to make it generic. :) It also should appear at the top of the file to make it easier to find. I'm not sure how to handle semanticdb-matlab.el. It needs an autoload cookie somewhere, but I'm not quite sure where, as not all users of matlab.el will want to install CEDET, and thus we cannot autoload anything that is Semantic specific. I have compiled your stuff, and will start configuring and using it. (I do hack a lot of M code.) >Regarding semantic-matlab, I wonder how one would best implement dealing >with MATLAB objects. I've only skimmed through the documentation of the >new object system in MATLAB 7.6, but I guess this can be parsed quite >nicely. However, I'm still working with lots of code which uses the old >object system, based the @-directories, so this would currently be my >priority. I figure there is not really a "nice" way of parsing this. My >starting point would be to simply search for assignments like > >A = anObject; > >and look if there exists an @anObject directory in the search path. This >directory could then be scanned and analyzed for methods and >attributes. I guess this shouldn't be too difficult, but maybe I'm >missing something here? [ ... ] The new MATLAB Object system is supported by regexps in matlab.el now. The old OOPS system is a littly funkier, but your semantic database is a great start. Eric -- Eric Ludlam: er...@si... Siege: www.siege-engine.com Emacs: http://cedet.sourceforge.net |