CEDET

Hi,

I'm starting to think perhaps there's a bug, possibly in
semanticdb-find.el (though possibly elsewhere), at least for Java.
The problem is this.  semantic-tag-include-filename for Java
(overridden in semantic-java.el) replaces periods (.) with forward
slashes (/), and sticks ".java" on the end.  So "import
server.Server;" translates to "server/Server.java".  Seems reasonable.
 But then in semanticdb-find-table-for-include-default (in
semanticdb-find.el), toward the end of the defun it loops over "roots"
(tables, I guess) returned by semanticdb-current-database-list, fishes
out the "reference directory" for each such table, forms a
fully-qualified filepath from the filename of the import
("server/Server.java" in my example) and that reference directory.  I
turns out that the only table in "roots" is for the file we're in (the
one we're parsing, which is the one that contains the "import"
statement, which in my example is "client/Client.java") and its
reference directory is "../src/client/".  So the expanded filename
it's looking for, for the include, is in my example
"../src/client/server/Server.java", which is definitely wrong.  No
such file exists, and so the lookup silently fails.  The file it
should be looking for, in my example, is "../src/server/Server.java".
In other words, it's resolving the relative pathname corresponding to
the "include" ("server/Server.java" in my case) with respect to the
path of the including file, rather than with respect to the project
root.

Now, I'm not sure what the correct behavior should be for C/C++ and
other languages that do "includes", but for Java, I think it should
always resolve the relative pathnames for "includes" with respect to
the project root.  If you're using EDE, then I think a file would
belong to just one project, and the EDE routines would find just that
one project and return the correct root.  But even if you're not using
EDE, and you just set semanticdb-project-roots, in the worst case if
it looked for the relative pathname of the "include" with respect to
each of the semanticdb-project-roots it would silently fail for all
but one (the correct one).  Well, I guess the worst case would be if
that include filename existed in several different
semanticdb-project-roots values.  I guess that would be a argument in
favor of using the more sophisticated EDE.  But that's besides the
point.

I know that this--as written--looks a bit complicated, so I'll try to
summarize.  The function semanticdb-find-table-for-include-default
tries to resolve the relative filenames corresponding to Java "import"
statements with respect to the including file.  This is wrong.  It
probably* should be resolving them with respect to the project root.

Cheers,
David

*Note:  It's actually even more complicated than this, for a variety of reasons.
1. Nothing in the "contract" or API of semanticdb leads me to believe
that semanticdb-project-roots has to support resolving filenames of
"include" statements.  It seems mainly (or only) about throttling
searches.  For instance, in my example, there's no reason the one
value in semanticdb-project-roots has to end in "../src".  The user
might set it to be higher up in the directory structure, which would
complicate things.  The meaning of semanticdb-project-roots could be
changed to "absolute pathname with respect to which relative pathnames
for includes will be resolved", but that would be a significant change
and I don't know what repurcussions it would have for semanticdb.
2. I believe Java does not actually require that package structures
(e.g., "server.Server") maps one-to-one to directory structures (e.g.,
"server/Server.java"), though that is the universal convention.  It's
slightly more than just a convention since, if you don't specify
included files on the command-line when compiling Java sources, the
compiler will (typically) guess their location in this way.  In
addition, emitted .class files will be organized this way.  BUT, if
you specify all the related files together on the command-line, they
can be anywhere.
3. To sum up, a quick and dirty solution that probably would cover 99%
to 100% of cases would be to resolve Java "includes" with respect to
the semanticdb-project-roots.  A slightly better solution would be to
leave semanticdb-project-roots intact, and introduce some other
variable to serve the resolving purpose.  The "correct" solution would
require a central database of all encountered Java files, indexed by
their package.  This is probably overkill.


On Sun, Oct 9, 2011 at 10:22 PM, David Ventimiglia <dventimi@...> wrote:
> Hi!
>
> After struggling to get semanticdb-global to work properly, I
> retreated to a simpler configuration using fewer working parts.
> Unfortunately, I'm still running into trouble, and the problem is
> this.  Tags cached onto disk by semanticdb seem not to get loaded, or
> loaded correctly, at least for Java.  This is something I inferred
> after running the following experiment, using GNU Emacs 24.0.50.1 on
> Ubuntu Linux.
>
> 1. Downloaded the bzr repository version of CEDET, getting the latest
> code as of 10/8/2011.
> 2. Compiled it using `make'.
> 3. Loaded it, turned on some coding helpers, and set up semanticdb
> project roots with these commands in my .emacs file.
> (load-file "~/work/cedet/common/cedet.el")
> (semantic-load-enable-excessive-code-helpers)
> (semantic-load-enable-semantic-debugging-helpers)
> (add-to-list 'semanticdb-project-roots (expand-file-name
> "~/work/semantic-java-test/src"))
> 4. Opened $HOME/work/semantic-java-test/src/server/Server.java.
> 5. Opened $HOME/work/semantic-java-test/src/client/Client.java, which
> has one import statement `import server.Server;'.
> 6. Noticed that in Client.java, the import statement `import
> server.Server;' is highlighted in green, and when I right-click on it
> and choose "What is this?", I get a buffer with the following
> contents.
> Include Tag: server.Server
>
> This header tag has been marked "Fileless".
> This means that Semantic cannot find a file associated with this tag
> on disk, but a database table of tags has been associated with it.
>
> This means that the include will still be used to find tags for
> searches, but you connot visit this include.
>
> This Header is now represented by the following database table:
>
>  #<semanticdb-table Server.java (2 tags)>
> 7. Closed Emacs, then looked in $HOME/.semanticdb and see
> semantic.cache files for both the `server' and `client' directories.
> 8. Opened Emacs then opened $HOME/work/semantic-java-test/src/Client.java.
> 9. Noticed that in Client.java, the import statement now is
> highlighted in pink, and when I right-click on it and choose "What is
> this?", I get a buffer with the following.
> Include File: server.Server
>
> This header file has been marked "Unknown".
> This means that Semantic has not been able to locate this file on disk.
>
> When Semantic cannot find an include file, this means that the
> idle summary mode and idle completion modes cannot use the contents of
> that file to provide coding assistance.
>
> If this is a system header and you want it excluded from Semantic's
> searches (which may be desirable for speed reasons) then you can
> safely ignore this state.
>
> If this is a system header, and you want to include it in Semantic's
> searches, then you will need to use:
>
> M-x semantic-add-system-include RET /path/to/includes RET
>
> or, in your .emacs file do:
>
>  (semantic-add-system-include "/path/to/include" 'java-mode)
>
> to add the path to Semantic's search.
>
> If this is an include file that belongs to your project, then you may
> need to update `semanticdb-project-roots' or better yet, use `ede'
> to manage your project.  See the ede manual for projects that will
> wrap existing project code for Semantic's benifit.
>
> See the Semantic manual node on SemanticDB for more about search paths.
>
>
> QUESTIONS:
> 1. I thought semanticdb would load tags from the semantic.cache files
> (if they exist) for Server.java, and so would figure out the import
> statement in Client.java without having to re-parse Server.java.  Do I
> understand this correctly?
> 2. If that's the case, then it seems not to be working correctly.  Is
> that a fair assessment?
> 3. Maybe I've mis-configured it.  My extremely spartan .emacs file is
> pasted in again below.  Is there more to it than just loading CEDET,
> turning on some code helpers, and creating a project root?
> 4. If it's actually broken, where do I need to start looking in the
> semantic code to debug this?
> 5. (side issue).  Even when semantic figures out the import statement
> in Client.java (i.e., when I re-parse Server.java by visiting that
> file), it says it's been marked "Fileless".  What exactly does that
> mean?  Does this also indicate a problem?
>
> I'd love it if someone else would run the same or a similar experiment
> to see if they get the same or similar results.  I would love to solve
> this puzzle since if I can get semantic working smoothly with Java
> code I'll be able to ditch IntelliJ.  :)
>
> Thanks!
> Cheers,
> David
>
> P.S. Here's my .emacs file
> (load-file "~/work/cedet/common/cedet.el")
> (semantic-load-enable-excessive-code-helpers)
> (semantic-load-enable-semantic-debugging-helpers)
> (add-to-list 'semanticdb-project-roots (expand-file-name
> "~/work/semantic-java-test/src"))
>
> P.P.S. Here're Server.java and Client.java
> package server;
> public class Server {
>    public void serve () {}
> }
> --------------------------------------------
> package client;
> import server.Server;
> public class Client {
>    public void doWork () {
>        Server s = new Server();
>    }
> }
>

Here's a patch with a hack fix to the problem.  Basically, I just
added another branch to the cond expression on line 692.  It just
expands the include filename with respect to the first element in
semantidbd-project-roots.  Obviously, not a real solution, but it
demonstrates at least one approach.

=== modified file 'semantic/semanticdb-find.el'
*** semantic/semanticdb-find.el	2011-10-06 17:13:59 +0000
--- semantic/semanticdb-find.el	2011-10-17 06:17:56 +0000
***************
*** 690,695 ****
--- 690,697 ----
  	(let* ((ref (if (slot-boundp (car roots) 'reference-directory)
  			(oref (car roots) reference-directory)))
  	       (fname (cond ((null ref) nil)
+ 			    ((file-exists-p (expand-file-name name (concat (car
semanticdb-project-roots) "/")))
+ 			     (expand-file-name name (concat (car semanticdb-project-roots) "/")))
  			    ((file-exists-p (expand-file-name name ref))
  			     (expand-file-name name ref))
  			    ((file-exists-p (expand-file-name (file-name-nondirectory name) ref))
***************
*** 697,703 ****
  	  (when (and ref fname)
  	    ;; There is an actual file.  Grab it.
  	    (setq ans (semanticdb-find-load-unloaded fname)))
!
  	  ;; ELSE
  	  ;;
  	  ;; NOTE: We used to look up omniscient databases here, but that
--- 699,705 ----
  	  (when (and ref fname)
  	    ;; There is an actual file.  Grab it.
  	    (setq ans (semanticdb-find-load-unloaded fname)))
! 	
  	  ;; ELSE
  	  ;;
  	  ;; NOTE: We used to look up omniscient databases here, but that



On Sun, Oct 16, 2011 at 10:43 PM, David Ventimiglia <dventimi@...> wrote:
> Hi,
>
> I'm starting to think perhaps there's a bug, possibly in
> semanticdb-find.el (though possibly elsewhere), at least for Java.
> The problem is this.  semantic-tag-include-filename for Java
> (overridden in semantic-java.el) replaces periods (.) with forward
> slashes (/), and sticks ".java" on the end.  So "import
> server.Server;" translates to "server/Server.java".  Seems reasonable.
>  But then in semanticdb-find-table-for-include-default (in
> semanticdb-find.el), toward the end of the defun it loops over "roots"
> (tables, I guess) returned by semanticdb-current-database-list, fishes
> out the "reference directory" for each such table, forms a
> fully-qualified filepath from the filename of the import
> ("server/Server.java" in my example) and that reference directory.  I
> turns out that the only table in "roots" is for the file we're in (the
> one we're parsing, which is the one that contains the "import"
> statement, which in my example is "client/Client.java") and its
> reference directory is "../src/client/".  So the expanded filename
> it's looking for, for the include, is in my example
> "../src/client/server/Server.java", which is definitely wrong.  No
> such file exists, and so the lookup silently fails.  The file it
> should be looking for, in my example, is "../src/server/Server.java".
> In other words, it's resolving the relative pathname corresponding to
> the "include" ("server/Server.java" in my case) with respect to the
> path of the including file, rather than with respect to the project
> root.
>
> Now, I'm not sure what the correct behavior should be for C/C++ and
> other languages that do "includes", but for Java, I think it should
> always resolve the relative pathnames for "includes" with respect to
> the project root.  If you're using EDE, then I think a file would
> belong to just one project, and the EDE routines would find just that
> one project and return the correct root.  But even if you're not using
> EDE, and you just set semanticdb-project-roots, in the worst case if
> it looked for the relative pathname of the "include" with respect to
> each of the semanticdb-project-roots it would silently fail for all
> but one (the correct one).  Well, I guess the worst case would be if
> that include filename existed in several different
> semanticdb-project-roots values.  I guess that would be a argument in
> favor of using the more sophisticated EDE.  But that's besides the
> point.
>
> I know that this--as written--looks a bit complicated, so I'll try to
> summarize.  The function semanticdb-find-table-for-include-default
> tries to resolve the relative filenames corresponding to Java "import"
> statements with respect to the including file.  This is wrong.  It
> probably* should be resolving them with respect to the project root.
>
> Cheers,
> David
>
> *Note:  It's actually even more complicated than this, for a variety of reasons.
> 1. Nothing in the "contract" or API of semanticdb leads me to believe
> that semanticdb-project-roots has to support resolving filenames of
> "include" statements.  It seems mainly (or only) about throttling
> searches.  For instance, in my example, there's no reason the one
> value in semanticdb-project-roots has to end in "../src".  The user
> might set it to be higher up in the directory structure, which would
> complicate things.  The meaning of semanticdb-project-roots could be
> changed to "absolute pathname with respect to which relative pathnames
> for includes will be resolved", but that would be a significant change
> and I don't know what repurcussions it would have for semanticdb.
> 2. I believe Java does not actually require that package structures
> (e.g., "server.Server") maps one-to-one to directory structures (e.g.,
> "server/Server.java"), though that is the universal convention.  It's
> slightly more than just a convention since, if you don't specify
> included files on the command-line when compiling Java sources, the
> compiler will (typically) guess their location in this way.  In
> addition, emitted .class files will be organized this way.  BUT, if
> you specify all the related files together on the command-line, they
> can be anywhere.
> 3. To sum up, a quick and dirty solution that probably would cover 99%
> to 100% of cases would be to resolve Java "includes" with respect to
> the semanticdb-project-roots.  A slightly better solution would be to
> leave semanticdb-project-roots intact, and introduce some other
> variable to serve the resolving purpose.  The "correct" solution would
> require a central database of all encountered Java files, indexed by
> their package.  This is probably overkill.
>
>
> On Sun, Oct 9, 2011 at 10:22 PM, David Ventimiglia <dventimi@...> wrote:
>> Hi!
>>
>> After struggling to get semanticdb-global to work properly, I
>> retreated to a simpler configuration using fewer working parts.
>> Unfortunately, I'm still running into trouble, and the problem is
>> this.  Tags cached onto disk by semanticdb seem not to get loaded, or
>> loaded correctly, at least for Java.  This is something I inferred
>> after running the following experiment, using GNU Emacs 24.0.50.1 on
>> Ubuntu Linux.
>>
>> 1. Downloaded the bzr repository version of CEDET, getting the latest
>> code as of 10/8/2011.
>> 2. Compiled it using `make'.
>> 3. Loaded it, turned on some coding helpers, and set up semanticdb
>> project roots with these commands in my .emacs file.
>> (load-file "~/work/cedet/common/cedet.el")
>> (semantic-load-enable-excessive-code-helpers)
>> (semantic-load-enable-semantic-debugging-helpers)
>> (add-to-list 'semanticdb-project-roots (expand-file-name
>> "~/work/semantic-java-test/src"))
>> 4. Opened $HOME/work/semantic-java-test/src/server/Server.java.
>> 5. Opened $HOME/work/semantic-java-test/src/client/Client.java, which
>> has one import statement `import server.Server;'.
>> 6. Noticed that in Client.java, the import statement `import
>> server.Server;' is highlighted in green, and when I right-click on it
>> and choose "What is this?", I get a buffer with the following
>> contents.
>> Include Tag: server.Server
>>
>> This header tag has been marked "Fileless".
>> This means that Semantic cannot find a file associated with this tag
>> on disk, but a database table of tags has been associated with it.
>>
>> This means that the include will still be used to find tags for
>> searches, but you connot visit this include.
>>
>> This Header is now represented by the following database table:
>>
>>  #<semanticdb-table Server.java (2 tags)>
>> 7. Closed Emacs, then looked in $HOME/.semanticdb and see
>> semantic.cache files for both the `server' and `client' directories.
>> 8. Opened Emacs then opened $HOME/work/semantic-java-test/src/Client.java.
>> 9. Noticed that in Client.java, the import statement now is
>> highlighted in pink, and when I right-click on it and choose "What is
>> this?", I get a buffer with the following.
>> Include File: server.Server
>>
>> This header file has been marked "Unknown".
>> This means that Semantic has not been able to locate this file on disk.
>>
>> When Semantic cannot find an include file, this means that the
>> idle summary mode and idle completion modes cannot use the contents of
>> that file to provide coding assistance.
>>
>> If this is a system header and you want it excluded from Semantic's
>> searches (which may be desirable for speed reasons) then you can
>> safely ignore this state.
>>
>> If this is a system header, and you want to include it in Semantic's
>> searches, then you will need to use:
>>
>> M-x semantic-add-system-include RET /path/to/includes RET
>>
>> or, in your .emacs file do:
>>
>>  (semantic-add-system-include "/path/to/include" 'java-mode)
>>
>> to add the path to Semantic's search.
>>
>> If this is an include file that belongs to your project, then you may
>> need to update `semanticdb-project-roots' or better yet, use `ede'
>> to manage your project.  See the ede manual for projects that will
>> wrap existing project code for Semantic's benifit.
>>
>> See the Semantic manual node on SemanticDB for more about search paths.
>>
>>
>> QUESTIONS:
>> 1. I thought semanticdb would load tags from the semantic.cache files
>> (if they exist) for Server.java, and so would figure out the import
>> statement in Client.java without having to re-parse Server.java.  Do I
>> understand this correctly?
>> 2. If that's the case, then it seems not to be working correctly.  Is
>> that a fair assessment?
>> 3. Maybe I've mis-configured it.  My extremely spartan .emacs file is
>> pasted in again below.  Is there more to it than just loading CEDET,
>> turning on some code helpers, and creating a project root?
>> 4. If it's actually broken, where do I need to start looking in the
>> semantic code to debug this?
>> 5. (side issue).  Even when semantic figures out the import statement
>> in Client.java (i.e., when I re-parse Server.java by visiting that
>> file), it says it's been marked "Fileless".  What exactly does that
>> mean?  Does this also indicate a problem?
>>
>> I'd love it if someone else would run the same or a similar experiment
>> to see if they get the same or similar results.  I would love to solve
>> this puzzle since if I can get semantic working smoothly with Java
>> code I'll be able to ditch IntelliJ.  :)
>>
>> Thanks!
>> Cheers,
>> David
>>
>> P.S. Here's my .emacs file
>> (load-file "~/work/cedet/common/cedet.el")
>> (semantic-load-enable-excessive-code-helpers)
>> (semantic-load-enable-semantic-debugging-helpers)
>> (add-to-list 'semanticdb-project-roots (expand-file-name
>> "~/work/semantic-java-test/src"))
>>
>> P.P.S. Here're Server.java and Client.java
>> package server;
>> public class Server {
>>    public void serve () {}
>> }
>> --------------------------------------------
>> package client;
>> import server.Server;
>> public class Client {
>>    public void doWork () {
>>        Server s = new Server();
>>    }
>> }
>>
>

Here's a better patch that fixes this bug.  Basically, in
semanticdb-find.el, in semanticdb-find-table-for-include-default, it
iterates over the elements in semanticdb-project-roots, comparing each
with the original file's directory.  If the original file (the one
doing the including) is in a directory that is contained in the
directory tree rooted at that particular element of
semanticdb-project-root, then it decides that must be this file's root
directory.  It then has a branch in the subsequent cond expression
that looks for the include file, resolved with respect to that root
directory.  Thus, it at least handles having multiple values in
semanticdb-project-root.  If multiple such directories contain the
original file's directory, then it takes the first one.  But then,
what else is it to do?  How would it ever know which is the right one?
 Anyway, there are many ways to do this, and this is just one.  Make
of it what you will.

Cheers,
David

=== modified file 'semantic/semanticdb-find.el'
*** semantic/semanticdb-find.el	2011-10-06 17:13:59 +0000
--- semantic/semanticdb-find.el	2011-10-23 04:59:27 +0000
***************
*** 686,695 ****
--- 686,705 ----

        (setq roots (semanticdb-current-database-list))

+       ;; Get the first--if any--of semanticdb-project-roots that
contains our file's directory.
+       ;; This identifies which among multiple
semanticdb-project-roots is OUR file's project root.
+       (setq semanticdb-root nil)
+       (setq semanticdb-roots (copy-sequence semanticdb-project-roots))
+       (while (not (and semanticdb-roots semanticdb-root))
+ 	(if (string-match (car semanticdb-roots) originfiledir) (setq
semanticdb-root (car semanticdb-roots))
+ 	  (setq semanticdb-roots (cdr semanticdb-roots))))
+
        (while (and (not ans) roots)
  	(let* ((ref (if (slot-boundp (car roots) 'reference-directory)
  			(oref (car roots) reference-directory)))
  	       (fname (cond ((null ref) nil)
+ 			    ((file-exists-p (expand-file-name name semanticdb-root))
+ 			     (expand-file-name name semanticdb-root))
  			    ((file-exists-p (expand-file-name name ref))
  			     (expand-file-name name ref))
  			    ((file-exists-p (expand-file-name (file-name-nondirectory name) ref))
***************
*** 697,703 ****
  	  (when (and ref fname)
  	    ;; There is an actual file.  Grab it.
  	    (setq ans (semanticdb-find-load-unloaded fname)))
!
  	  ;; ELSE
  	  ;;
  	  ;; NOTE: We used to look up omniscient databases here, but that
--- 707,713 ----
  	  (when (and ref fname)
  	    ;; There is an actual file.  Grab it.
  	    (setq ans (semanticdb-find-load-unloaded fname)))
! 	
  	  ;; ELSE
  	  ;;
  	  ;; NOTE: We used to look up omniscient databases here, but that

What you have discovered here is a bit of the organic growth of the 
Semantic libraries, which were not planned out for the long-term in the 
beginning.

There are a couple kinds of searches.  Originally, when you wanted to 
find something (with deprecated semanticdb-search.el functions.)  Those 
just searched all the databases under your project, which was controlled 
by the project roots variable.  Later, augmented by EDE.

The performance on my old dual processor 333 got rather sucky for big 
projects, and I couldn't use it at work, so I devised an improved system 
via EDE for finding header files and creating a much more targeted list 
of tables to search through.  Think C code where a symbol's location on 
disk is more ambiguous.  The list of headers needed to be derived based 
on the project.  In the end, what used to take minutes, could be done in 
a few seconds.  Huzzah!

Your solution mixes the old variable with the new search.  That seems 
like a fine idea.  My worry is that it may solve this problem for you, 
but you will run into other challenges for not just having a working EDE 
project for your situation.  Of course, there aren't good project types 
around for Java.  Do you think that is something easy to create?  There 
are some simple project types for Emacs/Linux, and generic ones in 
ede-generic (see the end of the file) that might make your life easier.

Eric

On 10/23/2011 09:59 PM, David Ventimiglia wrote:
> Here's a better patch that fixes this bug.  Basically, in
> semanticdb-find.el, in semanticdb-find-table-for-include-default, it
> iterates over the elements in semanticdb-project-roots, comparing each
> with the original file's directory.  If the original file (the one
> doing the including) is in a directory that is contained in the
> directory tree rooted at that particular element of
> semanticdb-project-root, then it decides that must be this file's root
> directory.  It then has a branch in the subsequent cond expression
> that looks for the include file, resolved with respect to that root
> directory.  Thus, it at least handles having multiple values in
> semanticdb-project-root.  If multiple such directories contain the
> original file's directory, then it takes the first one.  But then,
> what else is it to do?  How would it ever know which is the right one?
>   Anyway, there are many ways to do this, and this is just one.  Make
> of it what you will.
>
> Cheers,
> David
>
> === modified file 'semantic/semanticdb-find.el'
> *** semantic/semanticdb-find.el	2011-10-06 17:13:59 +0000
> --- semantic/semanticdb-find.el	2011-10-23 04:59:27 +0000
> ***************
> *** 686,695 ****
> --- 686,705 ----
>
>          (setq roots (semanticdb-current-database-list))
>
> +       ;; Get the first--if any--of semanticdb-project-roots that
> contains our file's directory.
> +       ;; This identifies which among multiple
> semanticdb-project-roots is OUR file's project root.
> +       (setq semanticdb-root nil)
> +       (setq semanticdb-roots (copy-sequence semanticdb-project-roots))
> +       (while (not (and semanticdb-roots semanticdb-root))
> + 	(if (string-match (car semanticdb-roots) originfiledir) (setq
> semanticdb-root (car semanticdb-roots))
> + 	  (setq semanticdb-roots (cdr semanticdb-roots))))
> +
>          (while (and (not ans) roots)
>    	(let* ((ref (if (slot-boundp (car roots) 'reference-directory)
>    			(oref (car roots) reference-directory)))
>    	       (fname (cond ((null ref) nil)
> + 			    ((file-exists-p (expand-file-name name semanticdb-root))
> + 			     (expand-file-name name semanticdb-root))
>    			    ((file-exists-p (expand-file-name name ref))
>    			     (expand-file-name name ref))
>    			    ((file-exists-p (expand-file-name (file-name-nondirectory name) ref))
> ***************
> *** 697,703 ****
>    	  (when (and ref fname)
>    	    ;; There is an actual file.  Grab it.
>    	    (setq ans (semanticdb-find-load-unloaded fname)))
> !
>    	  ;; ELSE
>    	  ;;
>    	  ;; NOTE: We used to look up omniscient databases here, but that
> --- 707,713 ----
>    	  (when (and ref fname)
>    	    ;; There is an actual file.  Grab it.
>    	    (setq ans (semanticdb-find-load-unloaded fname)))
> ! 	
>    	  ;; ELSE
>    	  ;;
>    	  ;; NOTE: We used to look up omniscient databases here, but that
>
> ------------------------------------------------------------------------------
> The demand for IT networking professionals continues to grow, and the
> demand for specialized networking skills is growing even more rapidly.
> Take a complimentary Learning@... Self-Assessment and learn
> about Cisco certifications, training, and career opportunities.
> http://p.sf.net/sfu/cisco-dev2dev
> _______________________________________________
> cedet-semantic mailing list
> cedet-semantic@...
> https://lists.sourceforge.net/lists/listinfo/cedet-semantic
>

Hi Eric,

I'm not committed to not using EDE.  I just wanted to build up from
simpler configurations to more complex ones.  EDE seems more
sophisticated and the code seems fresher, but it also covers things I
don't need, like project building (I have my own Autotools system...I
know, which probably could be hooked into EDE).  So I thought I'd
start with the dead-simple semanticdb-project-roots, then later look
at creating EDE project type(s) for Java.

Anyway, I'll see about burnishing my solution, and post a patch.

Best,
David

On Tue, Nov 15, 2011 at 4:59 PM, Eric M. Ludlam <ericludlam@...> wrote:
> What you have discovered here is a bit of the organic growth of the Semantic
> libraries, which were not planned out for the long-term in the beginning.
>
> There are a couple kinds of searches.  Originally, when you wanted to find
> something (with deprecated semanticdb-search.el functions.)  Those just
> searched all the databases under your project, which was controlled by the
> project roots variable.  Later, augmented by EDE.
>
> The performance on my old dual processor 333 got rather sucky for big
> projects, and I couldn't use it at work, so I devised an improved system via
> EDE for finding header files and creating a much more targeted list of
> tables to search through.  Think C code where a symbol's location on disk is
> more ambiguous.  The list of headers needed to be derived based on the
> project.  In the end, what used to take minutes, could be done in a few
> seconds.  Huzzah!
>
> Your solution mixes the old variable with the new search.  That seems like a
> fine idea.  My worry is that it may solve this problem for you, but you will
> run into other challenges for not just having a working EDE project for your
> situation.  Of course, there aren't good project types around for Java.  Do
> you think that is something easy to create?  There are some simple project
> types for Emacs/Linux, and generic ones in ede-generic (see the end of the
> file) that might make your life easier.
>
> Eric
>
> On 10/23/2011 09:59 PM, David Ventimiglia wrote:
>>
>> Here's a better patch that fixes this bug.  Basically, in
>> semanticdb-find.el, in semanticdb-find-table-for-include-default, it
>> iterates over the elements in semanticdb-project-roots, comparing each
>> with the original file's directory.  If the original file (the one
>> doing the including) is in a directory that is contained in the
>> directory tree rooted at that particular element of
>> semanticdb-project-root, then it decides that must be this file's root
>> directory.  It then has a branch in the subsequent cond expression
>> that looks for the include file, resolved with respect to that root
>> directory.  Thus, it at least handles having multiple values in
>> semanticdb-project-root.  If multiple such directories contain the
>> original file's directory, then it takes the first one.  But then,
>> what else is it to do?  How would it ever know which is the right one?
>>  Anyway, there are many ways to do this, and this is just one.  Make
>> of it what you will.
>>
>> Cheers,
>> David
>>
>> === modified file 'semantic/semanticdb-find.el'
>> *** semantic/semanticdb-find.el 2011-10-06 17:13:59 +0000
>> --- semantic/semanticdb-find.el 2011-10-23 04:59:27 +0000
>> ***************
>> *** 686,695 ****
>> --- 686,705 ----
>>
>>         (setq roots (semanticdb-current-database-list))
>>
>> +       ;; Get the first--if any--of semanticdb-project-roots that
>> contains our file's directory.
>> +       ;; This identifies which among multiple
>> semanticdb-project-roots is OUR file's project root.
>> +       (setq semanticdb-root nil)
>> +       (setq semanticdb-roots (copy-sequence semanticdb-project-roots))
>> +       (while (not (and semanticdb-roots semanticdb-root))
>> +       (if (string-match (car semanticdb-roots) originfiledir) (setq
>> semanticdb-root (car semanticdb-roots))
>> +         (setq semanticdb-roots (cdr semanticdb-roots))))
>> +
>>         (while (and (not ans) roots)
>>        (let* ((ref (if (slot-boundp (car roots) 'reference-directory)
>>                        (oref (car roots) reference-directory)))
>>               (fname (cond ((null ref) nil)
>> +                           ((file-exists-p (expand-file-name name
>> semanticdb-root))
>> +                            (expand-file-name name semanticdb-root))
>>                            ((file-exists-p (expand-file-name name ref))
>>                             (expand-file-name name ref))
>>                            ((file-exists-p (expand-file-name
>> (file-name-nondirectory name) ref))
>> ***************
>> *** 697,703 ****
>>          (when (and ref fname)
>>            ;; There is an actual file.  Grab it.
>>            (setq ans (semanticdb-find-load-unloaded fname)))
>> !
>>          ;; ELSE
>>          ;;
>>          ;; NOTE: We used to look up omniscient databases here, but that
>> --- 707,713 ----
>>          (when (and ref fname)
>>            ;; There is an actual file.  Grab it.
>>            (setq ans (semanticdb-find-load-unloaded fname)))
>> !
>>          ;; ELSE
>>          ;;
>>          ;; NOTE: We used to look up omniscient databases here, but that
>>
>>
>> ------------------------------------------------------------------------------
>> The demand for IT networking professionals continues to grow, and the
>> demand for specialized networking skills is growing even more rapidly.
>> Take a complimentary Learning@... Self-Assessment and learn
>> about Cisco certifications, training, and career opportunities.
>> http://p.sf.net/sfu/cisco-dev2dev
>> _______________________________________________
>> cedet-semantic mailing list
>> cedet-semantic@...
>> https://lists.sourceforge.net/lists/listinfo/cedet-semantic
>>
>

Here's a better version, I think.  Instead of trying to determine what
the current file's semanticdb-root is and then matching that to
include's filename in order to resolve it, it just goes straight to
matching each element of semanticdb-roots against the include's
filename.  First one wins.

=== modified file 'semantic/semanticdb-find.el'
*** semantic/semanticdb-find.el	2011-10-06 17:13:59 +0000
--- semantic/semanticdb-find.el	2011-11-16 06:28:27 +0000
***************
*** 687,715 ****
        (setq roots (semanticdb-current-database-list))

        (while (and (not ans) roots)
! 	(let* ((ref (if (slot-boundp (car roots) 'reference-directory)
! 			(oref (car roots) reference-directory)))
! 	       (fname (cond ((null ref) nil)
! 			    ((file-exists-p (expand-file-name name ref))
! 			     (expand-file-name name ref))
! 			    ((file-exists-p (expand-file-name (file-name-nondirectory name) ref))
! 			     (expand-file-name (file-name-nondirectory name) ref)))))
! 	  (when (and ref fname)
! 	    ;; There is an actual file.  Grab it.
! 	    (setq ans (semanticdb-find-load-unloaded fname)))
!
! 	  ;; ELSE
! 	  ;;
! 	  ;; NOTE: We used to look up omniscient databases here, but that
! 	  ;; is now handled one layer up.
! 	  ;;
! 	  ;; Missing: a database that knows where missing files are.  Hmm.
! 	  ;; perhaps I need an override function for that?
!
! 	  )
!
! 	(setq roots (cdr roots))))
!      )
      ans))

  
--- 687,719 ----
        (setq roots (semanticdb-current-database-list))

        (while (and (not ans) roots)
!         (let* ((ref (if (slot-boundp (car roots) 'reference-directory)
!                         (oref (car roots) reference-directory)))
!                (includefilename
!                 (let ((target nil)
!                       (semanticdb-roots (copy-sequence
semanticdb-project-roots)))
!                   (while (and semanticdb-roots (not target))
!                     (if (file-exists-p (expand-file-name name (car
semanticdb-roots))) (setq target (expand-file-name name (car
semanticdb-roots)))
!                       (setq semanticdb-roots (cdr semanticdb-roots))))
!                   (if (null target) ""
! 		    target)))
!                (fname (cond ((null ref) nil)
!                             ((file-exists-p includefilename) includefilename)
!                             ((file-exists-p (expand-file-name name ref))
!                              (expand-file-name name ref))
!                             ((file-exists-p (expand-file-name
(file-name-nondirectory name) ref))
!                              (expand-file-name
(file-name-nondirectory name) ref)))))
!           (when (and ref fname)
!             ;; There is an actual file.  Grab it.
!             (setq ans (semanticdb-find-load-unloaded fname))))
!         ;; ELSE
!         ;;
!         ;; NOTE: We used to look up omniscient databases here, but that
!         ;; is now handled one layer up.
!         ;;
!         ;; Missing: a database that knows where missing files are.  Hmm.
!         ;; perhaps I need an override function for that?
!         (setq roots (cdr roots)))))
      ans))



On Tue, Nov 15, 2011 at 5:30 PM, David Ventimiglia <dventimi@...> wrote:
> Hi Eric,
>
> I'm not committed to not using EDE.  I just wanted to build up from
> simpler configurations to more complex ones.  EDE seems more
> sophisticated and the code seems fresher, but it also covers things I
> don't need, like project building (I have my own Autotools system...I
> know, which probably could be hooked into EDE).  So I thought I'd
> start with the dead-simple semanticdb-project-roots, then later look
> at creating EDE project type(s) for Java.
>
> Anyway, I'll see about burnishing my solution, and post a patch.
>
> Best,
> David
>
> On Tue, Nov 15, 2011 at 4:59 PM, Eric M. Ludlam <ericludlam@...> wrote:
>> What you have discovered here is a bit of the organic growth of the Semantic
>> libraries, which were not planned out for the long-term in the beginning.
>>
>> There are a couple kinds of searches.  Originally, when you wanted to find
>> something (with deprecated semanticdb-search.el functions.)  Those just
>> searched all the databases under your project, which was controlled by the
>> project roots variable.  Later, augmented by EDE.
>>
>> The performance on my old dual processor 333 got rather sucky for big
>> projects, and I couldn't use it at work, so I devised an improved system via
>> EDE for finding header files and creating a much more targeted list of
>> tables to search through.  Think C code where a symbol's location on disk is
>> more ambiguous.  The list of headers needed to be derived based on the
>> project.  In the end, what used to take minutes, could be done in a few
>> seconds.  Huzzah!
>>
>> Your solution mixes the old variable with the new search.  That seems like a
>> fine idea.  My worry is that it may solve this problem for you, but you will
>> run into other challenges for not just having a working EDE project for your
>> situation.  Of course, there aren't good project types around for Java.  Do
>> you think that is something easy to create?  There are some simple project
>> types for Emacs/Linux, and generic ones in ede-generic (see the end of the
>> file) that might make your life easier.
>>
>> Eric
>>
>> On 10/23/2011 09:59 PM, David Ventimiglia wrote:
>>>
>>> Here's a better patch that fixes this bug.  Basically, in
>>> semanticdb-find.el, in semanticdb-find-table-for-include-default, it
>>> iterates over the elements in semanticdb-project-roots, comparing each
>>> with the original file's directory.  If the original file (the one
>>> doing the including) is in a directory that is contained in the
>>> directory tree rooted at that particular element of
>>> semanticdb-project-root, then it decides that must be this file's root
>>> directory.  It then has a branch in the subsequent cond expression
>>> that looks for the include file, resolved with respect to that root
>>> directory.  Thus, it at least handles having multiple values in
>>> semanticdb-project-root.  If multiple such directories contain the
>>> original file's directory, then it takes the first one.  But then,
>>> what else is it to do?  How would it ever know which is the right one?
>>>  Anyway, there are many ways to do this, and this is just one.  Make
>>> of it what you will.
>>>
>>> Cheers,
>>> David
>>>
>>> === modified file 'semantic/semanticdb-find.el'
>>> *** semantic/semanticdb-find.el 2011-10-06 17:13:59 +0000
>>> --- semantic/semanticdb-find.el 2011-10-23 04:59:27 +0000
>>> ***************
>>> *** 686,695 ****
>>> --- 686,705 ----
>>>
>>>         (setq roots (semanticdb-current-database-list))
>>>
>>> +       ;; Get the first--if any--of semanticdb-project-roots that
>>> contains our file's directory.
>>> +       ;; This identifies which among multiple
>>> semanticdb-project-roots is OUR file's project root.
>>> +       (setq semanticdb-root nil)
>>> +       (setq semanticdb-roots (copy-sequence semanticdb-project-roots))
>>> +       (while (not (and semanticdb-roots semanticdb-root))
>>> +       (if (string-match (car semanticdb-roots) originfiledir) (setq
>>> semanticdb-root (car semanticdb-roots))
>>> +         (setq semanticdb-roots (cdr semanticdb-roots))))
>>> +
>>>         (while (and (not ans) roots)
>>>        (let* ((ref (if (slot-boundp (car roots) 'reference-directory)
>>>                        (oref (car roots) reference-directory)))
>>>               (fname (cond ((null ref) nil)
>>> +                           ((file-exists-p (expand-file-name name
>>> semanticdb-root))
>>> +                            (expand-file-name name semanticdb-root))
>>>                            ((file-exists-p (expand-file-name name ref))
>>>                             (expand-file-name name ref))
>>>                            ((file-exists-p (expand-file-name
>>> (file-name-nondirectory name) ref))
>>> ***************
>>> *** 697,703 ****
>>>          (when (and ref fname)
>>>            ;; There is an actual file.  Grab it.
>>>            (setq ans (semanticdb-find-load-unloaded fname)))
>>> !
>>>          ;; ELSE
>>>          ;;
>>>          ;; NOTE: We used to look up omniscient databases here, but that
>>> --- 707,713 ----
>>>          (when (and ref fname)
>>>            ;; There is an actual file.  Grab it.
>>>            (setq ans (semanticdb-find-load-unloaded fname)))
>>> !
>>>          ;; ELSE
>>>          ;;
>>>          ;; NOTE: We used to look up omniscient databases here, but that
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> The demand for IT networking professionals continues to grow, and the
>>> demand for specialized networking skills is growing even more rapidly.
>>> Take a complimentary Learning@... Self-Assessment and learn
>>> about Cisco certifications, training, and career opportunities.
>>> http://p.sf.net/sfu/cisco-dev2dev
>>> _______________________________________________
>>> cedet-semantic mailing list
>>> cedet-semantic@...
>>> https://lists.sourceforge.net/lists/listinfo/cedet-semantic
>>>
>>
>

Ok, classic blunder. Made a change without testing it.  Attached is a
patch file with one fewer bugs (hopefully bring the bug count down to
0).