lxr-developer — Don't use, active developers only

[Lxr-dev] [ lxr-Feature Requests-3578666 ] Add ignorefiles and extend ignoredirs

[SourceForge.net <no...@so...>]

Feature Requests item #3578666, was opened at 2012-10-20 04:14
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390120&aid=3578666&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
>Category: None
>Group: None
Status: Open
Priority: 5
Private: No
Submitted By: Lukasz M (myny)
Assigned to: Nobody/Anonymous (nobody)
Summary: Add ignorefiles and extend ignoredirs

Initial Comment:
It would be nice to add possibility to add ignorefiles option for files just like ignoredirs for directories. I have added this for my lxr and it's just 3 lines of code.
Would it be also possible for ignoredirs option to handle regexp? I would like to exclude /include dir from indexing (as header files are also within libs).

----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2012-11-01 01:21

Message:
Transferred from "support request" to "feature request"

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2012-10-24 07:55

Message:
Mmmh! Your "specification" is hard to twist into the present
implementation. It was designed to be rather efficient: 'ignoredirs' is
taken into consideration when function getdir() is invoked to enumerate the
content of a directory. 'ignoredirs' subdirectories are filtered here. This
is also where 'ignorefiles' could be filtered. But, only this very "local"
path element is compared, not the whole absolute path.

This is very good for large sized projects such as the Linux kernel (~37
000 files and hundreds, maybe thousands, directories). I want to keep
performance on such projects.

'ignoredirs' is also scanned in toreal() function with pattern matching.
This is compatible with a longer path fragment (i.e. containing path
separators). But this function exists only ib Plain.pm and CVS.pm, not in
GIT.pm nor Subversion.pm. Consequently, this is not the place for
implementation.

While I think about an angle of attack, what about the following strategy
since your concern is to prevent duplicates from entering into the DB:
- before genxref step, disable (or remove) the links (ln) causing the
duplicates,
- launch genxref to create the DB without duplicates,
- recreate the links.

This could temporarily solve your problem. If there are too many links, you
can design a small script so that you only type a short command to do the
removal/creation.

----------------------------------------------------------------------

Comment By: Lukasz M (myny)
Date: 2012-10-21 08:48

Message:
Exactly, I have duplicate files in /include folder. Due to that any search
results in duplcate results. I also cannot add this folder to ignoredirs
because I have some other include dirs in some libs. So really I would like
to ignore only /include folder and not /somelib/include.

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2012-10-21 08:28

Message:
I experimented with 'filter' and finally got it right. To include only Perl
files for instance, add in lxr.conf:

, 'filter' => '(\\/$|\\.pm$)'

The first alternative keeps directories (they have a canonical trailing
slash as fixed in LXR::Common::httpinit); the second keeps only .pm files;

I admit that this INCLUDE rule is probably less flexible as in EXCLUDE
rule. Second, it does not prevent genxref from indexing. I'll add an
'ignorefiles' parameter for both genxref and source.

Could you better explain your "exclude /include dir from indexing (as
header files are also within libs)". Do you mean there is a link resulting
in duplicate files: one set accessed through /include and another accessed
through /libs? I'll see if the "already indexed" featured can cope with
this. Otherwise, add one of the set to 'ignoredirs'.

----------------------------------------------------------------------

Comment By: Lukasz M (myny)
Date: 2012-10-21 02:00

Message:
Actually, I do not mind displaying the file/directory. I would rather it
not be indexed. 
For ignoring files (from indexing) I just modified the following:

LXR/Files/Plain.pm

                # Check directories to ignore
                if (-d $dir . $node) {
                        foreach my $ignoredir (@{$config->{'ignoredirs'}})
{
                                next FILE if $node eq $ignoredir;
                        }
                        # Directory to keep: suffix name with a slash
                        push(@dirs, $node . '/');
                } else {
-->                        foreach my $ignorefile ($config->ignorefiles) {
-->                                next FILE if $node eq $ignorefile;
-->                        }
                        # File: don't change the name
                        push(@files, $node);
                }

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2012-10-20 07:12

Message:
Well, this is new feature which could be included in the next release.

1/ regexp for 'ignoredirs'
I had a quick look at the code sections related to 'ignoredirs'. The change
seems to involve the 'getdir' sub in the vatious Files/ handlers.

2/ files exclusion
I wonder if it is not already there. Look at the end of 'source' script
(lines 401-406 in release 1.0). There is an undocumented call to lxr.conf's
parameter 'filter'. It looks like it should be a regexp SELECTING (not
excluding) which file or directory is displayed. This has been lurking in
'source' for ages and I really never succeeded in setting it up correctly.
The main difficulty for the regexp is to be valid both for directories
(otherwise they can't be listed) and for wanted files. All failures end up
with 'fil does not exist' (which is what I always got!). You might
experiment with it.

Note that this does not exclude files from indexing, meaning you have no
speed improvement in genxref.

I suppose your 3-line solution is something equivalent to line 242 (release
1.0) which excludes *.o, *.a, core files (and also the index files of the
initial LXR implementation). Can you send your patch?

Best regards
ajl

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390120&aid=3578666&group_id=27350

[Lxr-dev] [ lxr-Bugs-3581966 ] Tabhint not activated in language parser

[SourceForge.net <no...@so...>]

Bugs item #3581966, was opened at 2012-10-30 11:34
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3581966&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Lang support
>Group: v1.0
>Status: Closed
>Resolution: Fixed
Priority: 3
Private: No
Submitted By: Andre-Littoz (ajlittoz)
Assigned to: Andre-Littoz (ajlittoz)
Summary: Tabhint not activated in language parser

Initial Comment:
v1.0 (but present in previous releases)

Language name from filetype.conf not captured in variable $langkey when creating a new language scanner (sub new in Lang.pm). Perl doc says: loop variable in foreach is localized when entering the loop and is restored in its previous value when exiting the loop. Consequently, you cannot keep the last iteration value in the variable.

Fix: use a different variable for the loop and transfer its value into $langkey before exiting.

----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2012-10-31 11:20

Message:
Fixed as proposed in CVS

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3581966&group_id=27350

[Lxr-dev] [ lxr-Bugs-3581487 ] Empty 'virtroot' not recognized

[SourceForge.net <no...@so...>]

Bugs item #3581487, was opened at 2012-10-28 19:21
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3581487&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: genxref
Group: v1.0
>Status: Closed
>Resolution: Fixed
Priority: 7
Private: No
Submitted By: Olivier Duclos (oliwer_)
Assigned to: Andre-Littoz (ajlittoz)
Summary: Empty 'virtroot' not recognized

Initial Comment:
When launching genxref with the command './genxref --url=http://lxr.domain.com --allversions' I get the following error:

No matching configuration at ./genxref line 312, <FILETYPE> line 1.

I tried to track down the error and found this a few lines above :

if ($option{'url'}) {   # Single 'url'
        @config = (1);          # Fake list to prevent looping
}

This may prevent looping but the rest of the script needed the data in @config. I tried commenting this passage, but then the script exits without any error and without having generating the database files!

This was on Debian Squeeze with Perl 5.10.1.

----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2012-10-31 11:16

Message:
Fixed in CVS
Doc already stated that 'virtroot' should contain at least '/'. Now, this
is enforced in Config.pm.

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2012-10-30 01:28

Message:
Changing name of bug from "genxref crash while reading configuration" to
"Empty 'virtroot' not recognized' with higher severity.

Fix in Config.pm sub _initialize: make sure script path is never empty
(should at least contain /), check 'virtroot' is also never empty (contains
at least /, internally changed if not)

Fix doc about 'virtroot' so that it always starts with /.

@olivier_: word "hostname" in doc may be confusing. What is needed is an
URL prefix containing scheme (option), // separator and hostname proper.
What would be an unambiguous word for that?

----------------------------------------------------------------------

Comment By: Olivier Duclos (oliwer_)
Date: 2012-10-29 12:44

Message:
Thanks! You're diagnostic was right but incomplete.

After modifying the regex at line 248, the script continued ($hits == 1
before the test), but I still got the error "Can't find the config
for...".

So I continued debugging. A few lines down there is this test to match URLs
:

if      (       $host eq $rt
                &&      $script_path eq $virtroot
        )

Here are the values of the variables just be fore the test :

HOST=http://lxr.sleepycat.fr
RT=lxr.sleepycat.fr
SCRIPT_PATH=/
VIRTROOT=

So the test failed. To fix the value of $rt, I just prefixed my host_names
with '//' in lxr.conf. I guess this is my fault because the doc says you
need to put // or http:// when entering a hostname. (But then it's not a
hostname anymore!)

I wasn't sure how to fix the value of $script_path as there are several
regex involved. I ended up adding this line after line 232 :

 $script_path = '' if ($script_path eq '/');

Now all the values match! My sources are being indexed as I write...

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2012-10-29 09:25

Message:
I must admit this is the only case I haven't tested in depth.

Though I'm rather busy professionally speaking, I may have found something
in in Config.pm sub _initialize. Target URL is first split into components
host and script path. Unhappily, this process assumes there is always a
script path after the hostname. If there is none, the regexp captures
nothing and the internal hostname is EMPTY. Consequently, this internal
host name never matches any in 'host_names'.

If you have more time than me, try the following:
in Config.pm, change line 248 from

       $hits += $virtroot =~ s!^/+!/!;     # and a single starting /

to

       $hits += $virtroot =~ s!^/*!/!;     # and a single starting /
(replace + by * inside regexp)

Explanation: your 'virtroot' contains an empty string and at least a path
separator is needed to build the URL path up to the script name.

Then try the following command:
./genxref --url=//lxr.domain.com/ --alversions

i.e. add a trailing / to your URL.

If it works, it will confirm the diagnostic and I'll design something clean
to cover this case.

Please report back.

----------------------------------------------------------------------

Comment By: Olivier Duclos (oliwer_)
Date: 2012-10-29 07:02

Message:
Thanks for your quick reply.

I should have started from the beginning. My lxr.conf contains this:

'host_names' =>       [ 'lxr.sleepycat.fr' ]
'virtroot'    => ''

When I launch ./genxref --url=//lxr.sleepycat.fr --allversions   I get
this:
Can't find config for http://lxr.sleepycat.fr/: ...

I tried adding 'http://' in the hostname and putting '/' in virtroot but it
doesn't change anything. Then I added this line to lxr.conf : 

, 'baseurl' => 'http://lxr.sleepycat.fr'

and it got me to the problem mentioned above. I'm stuck!

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2012-10-29 02:51

Message:
Hi,

Your analysis is wrong: inside the loop, access to data in @config (aka.
through variable $treedescr) is protected by another "if ($option("url"))"
to discriminate between single and multiple URL indexing. It should work as
is.

Check that your --url argument is correct. As I read it, it only contains
the target hostname, i.e. without 'virtroot'. The genxref line should read
something like:

./genxref --url=http://lxr.domain.com/virtroot --allversions

unless, of course, your 'virtroot' is only '/', meaning the host only
contains a single LXR tree (without any other page).

If you forgot nothing, send your lxr.conf.

Regards,
ajl

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3581487&group_id=27350

[Lxr-dev] [ lxr-Bugs-3581966 ] Tabhint not activated in language parser

[SourceForge.net <no...@so...>]

Bugs item #3581966, was opened at 2012-10-30 11:34
Message generated for change (Tracker Item Submitted) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3581966&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Lang support
Group: v1.1
Status: Open
Resolution: None
Priority: 3
Private: No
Submitted By: Andre-Littoz (ajlittoz)
Assigned to: Andre-Littoz (ajlittoz)
Summary: Tabhint not activated in language parser

Initial Comment:
v1.0 (but present in previous releases)

Language name from filetype.conf not captured in variable $langkey when creating a new language scanner (sub new in Lang.pm). Perl doc says: loop variable in foreach is localized when entering the loop and is restored in its previous value when exiting the loop. Consequently, you cannot keep the last iteration value in the variable.

Fix: use a different variable for the loop and transfer its value into $langkey before exiting.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3581966&group_id=27350

[Lxr-dev] [ lxr-Bugs-3581487 ] Empty 'virtroot' not recognized

[SourceForge.net <no...@so...>]

Bugs item #3581487, was opened at 2012-10-28 19:21
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3581487&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: genxref
Group: v1.0
Status: Open
Resolution: None
>Priority: 7
Private: No
Submitted By: Olivier Duclos (oliwer_)
>Assigned to: Andre-Littoz (ajlittoz)
>Summary: Empty 'virtroot' not recognized

Initial Comment:
When launching genxref with the command './genxref --url=http://lxr.domain.com --allversions' I get the following error:

No matching configuration at ./genxref line 312, <FILETYPE> line 1.

I tried to track down the error and found this a few lines above :

if ($option{'url'}) {   # Single 'url'
        @config = (1);          # Fake list to prevent looping
}

This may prevent looping but the rest of the script needed the data in @config. I tried commenting this passage, but then the script exits without any error and without having generating the database files!

This was on Debian Squeeze with Perl 5.10.1.

----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2012-10-30 01:28

Message:
Changing name of bug from "genxref crash while reading configuration" to
"Empty 'virtroot' not recognized' with higher severity.

Fix in Config.pm sub _initialize: make sure script path is never empty
(should at least contain /), check 'virtroot' is also never empty (contains
at least /, internally changed if not)

Fix doc about 'virtroot' so that it always starts with /.

@olivier_: word "hostname" in doc may be confusing. What is needed is an
URL prefix containing scheme (option), // separator and hostname proper.
What would be an unambiguous word for that?

----------------------------------------------------------------------

Comment By: Olivier Duclos (oliwer_)
Date: 2012-10-29 12:44

Message:
Thanks! You're diagnostic was right but incomplete.

After modifying the regex at line 248, the script continued ($hits == 1
before the test), but I still got the error "Can't find the config
for...".

So I continued debugging. A few lines down there is this test to match URLs
:

if      (       $host eq $rt
                &&      $script_path eq $virtroot
        )

Here are the values of the variables just be fore the test :

HOST=http://lxr.sleepycat.fr
RT=lxr.sleepycat.fr
SCRIPT_PATH=/
VIRTROOT=

So the test failed. To fix the value of $rt, I just prefixed my host_names
with '//' in lxr.conf. I guess this is my fault because the doc says you
need to put // or http:// when entering a hostname. (But then it's not a
hostname anymore!)

I wasn't sure how to fix the value of $script_path as there are several
regex involved. I ended up adding this line after line 232 :

 $script_path = '' if ($script_path eq '/');

Now all the values match! My sources are being indexed as I write...

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2012-10-29 09:25

Message:
I must admit this is the only case I haven't tested in depth.

Though I'm rather busy professionally speaking, I may have found something
in in Config.pm sub _initialize. Target URL is first split into components
host and script path. Unhappily, this process assumes there is always a
script path after the hostname. If there is none, the regexp captures
nothing and the internal hostname is EMPTY. Consequently, this internal
host name never matches any in 'host_names'.

If you have more time than me, try the following:
in Config.pm, change line 248 from

       $hits += $virtroot =~ s!^/+!/!;     # and a single starting /

to

       $hits += $virtroot =~ s!^/*!/!;     # and a single starting /
(replace + by * inside regexp)

Explanation: your 'virtroot' contains an empty string and at least a path
separator is needed to build the URL path up to the script name.

Then try the following command:
./genxref --url=//lxr.domain.com/ --alversions

i.e. add a trailing / to your URL.

If it works, it will confirm the diagnostic and I'll design something clean
to cover this case.

Please report back.

----------------------------------------------------------------------

Comment By: Olivier Duclos (oliwer_)
Date: 2012-10-29 07:02

Message:
Thanks for your quick reply.

I should have started from the beginning. My lxr.conf contains this:

'host_names' =>       [ 'lxr.sleepycat.fr' ]
'virtroot'    => ''

When I launch ./genxref --url=//lxr.sleepycat.fr --allversions   I get
this:
Can't find config for http://lxr.sleepycat.fr/: ...

I tried adding 'http://' in the hostname and putting '/' in virtroot but it
doesn't change anything. Then I added this line to lxr.conf : 

, 'baseurl' => 'http://lxr.sleepycat.fr'

and it got me to the problem mentioned above. I'm stuck!

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2012-10-29 02:51

Message:
Hi,

Your analysis is wrong: inside the loop, access to data in @config (aka.
through variable $treedescr) is protected by another "if ($option("url"))"
to discriminate between single and multiple URL indexing. It should work as
is.

Check that your --url argument is correct. As I read it, it only contains
the target hostname, i.e. without 'virtroot'. The genxref line should read
something like:

./genxref --url=http://lxr.domain.com/virtroot --allversions

unless, of course, your 'virtroot' is only '/', meaning the host only
contains a single LXR tree (without any other page).

If you forgot nothing, send your lxr.conf.

Regards,
ajl

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3581487&group_id=27350

[Lxr-dev] [ lxr-Bugs-3581487 ] genxref crash while reading configuration

[SourceForge.net <no...@so...>]

Bugs item #3581487, was opened at 2012-10-28 19:21
Message generated for change (Comment added) made by oliwer_
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3581487&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: genxref
Group: v1.0
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Olivier Duclos (oliwer_)
Assigned to: Nobody/Anonymous (nobody)
Summary: genxref crash while reading configuration

Initial Comment:
When launching genxref with the command './genxref --url=http://lxr.domain.com --allversions' I get the following error:

No matching configuration at ./genxref line 312, <FILETYPE> line 1.

I tried to track down the error and found this a few lines above :

if ($option{'url'}) {   # Single 'url'
        @config = (1);          # Fake list to prevent looping
}

This may prevent looping but the rest of the script needed the data in @config. I tried commenting this passage, but then the script exits without any error and without having generating the database files!

This was on Debian Squeeze with Perl 5.10.1.

----------------------------------------------------------------------

>Comment By: Olivier Duclos (oliwer_)
Date: 2012-10-29 12:44

Message:
Thanks! You're diagnostic was right but incomplete.

After modifying the regex at line 248, the script continued ($hits == 1
before the test), but I still got the error "Can't find the config
for...".

So I continued debugging. A few lines down there is this test to match URLs
:

if      (       $host eq $rt
                &&      $script_path eq $virtroot
        )

Here are the values of the variables just be fore the test :

HOST=http://lxr.sleepycat.fr
RT=lxr.sleepycat.fr
SCRIPT_PATH=/
VIRTROOT=

So the test failed. To fix the value of $rt, I just prefixed my host_names
with '//' in lxr.conf. I guess this is my fault because the doc says you
need to put // or http:// when entering a hostname. (But then it's not a
hostname anymore!)

I wasn't sure how to fix the value of $script_path as there are several
regex involved. I ended up adding this line after line 232 :

 $script_path = '' if ($script_path eq '/');

Now all the values match! My sources are being indexed as I write...

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2012-10-29 09:25

Message:
I must admit this is the only case I haven't tested in depth.

Though I'm rather busy professionally speaking, I may have found something
in in Config.pm sub _initialize. Target URL is first split into components
host and script path. Unhappily, this process assumes there is always a
script path after the hostname. If there is none, the regexp captures
nothing and the internal hostname is EMPTY. Consequently, this internal
host name never matches any in 'host_names'.

If you have more time than me, try the following:
in Config.pm, change line 248 from

       $hits += $virtroot =~ s!^/+!/!;     # and a single starting /

to

       $hits += $virtroot =~ s!^/*!/!;     # and a single starting /
(replace + by * inside regexp)

Explanation: your 'virtroot' contains an empty string and at least a path
separator is needed to build the URL path up to the script name.

Then try the following command:
./genxref --url=//lxr.domain.com/ --alversions

i.e. add a trailing / to your URL.

If it works, it will confirm the diagnostic and I'll design something clean
to cover this case.

Please report back.

----------------------------------------------------------------------

Comment By: Olivier Duclos (oliwer_)
Date: 2012-10-29 07:02

Message:
Thanks for your quick reply.

I should have started from the beginning. My lxr.conf contains this:

'host_names' =>       [ 'lxr.sleepycat.fr' ]
'virtroot'    => ''

When I launch ./genxref --url=//lxr.sleepycat.fr --allversions   I get
this:
Can't find config for http://lxr.sleepycat.fr/: ...

I tried adding 'http://' in the hostname and putting '/' in virtroot but it
doesn't change anything. Then I added this line to lxr.conf : 

, 'baseurl' => 'http://lxr.sleepycat.fr'

and it got me to the problem mentioned above. I'm stuck!

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2012-10-29 02:51

Message:
Hi,

Your analysis is wrong: inside the loop, access to data in @config (aka.
through variable $treedescr) is protected by another "if ($option("url"))"
to discriminate between single and multiple URL indexing. It should work as
is.

Check that your --url argument is correct. As I read it, it only contains
the target hostname, i.e. without 'virtroot'. The genxref line should read
something like:

./genxref --url=http://lxr.domain.com/virtroot --allversions

unless, of course, your 'virtroot' is only '/', meaning the host only
contains a single LXR tree (without any other page).

If you forgot nothing, send your lxr.conf.

Regards,
ajl

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3581487&group_id=27350

[Lxr-dev] [ lxr-Bugs-3581487 ] genxref crash while reading configuration

[SourceForge.net <no...@so...>]

Bugs item #3581487, was opened at 2012-10-28 19:21
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3581487&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: genxref
Group: v1.0
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Olivier Duclos (oliwer_)
Assigned to: Nobody/Anonymous (nobody)
Summary: genxref crash while reading configuration

Initial Comment:
When launching genxref with the command './genxref --url=http://lxr.domain.com --allversions' I get the following error:

No matching configuration at ./genxref line 312, <FILETYPE> line 1.

I tried to track down the error and found this a few lines above :

if ($option{'url'}) {   # Single 'url'
        @config = (1);          # Fake list to prevent looping
}

This may prevent looping but the rest of the script needed the data in @config. I tried commenting this passage, but then the script exits without any error and without having generating the database files!

This was on Debian Squeeze with Perl 5.10.1.

----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2012-10-29 09:25

Message:
I must admit this is the only case I haven't tested in depth.

Though I'm rather busy professionally speaking, I may have found something
in in Config.pm sub _initialize. Target URL is first split into components
host and script path. Unhappily, this process assumes there is always a
script path after the hostname. If there is none, the regexp captures
nothing and the internal hostname is EMPTY. Consequently, this internal
host name never matches any in 'host_names'.

If you have more time than me, try the following:
in Config.pm, change line 248 from

       $hits += $virtroot =~ s!^/+!/!;     # and a single starting /

to

       $hits += $virtroot =~ s!^/*!/!;     # and a single starting /
(replace + by * inside regexp)

Explanation: your 'virtroot' contains an empty string and at least a path
separator is needed to build the URL path up to the script name.

Then try the following command:
./genxref --url=//lxr.domain.com/ --alversions

i.e. add a trailing / to your URL.

If it works, it will confirm the diagnostic and I'll design something clean
to cover this case.

Please report back.

----------------------------------------------------------------------

Comment By: Olivier Duclos (oliwer_)
Date: 2012-10-29 07:02

Message:
Thanks for your quick reply.

I should have started from the beginning. My lxr.conf contains this:

'host_names' =>       [ 'lxr.sleepycat.fr' ]
'virtroot'    => ''

When I launch ./genxref --url=//lxr.sleepycat.fr --allversions   I get
this:
Can't find config for http://lxr.sleepycat.fr/: ...

I tried adding 'http://' in the hostname and putting '/' in virtroot but it
doesn't change anything. Then I added this line to lxr.conf : 

, 'baseurl' => 'http://lxr.sleepycat.fr'

and it got me to the problem mentioned above. I'm stuck!

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2012-10-29 02:51

Message:
Hi,

Your analysis is wrong: inside the loop, access to data in @config (aka.
through variable $treedescr) is protected by another "if ($option("url"))"
to discriminate between single and multiple URL indexing. It should work as
is.

Check that your --url argument is correct. As I read it, it only contains
the target hostname, i.e. without 'virtroot'. The genxref line should read
something like:

./genxref --url=http://lxr.domain.com/virtroot --allversions

unless, of course, your 'virtroot' is only '/', meaning the host only
contains a single LXR tree (without any other page).

If you forgot nothing, send your lxr.conf.

Regards,
ajl

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3581487&group_id=27350

[Lxr-dev] [ lxr-Bugs-3581487 ] genxref crash while reading configuration

[SourceForge.net <no...@so...>]

Bugs item #3581487, was opened at 2012-10-28 19:21
Message generated for change (Comment added) made by oliwer_
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3581487&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: genxref
Group: v1.0
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Olivier Duclos (oliwer_)
Assigned to: Nobody/Anonymous (nobody)
Summary: genxref crash while reading configuration

Initial Comment:
When launching genxref with the command './genxref --url=http://lxr.domain.com --allversions' I get the following error:

No matching configuration at ./genxref line 312, <FILETYPE> line 1.

I tried to track down the error and found this a few lines above :

if ($option{'url'}) {   # Single 'url'
        @config = (1);          # Fake list to prevent looping
}

This may prevent looping but the rest of the script needed the data in @config. I tried commenting this passage, but then the script exits without any error and without having generating the database files!

This was on Debian Squeeze with Perl 5.10.1.

----------------------------------------------------------------------

>Comment By: Olivier Duclos (oliwer_)
Date: 2012-10-29 07:02

Message:
Thanks for your quick reply.

I should have started from the beginning. My lxr.conf contains this:

'host_names' =>       [ 'lxr.sleepycat.fr' ]
'virtroot'    => ''

When I launch ./genxref --url=//lxr.sleepycat.fr --allversions   I get
this:
Can't find config for http://lxr.sleepycat.fr/: ...

I tried adding 'http://' in the hostname and putting '/' in virtroot but it
doesn't change anything. Then I added this line to lxr.conf : 

, 'baseurl' => 'http://lxr.sleepycat.fr'

and it got me to the problem mentioned above. I'm stuck!

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2012-10-29 02:51

Message:
Hi,

Your analysis is wrong: inside the loop, access to data in @config (aka.
through variable $treedescr) is protected by another "if ($option("url"))"
to discriminate between single and multiple URL indexing. It should work as
is.

Check that your --url argument is correct. As I read it, it only contains
the target hostname, i.e. without 'virtroot'. The genxref line should read
something like:

./genxref --url=http://lxr.domain.com/virtroot --allversions

unless, of course, your 'virtroot' is only '/', meaning the host only
contains a single LXR tree (without any other page).

If you forgot nothing, send your lxr.conf.

Regards,
ajl

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3581487&group_id=27350

[Lxr-dev] [ lxr-Bugs-3581487 ] genxref crash while reading configuration

[SourceForge.net <no...@so...>]

Bugs item #3581487, was opened at 2012-10-28 19:21
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3581487&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: genxref
Group: v1.0
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Olivier Duclos (oliwer_)
Assigned to: Nobody/Anonymous (nobody)
Summary: genxref crash while reading configuration

Initial Comment:
When launching genxref with the command './genxref --url=http://lxr.domain.com --allversions' I get the following error:

No matching configuration at ./genxref line 312, <FILETYPE> line 1.

I tried to track down the error and found this a few lines above :

if ($option{'url'}) {   # Single 'url'
        @config = (1);          # Fake list to prevent looping
}

This may prevent looping but the rest of the script needed the data in @config. I tried commenting this passage, but then the script exits without any error and without having generating the database files!

This was on Debian Squeeze with Perl 5.10.1.

----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2012-10-29 02:51

Message:
Hi,

Your analysis is wrong: inside the loop, access to data in @config (aka.
through variable $treedescr) is protected by another "if ($option("url"))"
to discriminate between single and multiple URL indexing. It should work as
is.

Check that your --url argument is correct. As I read it, it only contains
the target hostname, i.e. without 'virtroot'. The genxref line should read
something like:

./genxref --url=http://lxr.domain.com/virtroot --allversions

unless, of course, your 'virtroot' is only '/', meaning the host only
contains a single LXR tree (without any other page).

If you forgot nothing, send your lxr.conf.

Regards,
ajl

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3581487&group_id=27350

[Lxr-dev] [ lxr-Bugs-3581487 ] genxref crash while reading configuration

[SourceForge.net <no...@so...>]

Bugs item #3581487, was opened at 2012-10-28 19:21
Message generated for change (Tracker Item Submitted) made by oliwer_
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3581487&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: genxref
Group: v1.0
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Oliwer (oliwer_)
Assigned to: Nobody/Anonymous (nobody)
Summary: genxref crash while reading configuration

Initial Comment:
When launching genxref with the command './genxref --url=http://lxr.domain.com --allversions' I get the following error:

No matching configuration at ./genxref line 312, <FILETYPE> line 1.

I tried to track down the error and found this a few lines above :

if ($option{'url'}) {   # Single 'url'
        @config = (1);          # Fake list to prevent looping
}

This may prevent looping but the rest of the script needed the data in @config. I tried commenting this passage, but then the script exits without any error and without having generating the database files!

This was on Debian Squeeze with Perl 5.10.1.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3581487&group_id=27350

[Lxr-dev] [ lxr-Bugs-3580435 ] Files not filtered out with GIT

[SourceForge.net <no...@so...>]

Bugs item #3580435, was opened at 2012-10-26 00:46
Message generated for change (Tracker Item Submitted) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3580435&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: SCM support
Group: v1.0
Status: Open
Resolution: None
Priority: 4
Private: No
Submitted By: Andre-Littoz (ajlittoz)
Assigned to: Andre-Littoz (ajlittoz)
Summary: Files not filtered out with GIT

Initial Comment:
Release 1.0.0, GIT module

Files beginning with dot or ending with tilde are listed instead of being skipped.

Caused by reference to wrong variable ($node instead of $entryname) at line 115 in GIT.pm (unchecked copy-and-paste!)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3580435&group_id=27350

[Lxr-dev] [ lxr-Feature Requests-489929 ] function index (speedbar)

[SourceForge.net <no...@so...>]

Feature Requests item #489929, was opened at 2001-12-06 09:53
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390120&aid=489929&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Browsing Interface
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: David Oleszkiewicz (poppinfresh)
Assigned to: Nobody/Anonymous (nobody)
Summary: function index (speedbar)

Initial Comment:
I use the speedbar in xemacs alot to help be jump 
straight down to a function in a source file.  Well i 
actually map it to the mouse...but it would be nice 
to have a speebar (html frame) on the the left or 
right hand side of the source file which has links to 
all of the functions declared in a source file.  
presumably this information is already parsed in a 
manner that you know it is a function def.  this 
could be ignored in header files or contain the index 
of typedefs and #defines or something.  


----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2012-09-21 01:47

Message:
This indeed would be nice, but it needs much improvement on ctags. From
recent tests (September 2012), ctags has a lot of trouble finding
identifiers in languages other then C family.

----------------------------------------------------------------------

Comment By: Paul D. Smith (psmith)
Date: 2007-03-30 07:37

Message:
Logged In: YES 
user_id=30808
Originator: NO

<Edna Most> No Frames! </Edna Most>

Frames are a horrible, horrible invention and should be banned from the web
forever.  And even longer.

Having a function list is not a bad idea, but a simple drop-down box will
work.  Or, if we really want something on the sidebar, please use
CSS/tables.

BTW, in the SQL query provided by mbox, why check both the fileid AND the
filename?  It seems like just the fileid would be sufficient (and a lot
faster!)

----------------------------------------------------------------------

Comment By: Malcolm Box (mbox)
Date: 2002-01-23 08:08

Message:
Logged In: YES 
user_id=215386

Wouldn't be hard.  The sql to find all the symbols defined
in a file is:

select s.symname, i.line, d.declaration from indexes i,
symbols s, declarations d, files f where s.symi
d=i.symid and i.type = d.declid and i.fileid=f.fileid and
f.filename = &quot;filename&quot;;

From there, it should be easy to spin some HTML and links
round the results and shove it in a sidebar.  Presumably
this would be linked off the source script, so that it would
first generate the source listing, and then add the sidebar
panel.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390120&aid=489929&group_id=27350

[Lxr-dev] [ lxr-Feature Requests-788079 ] ESQL/C

[SourceForge.net <no...@so...>]

Feature Requests item #788079, was opened at 2003-08-13 07:28
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390120&aid=788079&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Language support
Group: None
>Status: Closed
Priority: 5
Private: No
Submitted By: Nobody/Anonymous (nobody)
Assigned to: Nobody/Anonymous (nobody)
Summary: ESQL/C

Initial Comment:
We use Informix's ESQL/C for our project... 
adding ESQL/C support to LXR involved the following:

$Id: genxref,v 1.5 1998/04/22 12:16:12 pergj Exp $
add '|ec' after '|cc' on line #329

# $Id: Common.pm,v 1.6 1998/05/14 11:59:22 argggh Exp $
add '|ec' after '|cc' on line #166

That's it... works great

Z

----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2012-09-21 01:40

Message:
Proposed patch seems outdated (according to the 2012 state of the code).
Closing this request.

If needed, please resubmit giving more technical details.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390120&aid=788079&group_id=27350

[Lxr-dev] [ lxr-Bugs-3570203 ] showconfig: incorrect tree links

[SourceForge.net <no...@so...>]

Bugs item #3570203, was opened at 2012-09-20 11:31
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3570203&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: current cvs
>Status: Closed
>Resolution: Fixed
Priority: 3
Private: No
Submitted By: Andre-Littoz (ajlittoz)
Assigned to: Andre-Littoz (ajlittoz)
Summary: showconfig: incorrect tree links

Initial Comment:
showconfig offers "previous" and "next" [parameter group] links to list configuration for other trees. Unhappily, the link is built using HTML-relative form which implies 'virtroot' is not changed and new modes links still point to the original tree.

The "previous" and "next" links should be computed according to the same algorithm as that used for the "speed switch" button in the page header area. The basic code already exist in Template.pm. Since this code manipulates configuration parameters, a new method should be created in Config.pm to be shared by Template.pm and Config.pm.

----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2012-09-21 01:21

Message:
Method treeurl created in Config.pm to return an HTML-absolute URL for a
tree or undef if it is the current tree. Caller may use this information to
use HTML-relative URL or ignore action.

showconfig and Template.pm modified to reflect new method.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3570203&group_id=27350

[Lxr-dev] [ lxr-Bugs-3535240 ] Typo error when check swish-e

[SourceForge.net <no...@so...>]

Bugs item #3535240, was opened at 2012-06-14 09:30
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3535240&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: genxref
Group: current cvs
>Status: Closed
Resolution: Fixed
Priority: 5
Private: No
Submitted By: squall.ho (squallho)
Assigned to: Andre-Littoz (ajlittoz)
Summary: Typo error when check swish-e

Initial Comment:
at line 240, the wrong typo of 
$toolloc = `which swhish-e 2>/dev/null`;

an extra "h" will make the $toolloc always be empty, since there is no binary named sw"h"ish-e.
should be correct as
$toolloc = `which swish-e 2>/dev/null`;

----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2012-09-20 11:35

Message:
After reading thoroughly the standards and other papers, decided to use
"command -v" instead of "which", which would also remove a dependency on
package which (at least in Fedora).

Implemented in the new genxref for release 1.0

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2012-07-22 08:50

Message:
Typo corrected in CVS
Still considering whether to change "which" to "command" or not.

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2012-06-25 07:10

Message:
jbglaw, sorry! I misintrepreted 'command -v' for a "template" where command
would be replaced by swish-e, glimpse or anything else. It is OK on my
system.
To your knowledge, is 'which' deprecated? is 'command' guaranteed to be
accepted in ANY implementation of Perl?

Regards,
ajl

----------------------------------------------------------------------

Comment By: Jan-Benedict Glaw (jbglaw)
Date: 2012-06-25 06:00

Message:
`command -v ....' should basically do what `which' did:

jbglaw@pluto:~$ command -v vim
/usr/bin/vim

Does it behave differently on your system? So I'd just use

$toolloc = `command -v swish-e 2>/dev/null`;

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2012-06-24 02:58

Message:
Thanks for pointing out the typo.


To jbglaw:
Why do you recommend not to use 'which'? I want to get the path to the
executable, while 'command -v' will only print identification and version
information.

But portability-wise, I may be wrong. Instead of comparing path with value
of parameter 'xxx-bin', I should blindly use parameter 'xxx-bin' with -v
option and see if it returns the correct name for the tool. What is your
opinion?

----------------------------------------------------------------------

Comment By: Jan-Benedict Glaw (jbglaw)
Date: 2012-06-15 07:08

Message:
Please don't use `which' any longer. To my knowledge, `command -v ...' is a
portable replacement.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3535240&group_id=27350

[Lxr-dev] [ lxr-Bugs-3570203 ] showconfig: incorrect tree links

[SourceForge.net <no...@so...>]

Bugs item #3570203, was opened at 2012-09-20 11:31
Message generated for change (Tracker Item Submitted) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3570203&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: current cvs
Status: Open
Resolution: None
Priority: 3
Private: No
Submitted By: Andre-Littoz (ajlittoz)
Assigned to: Andre-Littoz (ajlittoz)
Summary: showconfig: incorrect tree links

Initial Comment:
showconfig offers "previous" and "next" [parameter group] links to list configuration for other trees. Unhappily, the link is built using HTML-relative form which implies 'virtroot' is not changed and new modes links still point to the original tree.

The "previous" and "next" links should be computed according to the same algorithm as that used for the "speed switch" button in the page header area. The basic code already exist in Template.pm. Since this code manipulates configuration parameters, a new method should be created in Config.pm to be shared by Template.pm and Config.pm.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3570203&group_id=27350

[Lxr-dev] [ lxr-Feature Requests-2755517 ] Support case-insensitive languages

[SourceForge.net <no...@so...>]

Feature Requests item #2755517, was opened at 2009-04-12 04:04
Message generated for change (Settings changed) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390120&aid=2755517&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Language support
>Group: Next Release
>Status: Pending
Priority: 5
Private: No
Submitted By: AdrianIssott (adrianissott)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: Support case-insensitive languages

Initial Comment:
This feature would make LXR better suited to at least the following languages:
* Pascal (fully case-insensitive)
* Visual Basic (fully case-insensitive)
* SQL (partially case-insensitive - reserved words are CI but table / columns names may not be)
* PHP (partially case-insensitive - fn / class names are CI but variable names aren't)

Two particular issues to keep in mind are:
* It's too simplistic to assume languages are either case-sensitive or not.
* Allowing searches to be case-insensitive (e.g. for identifiers) will increase the number of false positives returned (i.e. matches that the user doesn't care about) and make searching less useful.

I propose this feature is implemented in the following stages:
1) Feature Request 2755445 Language Specific Searches to mitigate the issue of increased false positives when searching by allow case-insensitive searches to be restricted to just those languages in which it makes sense. See https://sourceforge.net/tracker/?func=detail&aid=2755445&group_id=27350&atid=390120 for more details.
2) The format of generic.conf should be updated to allow case-insensitivity to be specified per language as follows:

    'caseinsensitivity' => {
        'reserved' => [01],
        'type_a'    => [01],
        'type_b'    => [01],
        ...
    },
  
    Where type_a, type_b etc are the types listed in the typemap for the language.
3) Add the case-insensitivity functionality for reserved words when locating them for mark up in source files being browsed (has no impact on searching).
4) Add the case-insensitivity functionality per type when locating symbols for mark up in source files being browsed.
5) Add the case-insensitivity functionality per type when adding links to identifier searches in source files being browsed.
6) Allow identifier searches to be specified as case-insensitive by the user. [Don't try to do anything fancy such as preset this according to what languages the search is restricted to].
7) Ensure general searches on file contents can be made case insensitive by the user.

----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2012-09-17 08:30

Message:
Preliminary implementation in release 1.0:

- new flag added in language description 'case_insensitive'
According to specifications above, this might be too coarse since it
applies globally to a language.
When declared symbols are added to the DB dictionary, they are converted to
upper-case if flag is true. Same for references.
In processcode(), if flag is true, the key to dictionary search is
converted to upper-case (the symbol is kept as is in the display, it will
be highlighted in any case).

- ident is not modified: meaning that if cases (in file and in dictionary)
do not correspond), the target symbol will not be found. The difficulty is:
'case_insensitive' is a language flag while dictionary lookup is global
(for all languages making up a tree)..

As suggested by adrianissott, a finer grained strategy should be used:
maybe only consider case-insensitivity for keywords since it is a bad
coding method to vary case on different uses of the same object and let
native case in the dictionary.

----------------------------------------------------------------------

Comment By: AdrianIssott (adrianissott)
Date: 2009-04-12 04:05

Message:
Will fix bug 1832713 Implement case-insensitive language processing.

See
https://sourceforge.net/tracker/index.php?func=detail&aid=1832713&group_id=27350&atid=390119

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390120&aid=2755517&group_id=27350

[Lxr-dev] [ lxr-Feature Requests-942911 ] Subversion Support

[SourceForge.net <no...@so...>]

Feature Requests item #942911, was opened at 2004-04-27 03:23
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390120&aid=942911&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: General
Group: Next Release
>Status: Closed
Priority: 5
Private: No
Submitted By: teewurstmann (teewurstmann)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: Subversion Support

Initial Comment:
It would be nice to integrate support for Subversion
into a future release of LXR. Since LXR already
supports CVS support for Subversion shouldn't be too
hard to implement.

----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2012-09-16 09:53

Message:
Will be in release 1.0.

Please report experience since the current maintainer is not familiar wit
Subversion.

An important issue is navigation through branches and tags: this first
solution is not really user-friendly: the menu items are not really
self-explanatory. Can anyone suggest a better approach?

----------------------------------------------------------------------

Comment By: Marc W. Mengel (mengel)
Date: 2010-10-01 08:24

Message:
I've added patch   390119

https://sourceforge.net/tracker/?func=detail&aid=3079487&group_id=27350&atid=390119


----------------------------------------------------------------------

Comment By: Marc W. Mengel (mengel)
Date: 2010-10-01 08:14

Message:
I have this working, where you give a 
  'sourceroot' => 'svn:http://servername/path/to/repository'
I'll try to attach a patch...

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2007-11-27 03:19

Message:
Logged In: NO 

is it comming or not?
would be very nice to hear about...

think this is a good feature

----------------------------------------------------------------------

Comment By: Matt England (mengland)
Date: 2006-12-24 05:48

Message:
Logged In: YES 
user_id=10503
Originator: NO

Is any Subversion-support activity underway?

If not, can anyone provide some pointers on how to make an appropriate
lib/LXR/Files/SVN.pm and anything else that's needed to make LXR parse a
Subversion repository?

-Matt

----------------------------------------------------------------------

Comment By: Vaish (vaish)
Date: 2006-04-24 08:19

Message:
Logged In: YES 
user_id=1508835

Is there any workaround for LXR to access subversion repository?

----------------------------------------------------------------------

Comment By: Roland Schwingel (rolandschwingel)
Date: 2004-10-20 01:58

Message:
Logged In: YES 
user_id=710704

Oh yes!!

This would be really cool to have... I am at present
searching for an application doing this...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390120&aid=942911&group_id=27350

[Lxr-dev] [ lxr-Feature Requests-1691378 ] Rearchitect the DB backends

[SourceForge.net <no...@so...>]

Feature Requests item #1691378, was opened at 2007-03-30 08:37
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390120&aid=1691378&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: General
Group: None
>Status: Closed
Priority: 5
Private: No
Submitted By: Paul D. Smith (psmith)
Assigned to: Andre-Littoz (ajlittoz)
Summary: Rearchitect the DB backends

Initial Comment:
The LXR index DB backends all use DBI, which is perfect, but they don't take enough advantage of it and thus end up duplicating a significant amount of code with all the problems that generally brings.

Perl's DBI is ALREADY a generic front-end for SQL databases of all different types, so having separate LXR::Index::* subclasses for each database is largely unnecessary: all databases can use the DBI interface almost exactly the same way.

There are minor differences, though, so it's not true that we can get rid of the LXR::Index::* database specific classes entirely.

I propose that we make LXR::Index a proper superclass for the DB-specific subclasses, and we push up as much functionality as possible into the superclass.  The subclasses should be pretty small: they will deal only with DB-specific issues (for example, auto-incremented fields which work differently in PostgreSQL and MySQL).

----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2012-09-10 07:05

Message:
The final release 1.0 seems to have reached a satisfying state.

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2012-08-03 10:54

Message:
This is partially done in release 1.0. The differences in SQL dialects
forces to maintain different LXR::Index::* though some of them are reduced
to init (to create the queries) and DESTROY.

----------------------------------------------------------------------

Comment By: AdrianIssott (adrianissott)
Date: 2009-04-13 09:39

Message:
I'd like to take this on as I'd also noticed a lot of redundancy in the
LXR::Index modules. Malcolm can you assign it to me?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390120&aid=1691378&group_id=27350

[Lxr-dev] [ lxr-Bugs-3561288 ] Spurious symbols in usage table with MySQL

[SourceForge.net <no...@so...>]

Bugs item #3561288, was opened at 2012-08-24 06:15
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3561288&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Database interface
Group: current cvs
>Status: Closed
>Resolution: Fixed
Priority: 7
Private: No
Submitted By: Andre-Littoz (ajlittoz)
Assigned to: Andre-Littoz (ajlittoz)
Summary: Spurious symbols in usage table with MySQL

Initial Comment:
When comparing indexation results between MySQL, PostgreSQL and SQLite, MySQL records more symbol usages in table lxr_usages. After diff'ing the tables, it appears the extra usages hit on similar looking symbols (with a different case). MySQL manual says: "By default, string comparisons are not case sensitive and use the current character set." This may be correct for natural languages but definitely wrong for computer science and case-sensitive languages.

The easiest way to solve that issue is to change the type of the columns in the table description to BINARY. Thus, we keep method common factoring in the Index class.

----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2012-09-10 06:54

Message:
CHAR data in MySQL tables changed to BINARY to force bit comparisons
instead of locale.
Did the same for PostgreSQL(failsafe though it works as is): CHAR ->
BYTEA.
Nothing done for SQLite (it works as is).
Could not do anything for Oracle since I could not test with it.

Fix effective as of release 1.0

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3561288&group_id=27350

[Lxr-dev] [ lxr-Bugs-3546289 ] Unable to connect to PostgreSQL database

[SourceForge.net <no...@so...>]

Bugs item #3546289, was opened at 2012-07-20 04:54
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3546289&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Database interface
Group: current cvs
>Status: Closed
>Resolution: Fixed
Priority: 3
Private: No
Submitted By: Andre-Littoz (ajlittoz)
Assigned to: Andre-Littoz (ajlittoz)
Summary: Unable to connect to PostgreSQL database

Initial Comment:
Under some obscure circumstances (related to PostgreSQL or Linux distro upgrade???), LXR can't connect to PostgreSQL through Unix domain socket. Work around is to connect through TCP socket. To play it safe, parameter 'dbname' in lxr.conf can be modified to include ';host=localhost' after the DB name to force TCP socket.

----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2012-09-10 06:49

Message:
Fixed in release 1.0 as suggested

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3546289&group_id=27350

[Lxr-dev] [ lxr-Bugs-3520299 ] Incorrect DB purge with --reindexall

[SourceForge.net <no...@so...>]

Bugs item #3520299, was opened at 2012-04-22 05:56
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3520299&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Database interface
Group: current cvs
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Andre-Littoz (ajlittoz)
Assigned to: Andre-Littoz (ajlittoz)
Summary: Incorrect DB purge with --reindexall

Initial Comment:
Applies to all versions of LXR, probably since 0.9.3 in 2004!

Statements delete_releases and delete_files have implicit interrelatioship: a "release" is some alias to a base "revision" file. A base "revision" can thus only be deleted when all "releases" pointing to it have been destroyed. In the present implementation, "releases" are first deleted. Then base "revisions" are scanned: at that time, there is not any more "releases" thus the delete statement for table 'files' can't succeed since it can't compared the files' fileid to releases' (which no longer exist).

This leaves the 'files' records in the DB. In the case of the kernel, that amounts to more than 37000 items per version.

When reindexing, it is very likely that the new "revision" for the file will be different (it is almost certain with the VCSes, only slightly probable with plain files). This means the orphaned "revisions" will not be reused and new records will be created. DB grows uselessly and performance is impacted.

Even without that, the whole process is too slow (same order of magnitude as the indexing itself!) since it involves a selective query with references to several tables.

The first issue ('files' table) could be resolved with a reference count.



----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2012-09-10 06:48

Message:
The whole DB cleaning process has been rewritten for release 1.0.

DB purge is now correct. Performance has not been tested, but should
benefit from the DB backend reorganization.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3520299&group_id=27350

[Lxr-dev] [ lxr-Bugs-3468352 ] Inconsistent identifier recognition

[SourceForge.net <no...@so...>]

Bugs item #3468352, was opened at 2012-01-02 01:50
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3468352&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
>Category: genxref
Group: current cvs
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Andre-Littoz (ajlittoz)
Assigned to: Andre-Littoz (ajlittoz)
Summary: Inconsistent identifier  recognition

Initial Comment:
While the different parsers try to use language parameter regex 'identdef' to find potential identifier, subs used by genxref try to find identifiers using their own regex. Results are different and some identifiers can be missed.

See if it is feasible to use 'identdef' everywhere.

----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2012-09-10 06:43

Message:
Fixed in release 1.0

referencefile now uses parameter 'identdef' like processcode.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3468352&group_id=27350

[Lxr-dev] [ lxr-Bugs-2721915 ] Install script

[SourceForge.net <no...@so...>]

Bugs item #2721915, was opened at 2009-03-30 03:43
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=2721915&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
>Resolution: Accepted
Priority: 6
Private: No
Submitted By: Malcolm Box (mbox)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: Install script

Initial Comment:
LXR needs an install script that automates the installation.

----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2012-09-10 06:37

Message:
Release 1.0 now contains an automatic installer/configurator.

Feedback welcome to improve it

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=2721915&group_id=27350

[Lxr-dev] [ lxr-Bugs-518365 ] Indexing of files once indexed is buggy!

[SourceForge.net <no...@so...>]

Bugs item #518365, was opened at 2002-02-16 02:04
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=518365&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: genxref
Group: current cvs
>Status: Closed
>Resolution: Fixed
Priority: 7
Private: No
Submitted By: Shree (shreekumar)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: Indexing of files once indexed is buggy!

Initial Comment:
I am using LXR-0.9.1

Consider this scenario :

There is a source tree "test" having only one file - test.c

test.c
-------
#define TEST 100

now, I run genxref & when I search for TEST in identifiers, I get that it is a macro defined in  
test.c at line 1

now I change test.c  to
-------
#define T 1
#define TEST 100

& run genxref

Now what I get is - TEST is defined as a macro in test.c in line 1 and line 2 !

The culprit is this piece of code in function processfile() [ Tagger.pm ]
------
          if ($index->toindex($fileid)) {
                $index->empty_cache();
                print(STDERR "--- $pathname $fileid\n");

                my $path = $files->tmpfile($pathname, $release);

                $lang->indexfile($pathname, $path, $fileid, $index, $config);
                unlink($path);
          } else {
                print(STDERR "$pathname was already indexed\n");
          }
------
The problem is that if the file already existed and has changed since then [based on the 
timestamp], the identifiers added to the database due to this file in the previous run of genxref are 
not removed from the database, hence the number of definitions will keep on growing...

The same problem is also present in processrefs().


----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2012-09-10 03:38

Message:
At least fixed in 1.0 release after 10 years! Always be patient.

The fix is based on the suggestion submitted by bjjohnson but requires
extra subtleties to avoid erasing a base revision still in use in another
version. Since the DB schemas now include reference counts, some tests are
easier and faster because they do not involve a extensive query through the
whole DB only to count referencing records.

One feature is not handled is the suggestion: the "member of" relation in a
definition. It is necessary to take it into account to update a symbol
reference count before removing a definition. This action disturbs the
"sequential" processing of the definition. Consequently a symbol cache is
used to minimize DB queries (important for large projects such as the Linux
kernel).

Symbols are left in the DB on the assumption that cleaning was caused by an
indexing request: the new file revision will likely differ little from the
previous one in the parsed symbols.

Since the fix is rather complex, read CVS for details.

Note: with Index.pm rewriting (common factoring what is DB-engine
independent), the fix is valid for all the currently supported DB.

----------------------------------------------------------------------

Comment By: Brian J. Johnson (bjjohnson)
Date: 2004-10-21 12:01

Message:
Logged In: YES 
user_id=85501

Here's a patch to provide incremental reindexing.  The
--reindexall
option was useful, but not really what's required (at least,
not what
I really wanted):  I want to avoid reindexing files which
haven't
changed, remove all index info. for files which have been
deleted, and
correctly reindex (i.e. without duplication) files which
which have
changed.  This patch does so.  ItWorksForMe, please let me
know if it
breaks for you.  (And if someone wants to add support for
databases
besides mysql, please do!)

I _think_ I got the database manipulation right; please let
me know if
I've made any bad assumptions about how the tables relate,
or about
the semantics of the various fields.  I've tested this with
Files::Plain and Index::Mysql.

The patch is against lxr 0.9.3.

Thanks,
Brian

----------------------------------------------------------------------
Patch to add incremental indexing to lxr-0.9.3
By Brian J. Johnson 10/21/04

This patch adds incremental indexing to lxr-0.9.3.  That is,
files
which have not changed are not reindexed, and files which
have changed
or been removed have their old information erased from the
database
before they are reindexed.  On my machine, this saves _hours_ of
reindexing time on trees which don't change much from day to
day.

The patch modifies genxref to add an extra pass before the
existing
gensearch, genindex, and genrefs passes.  For each file in
the current
release, it retrieves the file's revision from the database
and checks
if the file has changed in the file store:  i.e.
$files->filerev() is
different from the revision in the database.  If the file
has changed,
it calls $index->purgefile() to remove that fileid (i.e.
[filename,
revision] pair) from the release.  If that fileid is no
longer active
in any release, $index->purgefile() removes it from the
other tables
as well.  Then the genindex and genrefs passes add the new
revision of
the file (with a new fileid) to the database.

Symbols can still be left around (as with the "reindexall"
option), so
the administrator should drop and regenerate the database
completely
from scratch every so often.

I haven't tried to add support for any database besides
mysql, as I
have no way to test the changes.  It should be pretty easy
for others
to add it, though.  (I don't really even know SQL, and I
could do the
mysql port in a few hours, working solely from the
documentation at
mysql.com.)

Questions and comments:
- $self->{releases_select} was not being undef-ed in the DESTROY
  routine.  Added that.
- When is it necessary to call $self->{xxx}->finish()?  Some dbi
  queries seem to do so, and others don't.



Index: lxr-0.9.3/genxref
===================================================================
--- lxr-0.9.3.orig/genxref	2004-10-21 13:19:44.000000000 -0500
+++ lxr-0.9.3/genxref	2004-10-21 13:38:51.000000000 -0500
@@ -89,11 +89,29 @@
 
 foreach my $version (@versions) {
 	$index->purge($version) if $option{'reindexall'};
+	cleanindex($version);
 	gensearch($version);
 	genindex('/', $version);
 	genrefs('/', $version);
 }
 
+sub cleanindex {
+	my ($release) = @_;
+	my ($f, @files);
+
+	@files = $index->getfiles($release);
+	foreach $f (@files) {
+		# $f == [filename, fileid, revision]
+		# Skip this file if it is still at the same revision
+		next if $files->filerev($f->[0], $release) == $f->[2];
+
+		print(STDERR "%%% DELETED/MODIFIED: ", join(" ", @$f), "\n");
+
+		# Remove old revision from this release.
+		$index->purgefile($f->[1], $release);
+	}
+}
+
 sub genindex {
 	my ($pathname, $release) = @_;
 
Index: lxr-0.9.3/lib/LXR/Index/Mysql.pm
===================================================================
--- lxr-0.9.3.orig/lib/LXR/Index/Mysql.pm	2004-10-21
13:19:44.000000000 -0500
+++ lxr-0.9.3/lib/LXR/Index/Mysql.pm	2004-10-21
13:39:07.000000000 -0500
@@ -56,6 +56,11 @@
 	  $self->{dbh}
 	  ->prepare("insert into ${prefix}files (filename,
revision, fileid) values (?, ?, NULL)");
 
+	$self->{allfiles_select} =
+		$self->{dbh}->prepare("select f.filename, f.fileid,
f.revision "
+		  . "from ${prefix}files f, ${prefix}releases r "
+		  . "where  f.fileid = r.fileid and r.release = ?");
+
 	$self->{symbols_byname} =
 	  $self->{dbh}->prepare("select symid from
${prefix}symbols where  symname = ?");
 	$self->{symbols_byid} =
@@ -81,6 +86,10 @@
 	  $self->{dbh}->prepare("select * from ${prefix}releases
where fileid = ? and  release = ?");
 	$self->{releases_insert} =
 	  $self->{dbh}->prepare("insert into ${prefix}releases
(fileid, release) values (?, ?)");
+	$self->{releases_delete} =
+	  $self->{dbh}->prepare("delete from ${prefix}releases
where fileid = ? and  release = ?");
+	$self->{releases_select_file} =
+	  $self->{dbh}->prepare("select * from ${prefix}releases
where fileid = ?");
 
 	$self->{status_get} =
 	  $self->{dbh}->prepare("select status from
${prefix}status where fileid = ?");
@@ -134,6 +143,19 @@
 		  . "where f.fileid = r.fileid "
 		  . "and r.release = ?");
 
+	$self->{indexes_del_fileid} =
+	  $self->{dbh}->prepare("delete from ${prefix}indexes "
+		  . "where ${prefix}indexes.fileid = ?");
+	$self->{useage_del_fileid} =
+	  $self->{dbh}->prepare("delete from ${prefix}useage "
+		  . "where ${prefix}useage.fileid = ?");
+	$self->{status_del_fileid} =
+	  $self->{dbh}->prepare("delete from ${prefix}status "
+		  . "where ${prefix}status.fileid = ?");
+	$self->{files_del_fileid} =
+	  $self->{dbh}->prepare("delete from ${prefix}files "
+		  . "where ${prefix}files.fileid = ?");
+
 	return $self;
 }
 
@@ -326,6 +348,47 @@
 	return $id;
 }
 
+# List all files in this release
+sub getfiles {
+	my ($self, $release) = @_;
+	my ($rows, @ret);
+
+	$rows = $self->{allfiles_select}->execute("$release");
+
+	while ($rows-- > 0) {
+		push(@ret, [ $self->{allfiles_select}->fetchrow_array ] );
+	}
+
+	$self->{allfiles_select}->finish();
+
+	return @ret;
+}
+
+# Remove all references to $fileid from $release
+sub purgefile {
+	my ($self, $fileid, $release) = @_;
+
+    # Remove $fileid from $release
+	$self->{releases_delete}->execute($fileid, $release);
+	$self->{releases_delete}->finish();
+
+	# Find how many releases still reference $fileid
+	my $rows = $self->{releases_select_file}->execute($fileid
+ 0);
+	$self->{releases_select_file}->finish();
+
+	# If none, remove fileid from all other tables
+	unless ($rows > 0) {
+		# we don't delete symbols, because they might be used by
other
+		# versions so we can end up with unused symbols, but that
+		# doesn't cause any problems.  Drop and rebuild the database
+		# from time to time if it bothers you.
+		$self->{indexes_del_fileid}->execute($fileid);
+		$self->{useage_del_fileid}->execute($fileid);
+		$self->{status_del_fileid}->execute($fileid);
+		$self->{files_del_fileid}->execute($fileid);
+	}
+}
+	
 sub purge {
 	my ($self, $version) = @_;
 
@@ -342,11 +405,15 @@
 	my ($self) = @_;
 	$self->{files_select}    = undef;
 	$self->{files_insert}    = undef;
+	$self->{allfiles_select} = undef;
 	$self->{symbols_byname}  = undef;
 	$self->{symbols_byid}    = undef;
 	$self->{symbols_insert}  = undef;
 	$self->{indexes_insert}  = undef;
+	$self->{releases_select} = undef;
 	$self->{releases_insert} = undef;
+	$self->{releases_delete} = undef;
+	$self->{releases_select_file} = undef;
 	$self->{status_insert}   = undef;
 	$self->{status_update}   = undef;
 	$self->{usage_insert}    = undef;
@@ -358,6 +425,10 @@
 	$self->{delete_status}   = undef;
 	$self->{delete_releases} = undef;
 	$self->{delete_files}    = undef;
+	$self->{indexes_del_fileid} = undef;
+	$self->{useage_del_fileid}  = undef;
+	$self->{status_del_fileid}  = undef;
+	$self->{files_del_fileid}   = undef;
 
 	if ($self->{dbh}) {
 		$self->{dbh}->disconnect();



----------------------------------------------------------------------

Comment By: Dave Brondsema (brondsem)
Date: 2004-07-20 08:25

Message:
Logged In: YES 
user_id=341298

Yes.  For the version being indexed, it deletes all data
directly related to that version.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2004-07-20 07:42

Message:
Logged In: NO 

Sorry, I can't access CVS (firewall).
Does your switch do "intelligent" job like described in message 
dated "2002-02-18 23:21" below? 

----------------------------------------------------------------------

Comment By: Dave Brondsema (brondsem)
Date: 2004-07-20 05:21

Message:
Logged In: YES 
user_id=341298

I have recently added the --reindexall option to genxref (in
CVS).  Please try and see if this works.  Perhaps it should
be default and not an option if it does.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2004-07-20 04:36

Message:
Logged In: NO 

Shree, you've said you had a patch, could you attach to the 
Tracker?
Thanks,
Dennis

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2004-07-20 04:31

Message:
Logged In: NO 

Still no fix for this?

----------------------------------------------------------------------

Comment By: Heikki Toivonen (hjtoi)
Date: 2004-03-12 09:48

Message:
Logged In: YES 
user_id=972898

Anyone have a patch for this?

----------------------------------------------------------------------

Comment By: Richard Kisley (kisley)
Date: 2003-04-07 17:39

Message:
Logged In: YES 
user_id=102080

(1) How about an intermediate solution, where someone writes
a VERIFY script which compares the paths in the database
with the version they refer to and deletes entries for
invalid paths?  Same options as genxref?

I indexed a subtree of a sourcetree, then realized I needed
to index the whole source tree.  So I moved the revision
main dir (since it was really a subdir) up a level and added
the other directories at their proper top level as other
subdirs, then re-indexed.  Now the original links in tree
one are all dead links, with live dupes.  No files changed. 

(2) So we all don't have to scurry for our SQL books, buried
in a box in the back of a closet at home (not work) how
about posting the exact drop syntax?  That might also be a
good thing to add to doc short-term, since genxref doesn't
work (prune) as expected. 

----------------------------------------------------------------------

Comment By: Gregor Hartmann (grex)
Date: 2002-06-07 06:10

Message:
Logged In: YES 
user_id=559509

Another similar problem would be files ore whole directories that are
deleted from the source tree. They would stay 
in the database forever as well. 

Maybe it could be fixed by iterating through all files in the database and
removing those (from the database) which 
have changed or were removed in the source tree.
then proceed indexing as before.


----------------------------------------------------------------------

Comment By: Shree (shreekumar)
Date: 2002-02-18 23:21

Message:
Logged In: YES 
user_id=142912

Here's my fix for this bug:

Add a field "timestamp" to the "status" table. And
remove 
the "status" field.

Before finding identifiers in a file, check whether it's 
modification time is greater that it was previously. If 
yes, then remove all the identifier definitions due to this 
file [and release] from the database. Store the new 
timestamp in the database.

Before finding references in a file, remove all identifier 
references due to this file [and release] from the database.
[ No need to check the timestamp in this case since 
the "definitions" are always found before the references]. 

In a large CVS tree, it is quite possible that a file may 
change between the time it is "indexed" and
"referenced". 
An easy way out of this seems to be to "index" a file and 
immediately "reference" it.

Related to this there is a problem in "Plain.pm" - the 
current  "filerev" function returns a value based on the 
timestamp.  Problem arises if a file changes between runs 
of genxref. What happens is that different values are 
returned by "filerev" even though it is the same 
(file,revision) pair is being indexed [or referenced].

I have changed filerev() for this purpose as

sub filerev {
 my ($self, $filename, $release) = @_;

 # TODO: length of filename+revision
 #       might turn out to be > 255 chars 
 #       [length used in the db]
        return join("-", $filename,
                                $release);
}

With this modification filerev() will return the same value 
for (file,revision) pair everytime - thus solving the 
problem.

I have a patch ready for this.


----------------------------------------------------------------------

Comment By: Malcolm Box (mbox)
Date: 2002-02-18 06:20

Message:
Logged In: YES 
user_id=215386

Yes, you're right, this is a bug.  The underlying assumption
that is being broken is that the files in a version are
static - which is true if one is indexing released software,
but not if it is a development tree.

The simplest work-around is to drop and recreate the
database each time, thus avoiding the problem.  For small to
medium repositories with the index updated nightly this
should work fine, but it doesn't work for large repositories.

The full solution would appear to be to check for an
existing entry for the (filename, release) pair and if it is
found delete it and all associated information.

----------------------------------------------------------------------

Comment By: Shree (shreekumar)
Date: 2002-02-16 05:32

Message:
Logged In: YES 
user_id=142912

There are two cases where the scenario that I've referred to applies:

1. Files are not in CVS [ ie usage of "Files.pm" ]. You run
genxref, then change a file & genxref again

2. Files are in CVS, and you want to index the "head" tag. Files
change regularly, and you want to keep the 
 
cross reference in sync - probably by running genxref once an hour or so
[as a cron job].

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-02-16 04:47

Message:
Logged In: NO 

I was in the impression that a file may never ever change again, except if
(and only if) the file was changed and has either got a new CVS revision
(or tag) or if there is a new directory for a new version of the whole
project (if it is not managed by CVS).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=518365&group_id=27350