lxr-developer — Don't use, active developers only

[Lxr-dev] combining freetext and file searches

[Dave B. <da...@br...>]

When doing freetext searches I often get a lot of results and thought it'd
be useful to filter the results like the file search does.  But I didn't
want to duplicate code between the file and freetext searches.  So, unless
anyone has objections or better ideas, I am going to combine the freetext
and file searches.  There will be two search boxes; if only one is used
it'll function just like the respective page does now.  If both are used,
the results will be further restricted.

Any thoughts?

-- 
Dave Brondsema : da...@br...
http://www.brondsema.net : personal
http://www.splike.com : programming
http://csx.calvin.edu : student org

Re: [Lxr-dev] multi-language identifiers

[Dave B. <da...@br...>]

On Thu, 22 Jul 2004, Malcolm Box wrote:

> Dave Brondsema wrote:
>
> | On Wed, 21 Jul 2004, Malcolm Box wrote:
> |>I think that's a misleading comment :-)  The code does a !grep(/^$2$/,
> |>$self->langinfo('reserved')) which should filter out any reserved words
> |>as specified in the language info block.
> |>
> |>So there is a bug in the comment :-)
> |
> | Yes, reserved words are correctly limited to the current language.  But
> | identifiers are not.
>
> Good point - the comment and the problem you originally reported are
> talking about different things:
>
> 1. ignoring any string which is a reserved word when checking if it is a
> symbol
> and
> 2. making the lookup of identifiers language dependent so that if e.g. a
> perl file defines foo and a C file defines foo then the correct one will
> be linked to.
>
> AFAICT the code does the former correctly, but as commented doesn't do
> the latter. This would require tagging each symbol with the language it
> was defined in.  Also suspect that it might break clever code like the
> Javascript/C++ bridging in Mozilla - or at least make the LXR less useful.
>

Right.  It'll be configurable then :-)  The database already holds all the
relationships, the code just has to take advantage of that.

If configured to do so, it should be limited by version too.

-- 
Dave Brondsema : da...@br...
http://www.brondsema.net : personal
http://www.splike.com : programming
http://csx.calvin.edu : student org

Re: [Lxr-dev] multi-language identifiers

[Malcolm B. <ma...@br...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dave Brondsema wrote:

| On Wed, 21 Jul 2004, Malcolm Box wrote:
|>I think that's a misleading comment :-)  The code does a !grep(/^$2$/,
|>$self->langinfo('reserved')) which should filter out any reserved words
|>as specified in the language info block.
|>
|>So there is a bug in the comment :-)
|
| Yes, reserved words are correctly limited to the current language.  But
| identifiers are not.

Good point - the comment and the problem you originally reported are
talking about different things:

1. ignoring any string which is a reserved word when checking if it is a
symbol
and
2. making the lookup of identifiers language dependent so that if e.g. a
perl file defines foo and a C file defines foo then the correct one will
be linked to.

AFAICT the code does the former correctly, but as commented doesn't do
the latter. This would require tagging each symbol with the language it
was defined in.  Also suspect that it might break clever code like the
Javascript/C++ bridging in Mozilla - or at least make the LXR less useful.

Cheers,

Malcolm
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (MingW32)
Comment: Using GnuPG with MultiZilla - http://enigmail.mozdev.org

iD8DBQFA//KOQeMefPKyX/QRAsfnAJ9CeWy/Ulu5Zz4a1ZvM/i2EFwQjEQCfWsoB
+cQAw/vd32lewKKIrSoSG5w=
=2dR2
-----END PGP SIGNATURE-----

Re: [Lxr-dev] multi-language identifiers

[Dave B. <da...@br...>]

On Wed, 21 Jul 2004, Malcolm Box wrote:

> Dave Brondsema wrote:
> | On Tue, 20 Jul 2004, Malcolm Box wrote:
> |>If this isn't working for you, my guess would be that either the setup
> |>of $lang is hosed, or that processcode is not accurately reading the
> |>languageconf info.
> |>
> |>Cheers,
> |
> |
> | It's still marked as a TODO on processcode :-)
>
> I think that's a misleading comment :-)  The code does a !grep(/^$2$/,
> $self->langinfo('reserved')) which should filter out any reserved words
> as specified in the language info block.
>
> So there is a bug in the comment :-)
>

Yes, reserved words are correctly limited to the current language.  But
identifiers are not.

-- 
Dave Brondsema : da...@br...
http://www.brondsema.net : personal
http://www.splike.com : programming
http://csx.calvin.edu : student org

Re: [Lxr-dev] multi-language identifiers

[Malcolm B. <ma...@br...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dave Brondsema wrote:
| On Tue, 20 Jul 2004, Malcolm Box wrote:
|>If this isn't working for you, my guess would be that either the setup
|>of $lang is hosed, or that processcode is not accurately reading the
|>languageconf info.
|>
|>Cheers,
|
|
| It's still marked as a TODO on processcode :-)

I think that's a misleading comment :-)  The code does a !grep(/^$2$/,
$self->langinfo('reserved')) which should filter out any reserved words
as specified in the language info block.

So there is a bug in the comment :-)

Cheers,

Malcolm
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (MingW32)
Comment: Using GnuPG with MultiZilla - http://enigmail.mozdev.org

iD8DBQFA/ngOQeMefPKyX/QRAvYcAJ4oqIcH3vdcScb7QQ58HQP6Rkr0BgCgxQMj
D1qnBIMiVMjsCwLAyKLbPXA=
=UXio
-----END PGP SIGNATURE-----

Re: [Lxr-dev] multi-language identifiers

[Dave B. <da...@br...>]

On Tue, 20 Jul 2004, Malcolm Box wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Dave Brondsema wrote:
> | The 0.8 release announcement says:
> |
> | "In multi-language environments, keywords in a language are no longer
> | hyperlinked if a different language happens to have them as an identifier"
> |
> | This doesn't seem to be the case for me.  Does anybody remember where or
> | how this is supposed to happen?
>
> I recall coding this - the keywords are now per-language, and the code
> in Common.pm switches on the language of the file.  Once the file is
> fragmented by SimpleParse, each fragment recognised as code is fed to
> $lang->processcode where $lang should depend on the type of the file.
>
> This then strips out reserved words before making links to symbols - see
> Generic.pm line 162.
>
> If this isn't working for you, my guess would be that either the setup
> of $lang is hosed, or that processcode is not accurately reading the
> languageconf info.
>
> Cheers,

It's still marked as a TODO on processcode :-)

-- 
Dave Brondsema : da...@br...
http://www.brondsema.net : personal
http://www.splike.com : programming
http://csx.calvin.edu : student org

[Lxr-dev] [ lxr-Bugs-518365 ] Indexing of files once indexed is buggy!

[SourceForge.net <no...@so...>]

Bugs item #518365, was opened at 2002-02-16 05:04
Message generated for change (Comment added) made by brondsem
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=518365&group_id=27350

Category: genxref
Group: current cvs
Status: Open
Resolution: None
Priority: 7
Submitted By: Shree Kumar (shreekumar)
Assigned to: Malcolm Box (mbox)
Summary: Indexing of files once indexed is buggy!

Initial Comment:
I am using LXR-0.9.1

Consider this scenario :

There is a source tree "test" having only one file - test.c

test.c
-------
#define TEST 100

now, I run genxref & when I search for TEST in identifiers, I get that it is a macro defined in  
test.c at line 1

now I change test.c  to
-------
#define T 1
#define TEST 100

& run genxref

Now what I get is - TEST is defined as a macro in test.c in line 1 and line 2 !

The culprit is this piece of code in function processfile() [ Tagger.pm ]
------
          if ($index->toindex($fileid)) {
                $index->empty_cache();
                print(STDERR "--- $pathname $fileid\n");

                my $path = $files->tmpfile($pathname, $release);

                $lang->indexfile($pathname, $path, $fileid, $index, $config);
                unlink($path);
          } else {
                print(STDERR "$pathname was already indexed\n");
          }
------
The problem is that if the file already existed and has changed since then [based on the 
timestamp], the identifiers added to the database due to this file in the previous run of genxref are 
not removed from the database, hence the number of definitions will keep on growing...

The same problem is also present in processrefs().


----------------------------------------------------------------------

>Comment By: Dave Brondsema (brondsem)
Date: 2004-07-20 11:25

Message:
Logged In: YES 
user_id=341298

Yes.  For the version being indexed, it deletes all data
directly related to that version.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2004-07-20 10:42

Message:
Logged In: NO 

Sorry, I can't access CVS (firewall).
Does your switch do "intelligent" job like described in message 
dated "2002-02-18 23:21" below? 

----------------------------------------------------------------------

Comment By: Dave Brondsema (brondsem)
Date: 2004-07-20 08:21

Message:
Logged In: YES 
user_id=341298

I have recently added the --reindexall option to genxref (in
CVS).  Please try and see if this works.  Perhaps it should
be default and not an option if it does.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2004-07-20 07:36

Message:
Logged In: NO 

Shree, you've said you had a patch, could you attach to the 
Tracker?
Thanks,
Dennis

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2004-07-20 07:31

Message:
Logged In: NO 

Still no fix for this?

----------------------------------------------------------------------

Comment By: Heikki Toivonen (hjtoi)
Date: 2004-03-12 12:48

Message:
Logged In: YES 
user_id=972898

Anyone have a patch for this?

----------------------------------------------------------------------

Comment By: Richard Kisley (kisley)
Date: 2003-04-07 20:39

Message:
Logged In: YES 
user_id=102080

(1) How about an intermediate solution, where someone writes
a VERIFY script which compares the paths in the database
with the version they refer to and deletes entries for
invalid paths?  Same options as genxref?

I indexed a subtree of a sourcetree, then realized I needed
to index the whole source tree.  So I moved the revision
main dir (since it was really a subdir) up a level and added
the other directories at their proper top level as other
subdirs, then re-indexed.  Now the original links in tree
one are all dead links, with live dupes.  No files changed. 

(2) So we all don't have to scurry for our SQL books, buried
in a box in the back of a closet at home (not work) how
about posting the exact drop syntax?  That might also be a
good thing to add to doc short-term, since genxref doesn't
work (prune) as expected. 

----------------------------------------------------------------------

Comment By: Gregor Hartmann (grex)
Date: 2002-06-07 09:10

Message:
Logged In: YES 
user_id=559509

Another similar problem would be files ore whole directories that are deleted from the source tree. They would stay 
in the database forever as well. 

Maybe it could be fixed by iterating through all files in the database and removing those (from the database) which 
have changed or were removed in the source tree.
then proceed indexing as before.


----------------------------------------------------------------------

Comment By: Shree Kumar (shreekumar)
Date: 2002-02-19 02:21

Message:
Logged In: YES 
user_id=142912

Here's my fix for this bug:

Add a field "timestamp" to the "status" table. And remove 
the "status" field.

Before finding identifiers in a file, check whether it's 
modification time is greater that it was previously. If 
yes, then remove all the identifier definitions due to this 
file [and release] from the database. Store the new 
timestamp in the database.

Before finding references in a file, remove all identifier 
references due to this file [and release] from the database.
[ No need to check the timestamp in this case since 
the "definitions" are always found before the references]. 

In a large CVS tree, it is quite possible that a file may 
change between the time it is "indexed" and "referenced". 
An easy way out of this seems to be to "index" a file and 
immediately "reference" it.

Related to this there is a problem in "Plain.pm" - the 
current  "filerev" function returns a value based on the 
timestamp.  Problem arises if a file changes between runs 
of genxref. What happens is that different values are 
returned by "filerev" even though it is the same 
(file,revision) pair is being indexed [or referenced].

I have changed filerev() for this purpose as

sub filerev {
 my ($self, $filename, $release) = @_;

 # TODO: length of filename+revision
 #       might turn out to be > 255 chars 
 #       [length used in the db]
        return join("-", $filename,
                                $release);
}

With this modification filerev() will return the same value 
for (file,revision) pair everytime - thus solving the 
problem.

I have a patch ready for this.


----------------------------------------------------------------------

Comment By: Malcolm Box (mbox)
Date: 2002-02-18 09:20

Message:
Logged In: YES 
user_id=215386

Yes, you're right, this is a bug.  The underlying assumption
that is being broken is that the files in a version are
static - which is true if one is indexing released software,
but not if it is a development tree.

The simplest work-around is to drop and recreate the
database each time, thus avoiding the problem.  For small to
medium repositories with the index updated nightly this
should work fine, but it doesn't work for large repositories.

The full solution would appear to be to check for an
existing entry for the (filename, release) pair and if it is
found delete it and all associated information.

----------------------------------------------------------------------

Comment By: Shree Kumar (shreekumar)
Date: 2002-02-16 08:32

Message:
Logged In: YES 
user_id=142912

There are two cases where the scenario that I've referred to applies:

1. Files are not in CVS [ ie usage of "Files.pm" ]. You run genxref, then change a file & genxref again

2. Files are in CVS, and you want to index the "head" tag. Files change regularly, and you want to keep the 
 
cross reference in sync - probably by running genxref once an hour or so [as a cron job].

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-02-16 07:47

Message:
Logged In: NO 

I was in the impression that a file may never ever change again, except if (and only if) the file was changed and has either got a new CVS revision (or tag) or if there is a new directory for a new version of the whole project (if it is not managed by CVS).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=518365&group_id=27350

[Lxr-dev] [ lxr-Bugs-518365 ] Indexing of files once indexed is buggy!

[SourceForge.net <no...@so...>]

Bugs item #518365, was opened at 2002-02-16 02:04
Message generated for change (Comment added) made by nobody
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=518365&group_id=27350

Category: genxref
Group: current cvs
Status: Open
Resolution: None
Priority: 7
Submitted By: Shree Kumar (shreekumar)
Assigned to: Malcolm Box (mbox)
Summary: Indexing of files once indexed is buggy!

Initial Comment:
I am using LXR-0.9.1

Consider this scenario :

There is a source tree "test" having only one file - test.c

test.c
-------
#define TEST 100

now, I run genxref & when I search for TEST in identifiers, I get that it is a macro defined in  
test.c at line 1

now I change test.c  to
-------
#define T 1
#define TEST 100

& run genxref

Now what I get is - TEST is defined as a macro in test.c in line 1 and line 2 !

The culprit is this piece of code in function processfile() [ Tagger.pm ]
------
          if ($index->toindex($fileid)) {
                $index->empty_cache();
                print(STDERR "--- $pathname $fileid\n");

                my $path = $files->tmpfile($pathname, $release);

                $lang->indexfile($pathname, $path, $fileid, $index, $config);
                unlink($path);
          } else {
                print(STDERR "$pathname was already indexed\n");
          }
------
The problem is that if the file already existed and has changed since then [based on the 
timestamp], the identifiers added to the database due to this file in the previous run of genxref are 
not removed from the database, hence the number of definitions will keep on growing...

The same problem is also present in processrefs().


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2004-07-20 07:42

Message:
Logged In: NO 

Sorry, I can't access CVS (firewall).
Does your switch do "intelligent" job like described in message 
dated "2002-02-18 23:21" below? 

----------------------------------------------------------------------

Comment By: Dave Brondsema (brondsem)
Date: 2004-07-20 05:21

Message:
Logged In: YES 
user_id=341298

I have recently added the --reindexall option to genxref (in
CVS).  Please try and see if this works.  Perhaps it should
be default and not an option if it does.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2004-07-20 04:36

Message:
Logged In: NO 

Shree, you've said you had a patch, could you attach to the 
Tracker?
Thanks,
Dennis

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2004-07-20 04:31

Message:
Logged In: NO 

Still no fix for this?

----------------------------------------------------------------------

Comment By: Heikki Toivonen (hjtoi)
Date: 2004-03-12 09:48

Message:
Logged In: YES 
user_id=972898

Anyone have a patch for this?

----------------------------------------------------------------------

Comment By: Richard Kisley (kisley)
Date: 2003-04-07 17:39

Message:
Logged In: YES 
user_id=102080

(1) How about an intermediate solution, where someone writes
a VERIFY script which compares the paths in the database
with the version they refer to and deletes entries for
invalid paths?  Same options as genxref?

I indexed a subtree of a sourcetree, then realized I needed
to index the whole source tree.  So I moved the revision
main dir (since it was really a subdir) up a level and added
the other directories at their proper top level as other
subdirs, then re-indexed.  Now the original links in tree
one are all dead links, with live dupes.  No files changed. 

(2) So we all don't have to scurry for our SQL books, buried
in a box in the back of a closet at home (not work) how
about posting the exact drop syntax?  That might also be a
good thing to add to doc short-term, since genxref doesn't
work (prune) as expected. 

----------------------------------------------------------------------

Comment By: Gregor Hartmann (grex)
Date: 2002-06-07 06:10

Message:
Logged In: YES 
user_id=559509

Another similar problem would be files ore whole directories that are deleted from the source tree. They would stay 
in the database forever as well. 

Maybe it could be fixed by iterating through all files in the database and removing those (from the database) which 
have changed or were removed in the source tree.
then proceed indexing as before.


----------------------------------------------------------------------

Comment By: Shree Kumar (shreekumar)
Date: 2002-02-18 23:21

Message:
Logged In: YES 
user_id=142912

Here's my fix for this bug:

Add a field "timestamp" to the "status" table. And remove 
the "status" field.

Before finding identifiers in a file, check whether it's 
modification time is greater that it was previously. If 
yes, then remove all the identifier definitions due to this 
file [and release] from the database. Store the new 
timestamp in the database.

Before finding references in a file, remove all identifier 
references due to this file [and release] from the database.
[ No need to check the timestamp in this case since 
the "definitions" are always found before the references]. 

In a large CVS tree, it is quite possible that a file may 
change between the time it is "indexed" and "referenced". 
An easy way out of this seems to be to "index" a file and 
immediately "reference" it.

Related to this there is a problem in "Plain.pm" - the 
current  "filerev" function returns a value based on the 
timestamp.  Problem arises if a file changes between runs 
of genxref. What happens is that different values are 
returned by "filerev" even though it is the same 
(file,revision) pair is being indexed [or referenced].

I have changed filerev() for this purpose as

sub filerev {
 my ($self, $filename, $release) = @_;

 # TODO: length of filename+revision
 #       might turn out to be > 255 chars 
 #       [length used in the db]
        return join("-", $filename,
                                $release);
}

With this modification filerev() will return the same value 
for (file,revision) pair everytime - thus solving the 
problem.

I have a patch ready for this.


----------------------------------------------------------------------

Comment By: Malcolm Box (mbox)
Date: 2002-02-18 06:20

Message:
Logged In: YES 
user_id=215386

Yes, you're right, this is a bug.  The underlying assumption
that is being broken is that the files in a version are
static - which is true if one is indexing released software,
but not if it is a development tree.

The simplest work-around is to drop and recreate the
database each time, thus avoiding the problem.  For small to
medium repositories with the index updated nightly this
should work fine, but it doesn't work for large repositories.

The full solution would appear to be to check for an
existing entry for the (filename, release) pair and if it is
found delete it and all associated information.

----------------------------------------------------------------------

Comment By: Shree Kumar (shreekumar)
Date: 2002-02-16 05:32

Message:
Logged In: YES 
user_id=142912

There are two cases where the scenario that I've referred to applies:

1. Files are not in CVS [ ie usage of "Files.pm" ]. You run genxref, then change a file & genxref again

2. Files are in CVS, and you want to index the "head" tag. Files change regularly, and you want to keep the 
 
cross reference in sync - probably by running genxref once an hour or so [as a cron job].

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-02-16 04:47

Message:
Logged In: NO 

I was in the impression that a file may never ever change again, except if (and only if) the file was changed and has either got a new CVS revision (or tag) or if there is a new directory for a new version of the whole project (if it is not managed by CVS).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=518365&group_id=27350

[Lxr-dev] [ lxr-Bugs-518365 ] Indexing of files once indexed is buggy!

[SourceForge.net <no...@so...>]

Bugs item #518365, was opened at 2002-02-16 05:04
Message generated for change (Comment added) made by brondsem
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=518365&group_id=27350

Category: genxref
Group: current cvs
Status: Open
Resolution: None
Priority: 7
Submitted By: Shree Kumar (shreekumar)
Assigned to: Malcolm Box (mbox)
Summary: Indexing of files once indexed is buggy!

Initial Comment:
I am using LXR-0.9.1

Consider this scenario :

There is a source tree "test" having only one file - test.c

test.c
-------
#define TEST 100

now, I run genxref & when I search for TEST in identifiers, I get that it is a macro defined in  
test.c at line 1

now I change test.c  to
-------
#define T 1
#define TEST 100

& run genxref

Now what I get is - TEST is defined as a macro in test.c in line 1 and line 2 !

The culprit is this piece of code in function processfile() [ Tagger.pm ]
------
          if ($index->toindex($fileid)) {
                $index->empty_cache();
                print(STDERR "--- $pathname $fileid\n");

                my $path = $files->tmpfile($pathname, $release);

                $lang->indexfile($pathname, $path, $fileid, $index, $config);
                unlink($path);
          } else {
                print(STDERR "$pathname was already indexed\n");
          }
------
The problem is that if the file already existed and has changed since then [based on the 
timestamp], the identifiers added to the database due to this file in the previous run of genxref are 
not removed from the database, hence the number of definitions will keep on growing...

The same problem is also present in processrefs().


----------------------------------------------------------------------

>Comment By: Dave Brondsema (brondsem)
Date: 2004-07-20 08:21

Message:
Logged In: YES 
user_id=341298

I have recently added the --reindexall option to genxref (in
CVS).  Please try and see if this works.  Perhaps it should
be default and not an option if it does.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2004-07-20 07:36

Message:
Logged In: NO 

Shree, you've said you had a patch, could you attach to the 
Tracker?
Thanks,
Dennis

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2004-07-20 07:31

Message:
Logged In: NO 

Still no fix for this?

----------------------------------------------------------------------

Comment By: Heikki Toivonen (hjtoi)
Date: 2004-03-12 12:48

Message:
Logged In: YES 
user_id=972898

Anyone have a patch for this?

----------------------------------------------------------------------

Comment By: Richard Kisley (kisley)
Date: 2003-04-07 20:39

Message:
Logged In: YES 
user_id=102080

(1) How about an intermediate solution, where someone writes
a VERIFY script which compares the paths in the database
with the version they refer to and deletes entries for
invalid paths?  Same options as genxref?

I indexed a subtree of a sourcetree, then realized I needed
to index the whole source tree.  So I moved the revision
main dir (since it was really a subdir) up a level and added
the other directories at their proper top level as other
subdirs, then re-indexed.  Now the original links in tree
one are all dead links, with live dupes.  No files changed. 

(2) So we all don't have to scurry for our SQL books, buried
in a box in the back of a closet at home (not work) how
about posting the exact drop syntax?  That might also be a
good thing to add to doc short-term, since genxref doesn't
work (prune) as expected. 

----------------------------------------------------------------------

Comment By: Gregor Hartmann (grex)
Date: 2002-06-07 09:10

Message:
Logged In: YES 
user_id=559509

Another similar problem would be files ore whole directories that are deleted from the source tree. They would stay 
in the database forever as well. 

Maybe it could be fixed by iterating through all files in the database and removing those (from the database) which 
have changed or were removed in the source tree.
then proceed indexing as before.


----------------------------------------------------------------------

Comment By: Shree Kumar (shreekumar)
Date: 2002-02-19 02:21

Message:
Logged In: YES 
user_id=142912

Here's my fix for this bug:

Add a field "timestamp" to the "status" table. And remove 
the "status" field.

Before finding identifiers in a file, check whether it's 
modification time is greater that it was previously. If 
yes, then remove all the identifier definitions due to this 
file [and release] from the database. Store the new 
timestamp in the database.

Before finding references in a file, remove all identifier 
references due to this file [and release] from the database.
[ No need to check the timestamp in this case since 
the "definitions" are always found before the references]. 

In a large CVS tree, it is quite possible that a file may 
change between the time it is "indexed" and "referenced". 
An easy way out of this seems to be to "index" a file and 
immediately "reference" it.

Related to this there is a problem in "Plain.pm" - the 
current  "filerev" function returns a value based on the 
timestamp.  Problem arises if a file changes between runs 
of genxref. What happens is that different values are 
returned by "filerev" even though it is the same 
(file,revision) pair is being indexed [or referenced].

I have changed filerev() for this purpose as

sub filerev {
 my ($self, $filename, $release) = @_;

 # TODO: length of filename+revision
 #       might turn out to be > 255 chars 
 #       [length used in the db]
        return join("-", $filename,
                                $release);
}

With this modification filerev() will return the same value 
for (file,revision) pair everytime - thus solving the 
problem.

I have a patch ready for this.


----------------------------------------------------------------------

Comment By: Malcolm Box (mbox)
Date: 2002-02-18 09:20

Message:
Logged In: YES 
user_id=215386

Yes, you're right, this is a bug.  The underlying assumption
that is being broken is that the files in a version are
static - which is true if one is indexing released software,
but not if it is a development tree.

The simplest work-around is to drop and recreate the
database each time, thus avoiding the problem.  For small to
medium repositories with the index updated nightly this
should work fine, but it doesn't work for large repositories.

The full solution would appear to be to check for an
existing entry for the (filename, release) pair and if it is
found delete it and all associated information.

----------------------------------------------------------------------

Comment By: Shree Kumar (shreekumar)
Date: 2002-02-16 08:32

Message:
Logged In: YES 
user_id=142912

There are two cases where the scenario that I've referred to applies:

1. Files are not in CVS [ ie usage of "Files.pm" ]. You run genxref, then change a file & genxref again

2. Files are in CVS, and you want to index the "head" tag. Files change regularly, and you want to keep the 
 
cross reference in sync - probably by running genxref once an hour or so [as a cron job].

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-02-16 07:47

Message:
Logged In: NO 

I was in the impression that a file may never ever change again, except if (and only if) the file was changed and has either got a new CVS revision (or tag) or if there is a new directory for a new version of the whole project (if it is not managed by CVS).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=518365&group_id=27350

[Lxr-dev] [ lxr-Bugs-518365 ] Indexing of files once indexed is buggy!

[SourceForge.net <no...@so...>]

Bugs item #518365, was opened at 2002-02-16 02:04
Message generated for change (Comment added) made by nobody
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=518365&group_id=27350

Category: genxref
Group: current cvs
Status: Open
Resolution: None
Priority: 7
Submitted By: Shree Kumar (shreekumar)
Assigned to: Malcolm Box (mbox)
Summary: Indexing of files once indexed is buggy!

Initial Comment:
I am using LXR-0.9.1

Consider this scenario :

There is a source tree "test" having only one file - test.c

test.c
-------
#define TEST 100

now, I run genxref & when I search for TEST in identifiers, I get that it is a macro defined in  
test.c at line 1

now I change test.c  to
-------
#define T 1
#define TEST 100

& run genxref

Now what I get is - TEST is defined as a macro in test.c in line 1 and line 2 !

The culprit is this piece of code in function processfile() [ Tagger.pm ]
------
          if ($index->toindex($fileid)) {
                $index->empty_cache();
                print(STDERR "--- $pathname $fileid\n");

                my $path = $files->tmpfile($pathname, $release);

                $lang->indexfile($pathname, $path, $fileid, $index, $config);
                unlink($path);
          } else {
                print(STDERR "$pathname was already indexed\n");
          }
------
The problem is that if the file already existed and has changed since then [based on the 
timestamp], the identifiers added to the database due to this file in the previous run of genxref are 
not removed from the database, hence the number of definitions will keep on growing...

The same problem is also present in processrefs().


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2004-07-20 04:36

Message:
Logged In: NO 

Shree, you've said you had a patch, could you attach to the 
Tracker?
Thanks,
Dennis

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2004-07-20 04:31

Message:
Logged In: NO 

Still no fix for this?

----------------------------------------------------------------------

Comment By: Heikki Toivonen (hjtoi)
Date: 2004-03-12 09:48

Message:
Logged In: YES 
user_id=972898

Anyone have a patch for this?

----------------------------------------------------------------------

Comment By: Richard Kisley (kisley)
Date: 2003-04-07 17:39

Message:
Logged In: YES 
user_id=102080

(1) How about an intermediate solution, where someone writes
a VERIFY script which compares the paths in the database
with the version they refer to and deletes entries for
invalid paths?  Same options as genxref?

I indexed a subtree of a sourcetree, then realized I needed
to index the whole source tree.  So I moved the revision
main dir (since it was really a subdir) up a level and added
the other directories at their proper top level as other
subdirs, then re-indexed.  Now the original links in tree
one are all dead links, with live dupes.  No files changed. 

(2) So we all don't have to scurry for our SQL books, buried
in a box in the back of a closet at home (not work) how
about posting the exact drop syntax?  That might also be a
good thing to add to doc short-term, since genxref doesn't
work (prune) as expected. 

----------------------------------------------------------------------

Comment By: Gregor Hartmann (grex)
Date: 2002-06-07 06:10

Message:
Logged In: YES 
user_id=559509

Another similar problem would be files ore whole directories that are deleted from the source tree. They would stay 
in the database forever as well. 

Maybe it could be fixed by iterating through all files in the database and removing those (from the database) which 
have changed or were removed in the source tree.
then proceed indexing as before.


----------------------------------------------------------------------

Comment By: Shree Kumar (shreekumar)
Date: 2002-02-18 23:21

Message:
Logged In: YES 
user_id=142912

Here's my fix for this bug:

Add a field "timestamp" to the "status" table. And remove 
the "status" field.

Before finding identifiers in a file, check whether it's 
modification time is greater that it was previously. If 
yes, then remove all the identifier definitions due to this 
file [and release] from the database. Store the new 
timestamp in the database.

Before finding references in a file, remove all identifier 
references due to this file [and release] from the database.
[ No need to check the timestamp in this case since 
the "definitions" are always found before the references]. 

In a large CVS tree, it is quite possible that a file may 
change between the time it is "indexed" and "referenced". 
An easy way out of this seems to be to "index" a file and 
immediately "reference" it.

Related to this there is a problem in "Plain.pm" - the 
current  "filerev" function returns a value based on the 
timestamp.  Problem arises if a file changes between runs 
of genxref. What happens is that different values are 
returned by "filerev" even though it is the same 
(file,revision) pair is being indexed [or referenced].

I have changed filerev() for this purpose as

sub filerev {
 my ($self, $filename, $release) = @_;

 # TODO: length of filename+revision
 #       might turn out to be > 255 chars 
 #       [length used in the db]
        return join("-", $filename,
                                $release);
}

With this modification filerev() will return the same value 
for (file,revision) pair everytime - thus solving the 
problem.

I have a patch ready for this.


----------------------------------------------------------------------

Comment By: Malcolm Box (mbox)
Date: 2002-02-18 06:20

Message:
Logged In: YES 
user_id=215386

Yes, you're right, this is a bug.  The underlying assumption
that is being broken is that the files in a version are
static - which is true if one is indexing released software,
but not if it is a development tree.

The simplest work-around is to drop and recreate the
database each time, thus avoiding the problem.  For small to
medium repositories with the index updated nightly this
should work fine, but it doesn't work for large repositories.

The full solution would appear to be to check for an
existing entry for the (filename, release) pair and if it is
found delete it and all associated information.

----------------------------------------------------------------------

Comment By: Shree Kumar (shreekumar)
Date: 2002-02-16 05:32

Message:
Logged In: YES 
user_id=142912

There are two cases where the scenario that I've referred to applies:

1. Files are not in CVS [ ie usage of "Files.pm" ]. You run genxref, then change a file & genxref again

2. Files are in CVS, and you want to index the "head" tag. Files change regularly, and you want to keep the 
 
cross reference in sync - probably by running genxref once an hour or so [as a cron job].

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-02-16 04:47

Message:
Logged In: NO 

I was in the impression that a file may never ever change again, except if (and only if) the file was changed and has either got a new CVS revision (or tag) or if there is a new directory for a new version of the whole project (if it is not managed by CVS).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=518365&group_id=27350

[Lxr-dev] [ lxr-Bugs-518365 ] Indexing of files once indexed is buggy!

[SourceForge.net <no...@so...>]

Bugs item #518365, was opened at 2002-02-16 02:04
Message generated for change (Comment added) made by nobody
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=518365&group_id=27350

Category: genxref
Group: current cvs
Status: Open
Resolution: None
Priority: 7
Submitted By: Shree Kumar (shreekumar)
Assigned to: Malcolm Box (mbox)
Summary: Indexing of files once indexed is buggy!

Initial Comment:
I am using LXR-0.9.1

Consider this scenario :

There is a source tree "test" having only one file - test.c

test.c
-------
#define TEST 100

now, I run genxref & when I search for TEST in identifiers, I get that it is a macro defined in  
test.c at line 1

now I change test.c  to
-------
#define T 1
#define TEST 100

& run genxref

Now what I get is - TEST is defined as a macro in test.c in line 1 and line 2 !

The culprit is this piece of code in function processfile() [ Tagger.pm ]
------
          if ($index->toindex($fileid)) {
                $index->empty_cache();
                print(STDERR "--- $pathname $fileid\n");

                my $path = $files->tmpfile($pathname, $release);

                $lang->indexfile($pathname, $path, $fileid, $index, $config);
                unlink($path);
          } else {
                print(STDERR "$pathname was already indexed\n");
          }
------
The problem is that if the file already existed and has changed since then [based on the 
timestamp], the identifiers added to the database due to this file in the previous run of genxref are 
not removed from the database, hence the number of definitions will keep on growing...

The same problem is also present in processrefs().


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2004-07-20 04:31

Message:
Logged In: NO 

Still no fix for this?

----------------------------------------------------------------------

Comment By: Heikki Toivonen (hjtoi)
Date: 2004-03-12 09:48

Message:
Logged In: YES 
user_id=972898

Anyone have a patch for this?

----------------------------------------------------------------------

Comment By: Richard Kisley (kisley)
Date: 2003-04-07 17:39

Message:
Logged In: YES 
user_id=102080

(1) How about an intermediate solution, where someone writes
a VERIFY script which compares the paths in the database
with the version they refer to and deletes entries for
invalid paths?  Same options as genxref?

I indexed a subtree of a sourcetree, then realized I needed
to index the whole source tree.  So I moved the revision
main dir (since it was really a subdir) up a level and added
the other directories at their proper top level as other
subdirs, then re-indexed.  Now the original links in tree
one are all dead links, with live dupes.  No files changed. 

(2) So we all don't have to scurry for our SQL books, buried
in a box in the back of a closet at home (not work) how
about posting the exact drop syntax?  That might also be a
good thing to add to doc short-term, since genxref doesn't
work (prune) as expected. 

----------------------------------------------------------------------

Comment By: Gregor Hartmann (grex)
Date: 2002-06-07 06:10

Message:
Logged In: YES 
user_id=559509

Another similar problem would be files ore whole directories that are deleted from the source tree. They would stay 
in the database forever as well. 

Maybe it could be fixed by iterating through all files in the database and removing those (from the database) which 
have changed or were removed in the source tree.
then proceed indexing as before.


----------------------------------------------------------------------

Comment By: Shree Kumar (shreekumar)
Date: 2002-02-18 23:21

Message:
Logged In: YES 
user_id=142912

Here's my fix for this bug:

Add a field "timestamp" to the "status" table. And remove 
the "status" field.

Before finding identifiers in a file, check whether it's 
modification time is greater that it was previously. If 
yes, then remove all the identifier definitions due to this 
file [and release] from the database. Store the new 
timestamp in the database.

Before finding references in a file, remove all identifier 
references due to this file [and release] from the database.
[ No need to check the timestamp in this case since 
the "definitions" are always found before the references]. 

In a large CVS tree, it is quite possible that a file may 
change between the time it is "indexed" and "referenced". 
An easy way out of this seems to be to "index" a file and 
immediately "reference" it.

Related to this there is a problem in "Plain.pm" - the 
current  "filerev" function returns a value based on the 
timestamp.  Problem arises if a file changes between runs 
of genxref. What happens is that different values are 
returned by "filerev" even though it is the same 
(file,revision) pair is being indexed [or referenced].

I have changed filerev() for this purpose as

sub filerev {
 my ($self, $filename, $release) = @_;

 # TODO: length of filename+revision
 #       might turn out to be > 255 chars 
 #       [length used in the db]
        return join("-", $filename,
                                $release);
}

With this modification filerev() will return the same value 
for (file,revision) pair everytime - thus solving the 
problem.

I have a patch ready for this.


----------------------------------------------------------------------

Comment By: Malcolm Box (mbox)
Date: 2002-02-18 06:20

Message:
Logged In: YES 
user_id=215386

Yes, you're right, this is a bug.  The underlying assumption
that is being broken is that the files in a version are
static - which is true if one is indexing released software,
but not if it is a development tree.

The simplest work-around is to drop and recreate the
database each time, thus avoiding the problem.  For small to
medium repositories with the index updated nightly this
should work fine, but it doesn't work for large repositories.

The full solution would appear to be to check for an
existing entry for the (filename, release) pair and if it is
found delete it and all associated information.

----------------------------------------------------------------------

Comment By: Shree Kumar (shreekumar)
Date: 2002-02-16 05:32

Message:
Logged In: YES 
user_id=142912

There are two cases where the scenario that I've referred to applies:

1. Files are not in CVS [ ie usage of "Files.pm" ]. You run genxref, then change a file & genxref again

2. Files are in CVS, and you want to index the "head" tag. Files change regularly, and you want to keep the 
 
cross reference in sync - probably by running genxref once an hour or so [as a cron job].

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-02-16 04:47

Message:
Logged In: NO 

I was in the impression that a file may never ever change again, except if (and only if) the file was changed and has either got a new CVS revision (or tag) or if there is a new directory for a new version of the whole project (if it is not managed by CVS).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=518365&group_id=27350

Re: [Lxr-dev] Re: [Lxr-commits] CVS: lxr INSTALL,1.15,1.16 find,1.21,1.22 genxref,1.34,1.35 search,1.22,1.23

[Jan-Benedict G. <jb...@lu...>]

On Mon, 2004-07-19 16:58:22 -0400, Dave Brondsema <da...@br...>
wrote in message <Pine.LNX.4.58.0407191657170.6009@ferrari>:
> On Mon, 19 Jul 2004, Jan-Benedict Glaw wrote:
> > > --- genxref	19 Jul 2004 19:50:20 -0000	1.34
> > > +++ genxref	19 Jul 2004 20:03:31 -0000	1.35
> > > @@ -177,7 +177,7 @@
> > >  		system("chmod 644 $string");
> > >  	}
> > >
> > > -	if ( $config->swishdir and $config->swishindex ) {
> > > +	if ( $config->swishdir and $config->swishbin ) {
> > >  		my $swish =3D new IO::Handle;
> > >  		die $config->swishdir . " does not exist" unless -d $config->swish=
dir;
> > >  		my $filelist =3D new IO::File $config->swishdir . "/$release.filen=
ames", "w"
> >
> > Dito. Oh, and consistent space before an opening brace would also be
> > cool:) In the long run, of course...
> >
>=20
> I'll work on tweaking PerlTidy's configuration tomorrow.  Any other
> formatting suggestions?

*Personally*, I'd suggest something like

if ($blah) {
	something;
}

for braces, one space around concatenation dots and one space behind
comma (but none before).

Comments?

MfG, JBG

--=20
   Jan-Benedict Glaw       jb...@lu...    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier B=FCrger" | im Internet! |   im Ira=
k!
   ret =3D do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TC=
PA));

Re: [Lxr-dev] multi-language identifiers

[Malcolm B. <mb...@us...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dave Brondsema wrote:
| The 0.8 release announcement says:
|
| "In multi-language environments, keywords in a language are no longer
| hyperlinked if a different language happens to have them as an identifier"
|
| This doesn't seem to be the case for me.  Does anybody remember where or
| how this is supposed to happen?

I recall coding this - the keywords are now per-language, and the code
in Common.pm switches on the language of the file.  Once the file is
fragmented by SimpleParse, each fragment recognised as code is fed to
$lang->processcode where $lang should depend on the type of the file.

This then strips out reserved words before making links to symbols - see
Generic.pm line 162.

If this isn't working for you, my guess would be that either the setup
of $lang is hosed, or that processcode is not accurately reading the
languageconf info.

Cheers,

Malcolm
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (MingW32)
Comment: Using GnuPG with MultiZilla - http://enigmail.mozdev.org

iD8DBQFA/GxlQeMefPKyX/QRApFbAKDcq06wchQapfoBo6tHE3GMM0qxXwCgwEXT
W35JiA0dIP2MdNvuRlqLD10=
=NsQN
-----END PGP SIGNATURE-----

[Lxr-dev] multi-language identifiers

[Dave B. <da...@br...>]

The 0.8 release announcement says:

"In multi-language environments, keywords in a language are no longer
hyperlinked if a different language happens to have them as an identifier"

This doesn't seem to be the case for me.  Does anybody remember where or
how this is supposed to happen?


-- 
Dave Brondsema : da...@br...
http://www.brondsema.net : personal
http://www.splike.com : programming
http://csx.calvin.edu : student org

[Lxr-dev] Re: [Lxr-commits] CVS: lxr INSTALL,1.15,1.16 find,1.21,1.22 genxref,1.34,1.35 search,1.22,1.23

[Dave B. <da...@br...>]

On Mon, 19 Jul 2004, Jan-Benedict Glaw wrote:

> On Mon, 2004-07-19 13:03:34 -0700, Dave Brondsema <bro...@us...>
> wrote in message <E1B...@sc...>:
> > --- find	19 Jul 2004 19:50:20 -0000	1.21
> > +++ find	19 Jul 2004 20:03:31 -0000	1.22
> > @@ -112,7 +112,7 @@
> >  	my $casesensitive = $HTTP->{'param'}->{'casesensitive'};
> >
> >  	my $FILELISTING;
> > -	if ( $config->swishdir and $config->swishindex ) {
> > +	if ( $config->swishdir and $config->swishbin ) {
> >  		unless ( $FILELISTING = new IO::File( $config->swishdir . "/$release.filenames" ) ) {
>                       ^^^
> >  			&warning(
> >  "Version '$release' has not been indexed and is unavailable for searching<br>Could not open "
> >
>
> May I suggest to remove spaces after opening brace and before closing
> brace in the long term? At least, I think that looks better (well, for
> my eyes), but you may have another opinion about it...
>
> > --- genxref	19 Jul 2004 19:50:20 -0000	1.34
> > +++ genxref	19 Jul 2004 20:03:31 -0000	1.35
> > @@ -177,7 +177,7 @@
> >  		system("chmod 644 $string");
> >  	}
> >
> > -	if ( $config->swishdir and $config->swishindex ) {
> > +	if ( $config->swishdir and $config->swishbin ) {
> >  		my $swish = new IO::Handle;
> >  		die $config->swishdir . " does not exist" unless -d $config->swishdir;
> >  		my $filelist = new IO::File $config->swishdir . "/$release.filenames", "w"
>
> Dito. Oh, and consistent space before an opening brace would also be
> cool:) In the long run, of course...
>

I'll work on tweaking PerlTidy's configuration tomorrow.  Any other
formatting suggestions?

-- 
Dave Brondsema : da...@br...
http://www.brondsema.net : personal
http://www.splike.com : programming
http://csx.calvin.edu : student org

Re: [Lxr-dev] Re: [Lxr-commits] CVS: lxr diff,1.10,1.11 find,1.18,1.19 ident,1.17,1.18 search,1.19,1.20 source,1.40,1.41

[Dave B. <da...@br...>]

Malcolm Box wrote:
> Hi Dave,
> 
> Dave Brondsema wrote:
> | Modified Files:
> |     diff find ident search source
> | Log Message:
> | add -T switch for taint checking in CGI mode
> 
> So what happens in mod_perl mode?  There is definitely a problem with
> the current httpwash function, but are we sure that the new filtering
> doesn't open another security hole, including in mod_perl mode?
> 
> We've already had one security hole through bad parameters, I don't want
> another :-)
> 

I enabled taint checking for mod_perl too.

I had to "untaint" input in several places where it ended up in a 
vulnerable place.  I didn't have to do this for the exec() calls because 
(I think) they are set up so that perl passes the parameters directly to 
the executing program (diff, glimpse, swish-e), not through the shell. 
So there's no shell vulnerability.  Granted, weird parameters could make 
glimpse or swish-e do weird things, but I think that is out of our 
control.  I untainted with pretty strict regexps when rcs commands were 
called from CVS.pm.  I untainted without any validation when the config 
files are eval()'d.

I think a more likely exploit would be someone passing a perl regexp to 
the find page which used up tons of CPU.


-- 
Dave Brondsema : da...@br...
http://www.splike.com : programming
http://csx.calvin.edu : student org
http://www.brondsema.net : personal

[Lxr-dev] Re: [Lxr-commits] CVS: lxr diff,1.10,1.11 find,1.18,1.19 ident,1.17,1.18 search,1.19,1.20 source,1.40,1.41

[Malcolm B. <ma...@br...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Dave,

Dave Brondsema wrote:
| Modified Files:
| 	diff find ident search source
| Log Message:
| add -T switch for taint checking in CGI mode

So what happens in mod_perl mode?  There is definitely a problem with
the current httpwash function, but are we sure that the new filtering
doesn't open another security hole, including in mod_perl mode?

We've already had one security hole through bad parameters, I don't want
another :-)

Cheers,

Malcolm
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (MingW32)
Comment: Using GnuPG with MultiZilla - http://enigmail.mozdev.org

iD8DBQFA+DvcQeMefPKyX/QRAg+ZAKDPod6qhoUS9wIqXmKEh/4aubBL4QCdFtBZ
F8dunLZPF6oOluxbYOwFa8U=
=RPI4
-----END PGP SIGNATURE-----

[Lxr-dev] [ lxr-Bugs-525825 ] genxref gets wrong config

[SourceForge.net <no...@so...>]

Bugs item #525825, was opened at 2002-03-05 01:23
Message generated for change (Comment added) made by brondsem
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=525825&group_id=27350

Category: genxref
Group: v0.9.1
>Status: Closed
>Resolution: Fixed
Priority: 6
Submitted By: Malcolm Box (mbox)
Assigned to: Malcolm Box (mbox)
Summary: genxref gets wrong config

Initial Comment:
Using an lxr.conf file with two blocks, one
http://lon-xref/lxr and one http://lon-xref/lxr-new,
genxref would not complain for urls such as
http://lon-xref/lxr-wibble.

Instead, it appears to pick up the http://lon-xref/lxr
configuration.

----------------------------------------------------------------------

>Comment By: Dave Brondsema (brondsem)
Date: 2004-07-15 10:54

Message:
Logged In: YES 
user_id=341298

fixed in cvs

----------------------------------------------------------------------

Comment By: Malcolm Box (mbox)
Date: 2002-09-13 06:43

Message:
Logged In: YES 
user_id=215386

Cause lies in config.pm, where the url is matched against
the config root value.  If one config root is a substring of
another, then the wrong one will be picked up.  E.g with
http://example.com/lxr and
http://example.com/lxr-development the first will always be
selected.

The obvious fix of adding $ to the regexp doesn't work,
since the URL will include a script name (and possibly a
path, e.g. for source).



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=525825&group_id=27350

[Lxr-dev] [ lxr-Bugs-773276 ] Needs better compatibility with non-mod_perl

[SourceForge.net <no...@so...>]

Bugs item #773276, was opened at 2003-07-17 17:19
Message generated for change (Comment added) made by brondsem
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=773276&group_id=27350

Category: Browsing
Group: v0.9.1
>Status: Closed
Resolution: None
Priority: 5
Submitted By: Brendan Byrd (sineswiper)
>Assigned to: Dave Brondsema (brondsem)
Summary: Needs better compatibility with non-mod_perl

Initial Comment:
There are several hacks that need to be applied to make 
this compatible as a regular cgi-script, namely 
commenting out the HTTP/1.0 line, and change the icon 
names.  (internal-gopher-unknown?!  What is that?)

This needs to be its own variable.  As I may need to 
improve the Perl interface, anyway, I may imploy this 
change myself.

----------------------------------------------------------------------

>Comment By: Dave Brondsema (brondsem)
Date: 2004-07-15 10:23

Message:
Logged In: YES 
user_id=341298

both fixed

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2003-07-30 05:05

Message:
Logged In: NO 

internal-gopher-unknown is some internal images used in
Netscape4. They don't exist in Mozilla/IE.
But you can use Apache's icons instead, just by commenting
out he right lines in 'source' :)

HTTP/1.0 seems commented out already (at least in current
CVS), and indeed it works very well without mod-perl.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=773276&group_id=27350

[Lxr-dev] [ lxr-Feature Requests-510990 ] Don't require glimpse for file search

[SourceForge.net <no...@so...>]

Feature Requests item #510990, was opened at 2002-01-30 19:27
Message generated for change (Comment added) made by brondsem
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390120&aid=510990&group_id=27350

Category: General
Group: Next Release
>Status: Closed
Priority: 5
Submitted By: Nobody/Anonymous (nobody)
>Assigned to: Dave Brondsema (brondsem)
Summary: Don't require glimpse for file search

Initial Comment:
Since you already have the filenames from the 
database, you don't need glimpse for the file name 
search.

Instead you can just run a SQL query like:
select filename from files where filename like '%file%';

Which is equivalent to running glimpseindex on 'file'. 
Works in mysql, not sure about Postgres.

'_' is also equivalent to '.' in glimpse for match any 
character. It's not quite full regexp support, but 
it's better than nothing, especially when glimpse is 
not an option.




----------------------------------------------------------------------

>Comment By: Dave Brondsema (brondsem)
Date: 2004-07-14 17:02

Message:
Logged In: YES 
user_id=341298

a flat textfile is used for both glimpse and swish-e now

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390120&aid=510990&group_id=27350

[Lxr-dev] [ lxr-Feature Requests-758228 ] ignore some files in sourcdirs

[SourceForge.net <no...@so...>]

Feature Requests item #758228, was opened at 2003-06-20 20:24
Message generated for change (Comment added) made by brondsem
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390120&aid=758228&group_id=27350

Category: General
Group: None
>Status: Closed
Priority: 5
Submitted By: Richard Offer (offer)
>Assigned to: Dave Brondsema (brondsem)
Summary: ignore some files in sourcdirs

Initial Comment:

My sourcedirs include some binary files, I would like
to be able to exclude those either by path or extension
during the genxref stage, since it tries to cat them
and catting binaries to /dev/tty is only "interesting"
the first time it happens...


thanks

richard.


----------------------------------------------------------------------

>Comment By: Dave Brondsema (brondsem)
Date: 2004-07-14 17:01

Message:
Logged In: YES 
user_id=341298

binary files are now detected and ignored properly

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390120&aid=758228&group_id=27350

[Lxr-dev] [ lxr-Bugs-779639 ] Incorrect regexp in cleanstring

[SourceForge.net <no...@so...>]

Bugs item #779639, was opened at 2003-07-29 11:15
Message generated for change (Settings changed) made by brondsem
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=779639&group_id=27350

Category: genxref
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Nobody/Anonymous (nobody)
>Assigned to: Dave Brondsema (brondsem)
Summary: Incorrect regexp in cleanstring

Initial Comment:
in the cleanstring function (In lib/LXR/Files/CVS.pm),
one regexp is incorrect.

It is currently :

        s/[|&!`;$%<>[:cntrl:]]// ||   # drop these in
particular

where it should be :

        s/[|&!`;\$\%<>[:cntrl:]]// ||   # drop these in
particular

(i.e. $% is interpreted by Perl).

Testcase :
&&& /mccvsweb/repositories/i390/requestaccess.html head 
co:
/cvs/testcvs/cvs//mccvsweb/repositories/i39/requestaccess.html,v:
No such file or directory

the 0 has been replaced where it shouldn't.

Hope this helps...

-- 
Julien Wajsberg
fl...@mi...

----------------------------------------------------------------------

Comment By: Jesse Brandeburg (go_jesse)
Date: 2003-08-07 21:16

Message:
Logged In: YES 
user_id=631160

This bug actually effects all sorts of stuff, and is very major 
IMO

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=779639&group_id=27350

[Lxr-dev] [ lxr-Bugs-989603 ] Running with a hugh number of source versions.

[SourceForge.net <no...@so...>]

Mime-Version: 1.0
Content-Type: text/plain; charset="ISO-8859-1"

Bugs item #989603, was opened at 2004-07-12 19:49
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=989603&group_id=27350

Category: Browsing
Group: current cvs
Status: Open
Resolution: None
Priority: 5
Submitted By: Jan-Benedict Glaw (jbglaw)
Assigned to: Nobody/Anonymous (nobody)
Summary: Running with a hugh number of source versions.


Initial Comment:
Hi!

This is a whishlist bug.

If you're running with a hugh number of source versions
(think of all linux kernel versions or the like),
showing them
off in the head part takes quite some space. Putting them
into a drop-down list would make it look better.

Thanks, JBG


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=989603&group_id=27350

[Lxr-dev] [ lxr-Bugs-622920 ] Freetext search wíth swish-e

[SourceForge.net <no...@so...>]

Bugs item #622920, was opened at 2002-10-14 10:12
Message generated for change (Comment added) made by mbox
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=622920&group_id=27350

Category: None
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Bernhard Penz (bpenz)
Assigned to: Nobody/Anonymous (nobody)
Summary: Freetext search wíth swish-e

Initial Comment:
Hi,

When explicitly selecting a (cvs) version in the source 
dialog, and then going to free text seach with swish-e, 
search fails with the following error:

"Search failed
err: Index file error: Could not open the index 
file '/home/pages/lxr/indexed_src/public/hello/swish-
e/1.11.index': No such file or directory "

1.11 is the version number chosen. I did not succeed 
with getting any files generated with --allversions. Using 
the latest tarball of lxr.


greetings,
Berni

----------------------------------------------------------------------

>Comment By: Malcolm Box (mbox)
Date: 2004-07-12 18:27

Message:
Logged In: YES 
user_id=215386

There has been a bunch of work on swish-e and CVS recently
(thanks Dave B!) so this should now be working.

So if you try the latest CVS version, it should solve your
problem.  If not, please reopen this bug.

Thanks

Malcolm

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=622920&group_id=27350

[Lxr-dev] [ lxr-Bugs-716701 ] $SIG{__WARN__} and mod_perl

[SourceForge.net <no...@so...>]

Bugs item #716701, was opened at 2003-04-07 13:17
Message generated for change (Comment added) made by mbox
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=716701&group_id=27350

Category: Browsing
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Pavel Hlavnicka (pavel_hlavnicka)
Assigned to: Nobody/Anonymous (nobody)
Summary: $SIG{__WARN__} and mod_perl

Initial Comment:
To be honest, this problem brought many dreamless
nights to me  :)

We run LXR in the mod_perl context together with
another applications. Problem is simple.

1) LXR sets the $SIG{__WARN__} handler (i Common.pm),
and it is, of course, global for the whole mod_perl context

2) the handler prints to the standard output(!!!) bu
default (the $wwwdeug variable)

As a result of this, in some environments (where LXR is
running), under certain conditions (pretty rare in my
case), some 'random' output is displayed in the page.
(typically if something goes wrong in eval {}; block,
what may be exactly what autor supposed).

Uff, is think, $www debug should be cleared in the
distribution, and setting the warning handler at all is
a bit disputable, as some other application may be in
trouble due this.

Thanks for LXR

Pavel

----------------------------------------------------------------------

>Comment By: Malcolm Box (mbox)
Date: 2004-07-12 18:25

Message:
Logged In: YES 
user_id=215386

I've switched off wwwdebug in the CVS version, but the
underlying problem of the use of global signal handlers is
still there.

Any suggestions for how to remove these would be welcome!

----------------------------------------------------------------------

Comment By: Pavel Hlavnicka (pavel_hlavnicka)
Date: 2003-04-07 13:17

Message:
Logged In: YES 
user_id=302801

I need to tell, we are using 0.9.2.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=716701&group_id=27350