lxr-developer — Don't use, active developers only

[Lxr-dev] [ lxr-Bugs-3204171 ] Garbled output for include files

[SourceForge.net <no...@so...>]

Bugs item #3204171, was opened at 2011-03-09 14:23
Message generated for change (Settings changed) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204171&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Browsing
Group: current cvs
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Andre-Littoz (ajlittoz)
Assigned to: Andre-Littoz (ajlittoz)
Summary: Garbled output for include files

Initial Comment:
LXR v0.9.8

If include filename is <span or contains class or reserved, HTML output is garbled with <span > block embedded in <pan > block.

This is the same bug as that related to variable using HTML keywords and attributes (span, class, reserved).

Its cause is the same: a 2-stage process where substitutions from the first are not protected against substitutions of the second.

Proposed patch:

1- Lang.pm

110304 processinclude
  Same design flaw as in processcode (Generic.pm): all replacement/markings should be processed simultaneously. Since we are supposed to find a single include in the fragment, it can be dealt with without loop.
  Beware! It has been very tedious to debug.

!-77,80 110304   (replace lines 77 to 80 with:)
!    my $source = $$frag;
!
!	$source =~ s/^					# reminder: no initial space in the grammar
!				([\w\#]\s*[\w]*)  	# reserved keyword for include construct
!				(\s+)				# space
!				(?|	(\")(.+?)(\")	# C syntax
!				|	(\0<)(.+?)(\0>)	# C alternate syntax
!				|	()([\w:]+)(\b)	# Perl and others
!				)
!				//sx ;
!	$$frag =	( $self->isreserved($1)
!				? "<span class='reserved'>$1</span>"
!				: "$1"
!				)
!			.	"$2$3"
!			.	&LXR::Common::incref($4, "include" ,$4 ,$dir)
!			.	"$5"
!			. $source;		# tail if any (e.g. in Perl)
--------------------- end of patch -----------------------

2- Generic.pm

   Since this bug is closely related to the one for variables span class and reserved, I give the associated patch here for sub processcode

  This sub HTML-marks the sequences of character equal to a reserved word, then does the same for identifiers. If it happens that one identifier looks
  the same as an HTML tag, attribute or attribute value, the HTML marking gets marked itself. W3C says HTML sentences cannot be embedded inside a <...>. The net result is the browser gets confused and displays garbage.
  HTML-markings must be protected from rescan of file fragment. The only way to do it safely, whatever the other processings, is to do all replacements simultaneously, so that they are mutually exclusive.
  It has been chosen to scan the fragment from left to right without backtracking: candidate prefixes are removed from the fragment and processed for replacement. The replacement result is then appended to a string. This string is returned as the value of the sub when scanning is complete.

  NOTE: lines containing $identdef are related language dependent recognition of identifiers (patch dated 110304)

!155,155 110303	(replace line 155 with the following, effectively removing processreserved)
!    my $source = $$code;
!    my $answer = ''; 
!    my $identdef = $self->langinfo('identdef');
!-158,165 110303 replace regexp substitution
!    while ( $source =~ s/^(.*?)($identdef)\b//s){
!      $answer .= "$1" .
!                 ( $self->isreserved($2)
!                 ? "<span class='reserved'>$2</span>"
!                 :
!                   ( $index->issymbol($2, $$self{'releaseid'})
!                   ? join($2, @{$$self{'itag'}})
!                   : $2
!                   )
!                 );
!    }
!    $$code = $answer . $source; 
--------------------------- end of patch ------------------------


----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2011-03-12 09:42

Message:
Same cause as 3089456, same treatment

To allow for more future processing on identifiers, transformed two-pass
process (where result of first process gets rescanned by second process,
leading to eventual marking of HTML tags) in single-pass process with
switch-like routing to appropriate transformation.

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2011-03-12 09:42

Message:
This has now been fixed in CVS.

If you can install the new version and check that it solves your
problem, then it would be very useful.

Thanks for reporting this defect and helping to make LXR better.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204171&group_id=27350

[Lxr-dev] [ lxr-Bugs-3089456 ] variables span or class yeild nested tag ugliness

[SourceForge.net <no...@so...>]

Bugs item #3089456, was opened at 2010-10-18 03:49
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3089456&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Lang support
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Marc W. Mengel (mengel)
Assigned to: Andre-Littoz (ajlittoz)
Summary: variables span or class yeild nested tag ugliness

Initial Comment:
If you cross-reference C code that has variable "span" the markup engine first puts 
<span class='reserved'>typedef</span>
and then the identifier markup turns that into
<<a href="...ident?i=span">span</a> class='reserved'>typedef</span>

If you have a program with all 3 of "span", "class" and "reserved" as variables, the results are particualrly special.

I've worked around this somewhat by hacking up Generic.pm with the following patch, but there are still a few
such instances leaking through...
*** Generic.pm.orig	2010-09-27 11:35:23.000000000 -0500
--- Generic.pm	2010-10-17 13:09:59.000000000 -0500
***************
*** 155,168 ****
  
          $self->processreserved($code);
  
! 	# Replace identifier by link unless it's a reserved word
  	$$code =~ 
  	  s{ 
! 	     (^|[^\$\w\#])([-\w~\#][\w]*)\b 
  	   }
  	   {
  	     $1.
! 		   ( $index->issymbol($2, $$self{'releaseid'}) ? join($2, @{$$self{'itag'}}) : $2 );
    	 }gex;
  }
  
--- 155,168 ----
  
          $self->processreserved($code);
  
! 	# Replace identifier by link unless it's a reserved word or part of a span tag from above...
  	$$code =~ 
  	  s{ 
! 	     ((?:\s*<span.*?>.*?</span>\s*)+|[^\$\w\#]|^)([-\w~\#]?[\w]*)\b
  	   }
  	   {
  	     $1.
! 		    ((! $self->isreserved($2) && $index->issymbol($2, $$self{'releaseid'})) ? join($2, @{$$self{'itag'}}) : $2 );
    	 }gex;
  }
  


----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2011-03-12 09:36

Message:
To allow for more future processing on identifiers, transformed two-pass
process (where result of first process gets rescanned by second process,
leading to eventual marking of HTML tags) in single-pass process with
switch-like routing to appropriate transformation.

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2011-03-12 09:36

Message:
This has now been fixed in CVS.

If you can install the new version and check that it solves your
problem, then it would be very useful.

Thanks for reporting this defect and helping to make LXR better.

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2011-03-04 19:07

Message:
The same behaviour happens with processinclude if filename looks like a
reserved word or contains a pathological variable like above. Thus it
deserves the same kind of correction.

As you noticed, your proposed patch leaves some leaks. I tackled the issue
on a different principle: in my opinion, the cause is the multi-pass nature
of the replacements/taggings. All must be done simultaneously to prevent
the mis-behaviour.

Take a look to the help forum where I posted the whole bunch of my patches
under topic 'Help on non regression' as I'd like someone to check these on
different data than mine.

Regards,

Pat

----------------------------------------------------------------------

Comment By: Marc W. Mengel (mengel)
Date: 2010-10-18 16:46

Message:
Attached a patch which seems to work reliably now.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3089456&group_id=27350

[Lxr-dev] [ lxr-Bugs-3004819 ] Bad HTML output, part 2

[SourceForge.net <no...@so...>]

Bugs item #3004819, was opened at 2010-05-20 19:41
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3004819&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
>Resolution: Duplicate
Priority: 5
Private: No
Submitted By: Nobody/Anonymous (nobody)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: Bad HTML output, part 2

Initial Comment:
CLONE please close. Unfortunately, SF.net\\\'s web interface did not let me attach more than a single file.... attached is the example HTML output

----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2011-03-12 08:54

Message:
Duplicate, see 3089456 for correction

----------------------------------------------------------------------

Comment By: Marc W. Mengel (mengel)
Date: 2010-10-18 16:51

Message:
I put a fix for this in
https://sourceforge.net/tracker/?func=detail&aid=3089456&group_id=27350&atid=390117

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3004819&group_id=27350

[Lxr-dev] [ lxr-Bugs-3004815 ] Bad HTML output

[SourceForge.net <no...@so...>]

Bugs item #3004815, was opened at 2010-05-20 19:36
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3004815&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
>Category: Browsing
Group: None
>Status: Closed
>Resolution: Duplicate
Priority: 5
Private: No
Submitted By: Nobody/Anonymous (nobody)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: Bad HTML output

Initial Comment:
Using lxr-0.9.8, I get garbled output in the generated web pages. I have attached my lxr.conf and an example html page showing the problem. I am using MySQL as a backend with raw source files (not using CVS or git, etc.) I have reproduced this issue on Centos 5.3 and Ubuntu 10.04 Server.

----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2011-03-12 08:52

Message:
Same as  3089456; see this bug for correction

----------------------------------------------------------------------

Comment By: Marc W. Mengel (mengel)
Date: 2010-10-18 16:51

Message:
I put a fix for this in
https://sourceforge.net/tracker/?func=detail&aid=3089456&group_id=27350&atid=390117

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3004815&group_id=27350

[Lxr-dev] [ lxr-Bugs-3019336 ] ident makes bad getindex/getreference calls

[SourceForge.net <no...@so...>]

Bugs item #3019336, was opened at 2010-06-22 02:40
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3019336&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Database interface
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Nobody/Anonymous (nobody)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: ident makes bad getindex/getreference calls

Initial Comment:
http://lxr.cvs.sourceforge.net/viewvc/lxr/lxr/ident?r1=1.24&r2=1.25

When mbox made the fix for the XSS issue, he reverted the change of getindex -> symdeclarations and getreference -> symreferences.

If you trace the history of the Mysql module you'll see the change. (2009-04-25)

This is in 0.9.8 and it makes 0.9.8 unusable. 

The error message in my apache logs (scrubbed) is : 

 fatal: main, line 89: Can't locate object method "getreference" via package "LXR::Index::Mysql" at /opt/lxr/ident line 89.,
 fatal: main, line 52: Can't locate object method "getindex" via package "LXR::Index::Mysql" at /opt/lxr/ident line 52.


----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2011-03-12 08:42

Message:
Same as 2997787
If in need before next release, get update from CVS

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2011-03-12 08:42

Message:
This has now been fixed in CVS.

If you can install the new version and check that it solves your
problem, then it would be very useful.

Thanks for reporting this defect and helping to make LXR better.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3019336&group_id=27350

[Lxr-dev] [ lxr-Bugs-3089456 ] variables span or class yeild nested tag ugliness

[SourceForge.net <no...@so...>]

Bugs item #3089456, was opened at 2010-10-18 03:49
Message generated for change (Settings changed) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3089456&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Lang support
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Marc W. Mengel (mengel)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: variables span or class yeild nested tag ugliness

Initial Comment:
If you cross-reference C code that has variable "span" the markup engine first puts 
<span class='reserved'>typedef</span>
and then the identifier markup turns that into
<<a href="...ident?i=span">span</a> class='reserved'>typedef</span>

If you have a program with all 3 of "span", "class" and "reserved" as variables, the results are particualrly special.

I've worked around this somewhat by hacking up Generic.pm with the following patch, but there are still a few
such instances leaking through...
*** Generic.pm.orig	2010-09-27 11:35:23.000000000 -0500
--- Generic.pm	2010-10-17 13:09:59.000000000 -0500
***************
*** 155,168 ****
  
          $self->processreserved($code);
  
! 	# Replace identifier by link unless it's a reserved word
  	$$code =~ 
  	  s{ 
! 	     (^|[^\$\w\#])([-\w~\#][\w]*)\b 
  	   }
  	   {
  	     $1.
! 		   ( $index->issymbol($2, $$self{'releaseid'}) ? join($2, @{$$self{'itag'}}) : $2 );
    	 }gex;
  }
  
--- 155,168 ----
  
          $self->processreserved($code);
  
! 	# Replace identifier by link unless it's a reserved word or part of a span tag from above...
  	$$code =~ 
  	  s{ 
! 	     ((?:\s*<span.*?>.*?</span>\s*)+|[^\$\w\#]|^)([-\w~\#]?[\w]*)\b
  	   }
  	   {
  	     $1.
! 		    ((! $self->isreserved($2) && $index->issymbol($2, $$self{'releaseid'})) ? join($2, @{$$self{'itag'}}) : $2 );
    	 }gex;
  }
  


----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2011-03-04 19:07

Message:
The same behaviour happens with processinclude if filename looks like a
reserved word or contains a pathological variable like above. Thus it
deserves the same kind of correction.

As you noticed, your proposed patch leaves some leaks. I tackled the issue
on a different principle: in my opinion, the cause is the multi-pass nature
of the replacements/taggings. All must be done simultaneously to prevent
the mis-behaviour.

Take a look to the help forum where I posted the whole bunch of my patches
under topic 'Help on non regression' as I'd like someone to check these on
different data than mine.

Regards,

Pat

----------------------------------------------------------------------

Comment By: Marc W. Mengel (mengel)
Date: 2010-10-18 16:46

Message:
Attached a patch which seems to work reliably now.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3089456&group_id=27350

[Lxr-dev] [ lxr-Bugs-2997787 ] Unchased $release in ident script

[SourceForge.net <no...@so...>]

Bugs item #2997787, was opened at 2010-05-06 20:11
Message generated for change (Settings changed) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=2997787&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Browsing
Group: None
Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Nobody/Anonymous (nobody)
Assigned to: Andre-Littoz (ajlittoz)
Summary: Unchased $release in ident script

Initial Comment:
See the patch at http://codelabs.ru/patches/lxr/chase-dollar-release.diff

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2011-03-11 22:20

Message:
2 occurrences of $release leftover in lines 52 & 89 while revamping the DB
interface. Needs also changing function names.

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2011-03-11 22:20

Message:
This has now been fixed in CVS.

If you can install the new version and check that it solves your
problem, then it would be very useful.

Thanks for reporting this defect and helping to make LXR better.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=2997787&group_id=27350

[Lxr-dev] [ lxr-Bugs-3116684 ] Line continuation character "\" problem

[SourceForge.net <no...@so...>]

Bugs item #3116684, was opened at 2010-11-23 16:28
Message generated for change (Settings changed) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3116684&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Lang support
Group: None
Status: Pending
Resolution: None
>Priority: 8
Private: No
Submitted By: dcochlin (dcochlin)
Assigned to: Andre-Littoz (ajlittoz)
Summary: Line continuation character "\" problem

Initial Comment:
LXR Version: 0.9.8
Error message in genxref output: BTYPE was: atom
DB: mysql
Plain files.

When the code being parsed contains a line continuation character, like "\" in C, the line associated with "References:" to the tag are wrong.  For example, parsing this code (ignore the line numbers, they are included to show the problem with References)....

<<<
001 #define my_define (100 - \
002         25)
003 
004 char[] my_string;
005 
006 strcat(my_string, "Hello");
007 
008 for (i=0, i<my_define; ++i){
009         printf("The number is: %d", i);
010 }
011 
012 printf("%s", my_string);

>>>

Clicking on "my_define" results in the following. Note the line numbers in the references for "my_define" should be (1 and 8) not (1 and 7).

<<<
my_define

Declarations:
/deviceapps/aim_adams.c, line 1	macro (un)definition	

1 declarations in 1 files.

References:
  	/deviceapps/aim_adams.c, line 1
  	/deviceapps/aim_adams.c, line 7

2 references in 1 files.
>>>

Clicking on my_string results in something even stranger. There are only 3 references in the file and only one of the references displayed match the actual line number.

<<<
my_string

Declarations:
/deviceapps/aim_adams.c, line 4	variable definition	

1 declarations in 1 files.

References:
  	/deviceapps/aim_adams.c, line 3
  	/deviceapps/aim_adams.c, line 5
  	/deviceapps/aim_adams.c, line 11
  	/deviceapps/aim_adams.c, line 12

4 references in 1 files.
>>>

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2011-03-11 22:31

Message:
Origin of problem in the way source text is being parsed: to circumvent
erroneous detection of end of string in C when escaped delimiters are
present (\"), fragment "atom" has been introduced. Unhappily, 1/ this
fragment category is not local to string and can be activated any time, 2/
has a faulty definition pattern.

The pattern is triggered whenever a \ is encountered in source text.

Solution needs redesigning the parser in SimpleParse.pm and the language
specifications in generic.conf. Will be solved with other related bugs.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3116684&group_id=27350

[Lxr-dev] [ lxr-Bugs-3116684 ] Line continuation character "\" problem

[SourceForge.net <no...@so...>]

Bugs item #3116684, was opened at 2010-11-23 16:28
Message generated for change (Settings changed) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3116684&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Lang support
Group: None
>Status: Pending
Resolution: None
Priority: 5
Private: No
Submitted By: dcochlin (dcochlin)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: Line continuation character "\" problem

Initial Comment:
LXR Version: 0.9.8
Error message in genxref output: BTYPE was: atom
DB: mysql
Plain files.

When the code being parsed contains a line continuation character, like "\" in C, the line associated with "References:" to the tag are wrong.  For example, parsing this code (ignore the line numbers, they are included to show the problem with References)....

<<<
001 #define my_define (100 - \
002         25)
003 
004 char[] my_string;
005 
006 strcat(my_string, "Hello");
007 
008 for (i=0, i<my_define; ++i){
009         printf("The number is: %d", i);
010 }
011 
012 printf("%s", my_string);

>>>

Clicking on "my_define" results in the following. Note the line numbers in the references for "my_define" should be (1 and 8) not (1 and 7).

<<<
my_define

Declarations:
/deviceapps/aim_adams.c, line 1	macro (un)definition	

1 declarations in 1 files.

References:
  	/deviceapps/aim_adams.c, line 1
  	/deviceapps/aim_adams.c, line 7

2 references in 1 files.
>>>

Clicking on my_string results in something even stranger. There are only 3 references in the file and only one of the references displayed match the actual line number.

<<<
my_string

Declarations:
/deviceapps/aim_adams.c, line 4	variable definition	

1 declarations in 1 files.

References:
  	/deviceapps/aim_adams.c, line 3
  	/deviceapps/aim_adams.c, line 5
  	/deviceapps/aim_adams.c, line 11
  	/deviceapps/aim_adams.c, line 12

4 references in 1 files.
>>>

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2011-03-11 22:31

Message:
Origin of problem in the way source text is being parsed: to circumvent
erroneous detection of end of string in C when escaped delimiters are
present (\"), fragment "atom" has been introduced. Unhappily, 1/ this
fragment category is not local to string and can be activated any time, 2/
has a faulty definition pattern.

The pattern is triggered whenever a \ is encountered in source text.

Solution needs redesigning the parser in SimpleParse.pm and the language
specifications in generic.conf. Will be solved with other related bugs.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3116684&group_id=27350

[Lxr-dev] [ lxr-Bugs-3116684 ] Line continuation character "\" problem

[SourceForge.net <no...@so...>]

Bugs item #3116684, was opened at 2010-11-23 16:28
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3116684&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Lang support
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: dcochlin (dcochlin)
Assigned to: Nobody/Anonymous (nobody)
Summary: Line continuation character "\" problem

Initial Comment:
LXR Version: 0.9.8
Error message in genxref output: BTYPE was: atom
DB: mysql
Plain files.

When the code being parsed contains a line continuation character, like "\" in C, the line associated with "References:" to the tag are wrong.  For example, parsing this code (ignore the line numbers, they are included to show the problem with References)....

<<<
001 #define my_define (100 - \
002         25)
003 
004 char[] my_string;
005 
006 strcat(my_string, "Hello");
007 
008 for (i=0, i<my_define; ++i){
009         printf("The number is: %d", i);
010 }
011 
012 printf("%s", my_string);

>>>

Clicking on "my_define" results in the following. Note the line numbers in the references for "my_define" should be (1 and 8) not (1 and 7).

<<<
my_define

Declarations:
/deviceapps/aim_adams.c, line 1	macro (un)definition	

1 declarations in 1 files.

References:
  	/deviceapps/aim_adams.c, line 1
  	/deviceapps/aim_adams.c, line 7

2 references in 1 files.
>>>

Clicking on my_string results in something even stranger. There are only 3 references in the file and only one of the references displayed match the actual line number.

<<<
my_string

Declarations:
/deviceapps/aim_adams.c, line 4	variable definition	

1 declarations in 1 files.

References:
  	/deviceapps/aim_adams.c, line 3
  	/deviceapps/aim_adams.c, line 5
  	/deviceapps/aim_adams.c, line 11
  	/deviceapps/aim_adams.c, line 12

4 references in 1 files.
>>>

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2011-03-11 22:31

Message:
Origin of problem in the way source text is being parsed: to circumvent
erroneous detection of end of string in C when escaped delimiters are
present (\"), fragment "atom" has been introduced. Unhappily, 1/ this
fragment category is not local to string and can be activated any time, 2/
has a faulty definition pattern.

The pattern is triggered whenever a \ is encountered in source text.

Solution needs redesigning the parser in SimpleParse.pm and the language
specifications in generic.conf. Will be solved with other related bugs.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3116684&group_id=27350

[Lxr-dev] [ lxr-Bugs-2997787 ] Unchased $release in ident script

[SourceForge.net <no...@so...>]

Bugs item #2997787, was opened at 2010-05-06 20:11
Message generated for change (Comment added) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=2997787&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Browsing
Group: None
>Status: Closed
Resolution: None
Priority: 5
Private: No
Submitted By: Nobody/Anonymous (nobody)
Assigned to: Andre-Littoz (ajlittoz)
Summary: Unchased $release in ident script

Initial Comment:
See the patch at http://codelabs.ru/patches/lxr/chase-dollar-release.diff

----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2011-03-11 22:20

Message:
2 occurrences of $release leftover in lines 52 & 89 while revamping the DB
interface. Needs also changing function names.

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2011-03-11 22:20

Message:
This has now been fixed in CVS.

If you can install the new version and check that it solves your
problem, then it would be very useful.

Thanks for reporting this defect and helping to make LXR better.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=2997787&group_id=27350

[Lxr-dev] [ lxr-Bugs-2997787 ] Unchased $release in ident script

[SourceForge.net <no...@so...>]

Bugs item #2997787, was opened at 2010-05-06 20:11
Message generated for change (Settings changed) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=2997787&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Browsing
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Nobody/Anonymous (nobody)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: Unchased $release in ident script

Initial Comment:
See the patch at http://codelabs.ru/patches/lxr/chase-dollar-release.diff

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=2997787&group_id=27350

[Lxr-dev] [ lxr-Bugs-3116715 ] C/C++ language: #elsif should be #elseif

[SourceForge.net <no...@so...>]

Bugs item #3116715, was opened at 2010-11-23 17:00
Message generated for change (Settings changed) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3116715&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Lang support
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: dcochlin (dcochlin)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: C/C++ language: #elsif should be #elseif

Initial Comment:
- generic.conf should define #elseif instead of #elsif.
- Python language could use a triple quote comment
     'comment' => ('"""',   '"""'),

----------------------------------------------------------------------

Comment By: Damian Nycz (dnycz)
Date: 2011-01-29 18:14

Message:
In C/C++ language there is no '#elseif' nor '#elsif';
there is '#elif';
generic.conf should define '#elif' instead of '#elsif'.
There is missing '#line' directive in C/C++.
There is missing 'compl' C++ keyword.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3116715&group_id=27350

[Lxr-dev] [ lxr-Bugs-3204202 ] #directives not always recognised

[SourceForge.net <no...@so...>]

Bugs item #3204202, was opened at 2011-03-09 14:52
Message generated for change (Settings changed) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204202&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Lang support
Group: current cvs
Status: Open
Resolution: Works For Me
Priority: 5
Private: No
Submitted By: Andre-Littoz (ajlittoz)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: #directives not always recognised

Initial Comment:
C syntax allows to have spaces between # and directive names. This is correctly reflected in generic.conf patterns but not in isreserved code. The reserved keyword list in generic.conf uses the compact form of the directive.

Side effect of the proposed patch: do not forget to remove also spaces in Fortran key words the day they are implemented.

In Generic.pm, sub isreserved

!-170 110307   (after line 170, insert:)
!  $frag =~ s/\s//g ;        # for those who write # include
--------------- end of patch --------------------------

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204202&group_id=27350

[Lxr-dev] [ lxr-Bugs-3204218 ] Empty comment block at beginning of line

[SourceForge.net <no...@so...>]

Bugs item #3204218, was opened at 2011-03-09 15:08
Message generated for change (Settings changed) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204218&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Lang support
Group: None
Status: Open
Resolution: Works For Me
Priority: 3
Private: No
Submitted By: Andre-Littoz (ajlittoz)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: Empty comment block at beginning of line

Initial Comment:
With comments spanning multiple lines, an extraneous <span class="comment"></span> is left at the beginning of the line following the comment block.

This is caused by the \n of every line being changed to </span>\n<span class="comment">.

Though harmless, it is inelegant. It can easily be removed, contributing to transmit less data.

Proposed patch:

In Lang, sub processcomment

!-87 110304   (after line 87, insert:)
!	$$frag =~ s#<span class=\"comment\"></span>$## ; #remove excess marking
-------------------- end of patch ----------------------


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204218&group_id=27350

[Lxr-dev] [ lxr-Bugs-3204089 ] Image files never displayed

[SourceForge.net <no...@so...>]

Bugs item #3204089, was opened at 2011-03-09 13:07
Message generated for change (Settings changed) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204089&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Browsing
Group: current cvs
Status: Open
Resolution: Works For Me
Priority: 5
Private: No
Submitted By: Andre-Littoz (ajlittoz)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: Image files never displayed

Initial Comment:
LXR v0.9.8

Graphic file cannot be displayed because synthesized URL is a query for LXR to process a source file instead of a simple file path.

Proposed patch for Common.pm:

-316,321 110307 (replace lines 316 to 321 with:)
        &$outfun("<ul><table><tr><th valign=\"center\"><b>Image: </b></th></tr>\n");
        &$outfun("<tr><td>");
        &$outfun("<img src=\"$config->{virtroot}"
              . $pathname
              . "\" border=\"0\""
              . " alt=\"$pathname cannot be displayed by this browser\">\n");
        &$outfun("</td></tr></table></ul>");
--------------- end of patch -----------------

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204089&group_id=27350

[Lxr-dev] [ lxr-Bugs-3204096 ] Useless link on every line number

[SourceForge.net <no...@so...>]

Bugs item #3204096, was opened at 2011-03-09 13:14
Message generated for change (Settings changed) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204096&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Browsing
Group: current cvs
Status: Open
Resolution: Works For Me
Priority: 3
Private: No
Submitted By: Andre-Littoz (ajlittoz)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: Useless link on every line number

Initial Comment:
LXR v0.9.8
File Common.pm, sub markupfile

All lines of a file can be the target of a link. For that, they are tagged with an anchor <a name=#line>#line</a>. But in the standard implementation  they also contain href=(to itself) which does not make sense since we'll never need to jump from "here" to the "same place".

Getting rid of the href= will also contribute transferring less data.

Upon return &fileref(1,"fline",$pathname,1) is split in @ltag to later easily generate the tag. In the proposed change, the split is modified to "forget" the href:
  instead of /^(<a)(.*\#)001(\">)1(<\/a>)$/
  use /^(<a.*?)(?:href.*\#)001(\">)1(<\/a>)$/
  and change accordingly @ltag
  @ltag BEFORE:
    0 - <a (with name= appended)
    1 - class ... href="...#
    2 - ">
    3 - </a>
  @ltag AFTER:
    0 - <a class... (with name=" appended) -- double quote needed
    2 - ">
    3 - </a>
which sums up to:

-263,265 110226 (replace lines 263 to 265)
    my @ltag = &fileref(1, "fline", $pathname, 1) =~ /^(<a.*?)(?:href.*\#)001(\">)1(<\/a>)$/;
    $ltag[0] .= 'name="';
    $ltag[2] .= " ";
--------------- end of patch ----------

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204096&group_id=27350

[Lxr-dev] [ lxr-Bugs-3204208 ] Script language not always recognised

[SourceForge.net <no...@so...>]

Bugs item #3204208, was opened at 2011-03-09 15:00
Message generated for change (Settings changed) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204208&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Lang support
Group: None
Status: Open
Resolution: Works For Me
Priority: 5
Private: No
Submitted By: Andre-Littoz (ajlittoz)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: Script language not always recognised

Initial Comment:
LXR v0.9.8

In Lang.pm

  When trying to determine script language, test is for the beginning of the last part of a path, i.e. /interp. Matches if key of 'interpreters' is a prefix of the script interpreter name.
  Unhappily, prevents LXR-scripts from being recognised as they do not include a path.

  Test changed to match 'interpreters' name at end of string (without /)

  Potential problem: it is no longer a prefix test; are there cases where it matters?

!-53,53 110227   (replace line 53 of new with:)
!            if ($shebang =~ /$patt$/) {
---------------------- end of path --------------------------------

There is a related issue:

  Some files which are not scripts contain an emacs specification telling which language is used (e.g. the *.conf files in LXR). In that case, parse the spec to extract the language name and act just as if it was a script.

!-46,46 110308   (replace line 46 with:)
!        my $line = $fh->getline;
!        ($line =~ /^\#!\s*(\S+)/s)
!        || ($line =~ /^.*-[*]-.*?[ \t;]mode:[ \t]*(\w+).*-[*]-/);
--------------------------- end of patch ------------------------------


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204208&group_id=27350

[Lxr-dev] [ lxr-Bugs-3204197 ] Incorrect identifier recognition

[SourceForge.net <no...@so...>]

Bugs item #3204197, was opened at 2011-03-09 14:47
Message generated for change (Settings changed) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204197&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Lang support
Group: None
Status: Open
Resolution: Works For Me
Priority: 5
Private: No
Submitted By: Andre-Littoz (ajlittoz)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: Incorrect identifier recognition

Initial Comment:
LXR v0.9.8

Generic.pm::processcode uses a pattern for identifier recognition that encompasses all forms of identifiers in all languages. In Perl, identifiers may begin with a hyphen -. When this pattern is used for C, Pascal, Fortran, ... it may erroneously capture a variable preceded by the operator - (or part of --) and this string cannot be matched against the variable. It is preferable to customize the pattern per language. Thus, 'spec' in generic.conf is extended with a new key 'identdef' with the syntax:

'identdef' => pattern_string

pattern_string is not surrounded by pattern delimiters. It will be used by processcode to tag identifiers AND reserved words. It MUST then encompass these 2 classes of symbols. Do not add \b at the end, it will be done for you.

 Since it becomes a critical part of the process, a sensible default is always created (with the pattern used in version 0.9.8).

Proposed patch for Generic.pm

   in sub new

!-51 110304   (after line 51, insert:)
!    $$self{'langmap'}{$lang}{'identdef'} = '[-\w~\#][\w]*'
!        unless defined $self->langinfo('identdef');
!
-------------------------------- end of patch --------------------------------------

   in sub processcode

NOTE: this part of the patch is the same as the second part of patch for bug 3204171
  
!155,155 110303	remove processreserved
!    my $source = $$code;
!    my $answer = ''; 
!    my $identdef = $self->langinfo('identdef');
!-158,165 110303 replace regexp substitution
!    while ( $source =~ s/^(.*?)($identdef)\b//s){
!      $answer .= "$1" .
!                 ( $self->isreserved($2)
!                 ? "<span class='reserved'>$2</span>"
!                 :
!                   ( $index->issymbol($2, $$self{'releaseid'})
!                   ? join($2, @{$$self{'itag'}})
!                   : $2
!                   )
!                 );
!    }
!    $$code = $answer . $source; 
--------------------------- end of patch -------------------------


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204197&group_id=27350

[Lxr-dev] [ lxr-Bugs-3204085 ] Pattern graphicfile never matches

[SourceForge.net <no...@so...>]

Bugs item #3204085, was opened at 2011-03-09 13:02
Message generated for change (Settings changed) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204085&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Browsing
Group: current cvs
Status: Open
Resolution: Works For Me
Priority: 5
Private: No
Submitted By: Andre-Littoz (ajlittoz)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: Pattern graphicfile never matches

Initial Comment:
LXR v0.9.8

For an obscure reason (Perl syntax shortcoming or bug in current Linux Perl interpreter), test for "graphic" files in pattern /$config->graphicfile/ (Common.pm line 315) never succeeds. It looks like references $config->... cannot be use in a regexp no matter how you parenthesize them.

To correct, proceed in two steps. Proposed patch:

in Common.pm

-260 110307   (after line 260, insert:)
    my $graphic = $config->graphicfile;
-315,315 110307   (replace line 315 with:)
    } elsif ($pathname =~ /$graphic/) {
------------- end of patch ----------------

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204085&group_id=27350

[Lxr-dev] [ lxr-Bugs-3204114 ] Display nicely non ISO-8859-1 trees

[SourceForge.net <no...@so...>]

Bugs item #3204114, was opened at 2011-03-09 13:30
Message generated for change (Settings changed) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204114&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Browsing
Group: current cvs
Status: Open
Resolution: Works For Me
Priority: 5
Private: No
Submitted By: Andre-Littoz (ajlittoz)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: Display nicely non ISO-8859-1 trees

Initial Comment:
LXR v0.9.8

LXR sends an HTTP header advertising for ISO-8859-1 content which is not always the case with Unicode becoming ubiquitous.

This can be circumvented by creating a new tree-specific parameter for the charset to use with the tree. Its syntax is:

'encoding' => 'iso-8859-1'  # default value in order not to disrupt LXR operation

Instead of 'iso-8859-1', you can use any IANA defined charset.

This involves patches in several files:
1- Common.pm
   In sub printhttp, send correct header:

-448,448 110227  (replace line 448 with:)
        print("Content-Type: text/html; charset=", $config->{'encoding'}, "\n");
---------------------- end of patch --------------------

   In sub makeheader, add a variable substitution in case an HTML template wants to use 'encoding' for example in a <meta > tag:


-852 110227  (insert after line 852:)
                'encoding'   => sub { return $config->{'encoding'}; },
--------------------- end of patch ------------------------

2- Config.pm

   In sub initialize
  Make sure parameter 'encoding' has a reasonable default value

-109 110227   (insert after line 109)

    $$self{'encoding'} = "iso-8859-1" unless (exists $self->{'encoding'});
---------------------------- end of patch -----------------

3- html-head.html

   With the introduction of the 'encoding' config parameter to advertise the character set used in the content, add the following line after <title>:

-4   (insert after line 4)
<meta http-equiv="content-type" content="text/html; charset=$encoding">
---------------------- end of patch ---------------------


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204114&group_id=27350

[Lxr-dev] [ lxr-Bugs-3204171 ] Garbled output for include files

[SourceForge.net <no...@so...>]

Bugs item #3204171, was opened at 2011-03-09 14:23
Message generated for change (Settings changed) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204171&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Browsing
Group: current cvs
Status: Open
Resolution: Works For Me
Priority: 5
Private: No
Submitted By: Andre-Littoz (ajlittoz)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: Garbled output for include files

Initial Comment:
LXR v0.9.8

If include filename is <span or contains class or reserved, HTML output is garbled with <span > block embedded in <pan > block.

This is the same bug as that related to variable using HTML keywords and attributes (span, class, reserved).

Its cause is the same: a 2-stage process where substitutions from the first are not protected against substitutions of the second.

Proposed patch:

1- Lang.pm

110304 processinclude
  Same design flaw as in processcode (Generic.pm): all replacement/markings should be processed simultaneously. Since we are supposed to find a single include in the fragment, it can be dealt with without loop.
  Beware! It has been very tedious to debug.

!-77,80 110304   (replace lines 77 to 80 with:)
!    my $source = $$frag;
!
!	$source =~ s/^					# reminder: no initial space in the grammar
!				([\w\#]\s*[\w]*)  	# reserved keyword for include construct
!				(\s+)				# space
!				(?|	(\")(.+?)(\")	# C syntax
!				|	(\0<)(.+?)(\0>)	# C alternate syntax
!				|	()([\w:]+)(\b)	# Perl and others
!				)
!				//sx ;
!	$$frag =	( $self->isreserved($1)
!				? "<span class='reserved'>$1</span>"
!				: "$1"
!				)
!			.	"$2$3"
!			.	&LXR::Common::incref($4, "include" ,$4 ,$dir)
!			.	"$5"
!			. $source;		# tail if any (e.g. in Perl)
--------------------- end of patch -----------------------

2- Generic.pm

   Since this bug is closely related to the one for variables span class and reserved, I give the associated patch here for sub processcode

  This sub HTML-marks the sequences of character equal to a reserved word, then does the same for identifiers. If it happens that one identifier looks
  the same as an HTML tag, attribute or attribute value, the HTML marking gets marked itself. W3C says HTML sentences cannot be embedded inside a <...>. The net result is the browser gets confused and displays garbage.
  HTML-markings must be protected from rescan of file fragment. The only way to do it safely, whatever the other processings, is to do all replacements simultaneously, so that they are mutually exclusive.
  It has been chosen to scan the fragment from left to right without backtracking: candidate prefixes are removed from the fragment and processed for replacement. The replacement result is then appended to a string. This string is returned as the value of the sub when scanning is complete.

  NOTE: lines containing $identdef are related language dependent recognition of identifiers (patch dated 110304)

!155,155 110303	(replace line 155 with the following, effectively removing processreserved)
!    my $source = $$code;
!    my $answer = ''; 
!    my $identdef = $self->langinfo('identdef');
!-158,165 110303 replace regexp substitution
!    while ( $source =~ s/^(.*?)($identdef)\b//s){
!      $answer .= "$1" .
!                 ( $self->isreserved($2)
!                 ? "<span class='reserved'>$2</span>"
!                 :
!                   ( $index->issymbol($2, $$self{'releaseid'})
!                   ? join($2, @{$$self{'itag'}})
!                   : $2
!                   )
!                 );
!    }
!    $$code = $answer . $source; 
--------------------------- end of patch ------------------------


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204171&group_id=27350

[Lxr-dev] [ lxr-Bugs-3204145 ] Display something if initialization failed

[SourceForge.net <no...@so...>]

Bugs item #3204145, was opened at 2011-03-09 14:01
Message generated for change (Settings changed) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204145&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Browsing
Group: current cvs
Status: Open
Resolution: Works For Me
Priority: 5
Private: No
Submitted By: Andre-Littoz (ajlittoz)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: Display something if initialization failed

Initial Comment:
LXR v0.8.9


In case something goes wrong with initialization (config read in), LXR silently dies (from the user point of view). Error message is sent to server
error log which cannot be read by ordinary user. In the best case, displayed page does not change hinting at a no-op for the click. In the worst case, you get a blank screen which is really puzzling.

If it is not too bad, try to send an error page (only case presently managed: no baseurl found). For that, modify Config.pm so that a new property of object $config signals the cause of error. This property has been chosen to be 'configerror'. It is created when Config.pm::initialize detects an error. Caller may then test for this existence of this property and act accordingly.

Since this error signalling was needed by my use of URL to tell which tree to display in a multi-tree DB, there is also an addition to manipulate the URL to provide a meaningful informative message. In my usage, the tree name is located right before the script (i.e. source, ident, ...). It is printed in the message to show the eventual typo.

In lxr.conf, you may add a global parameter like:
		'treeextract' => '([^/]*)/[^/]*$'
which is pattern against the SCRIPT_NAME part of the URL. $1 is used in the error message. If you need grouping before your target, use (?: instead of (. 

Proposed patch:
1- Common.pm
   in sub httpinit:

!-496,496 110226   (replace line 496 with:)
!    if (exists $config->{'configerror'}) {
!        makeerrorpage('htmlfatal');
!        die "Can't find config for " . $HTTP->{'this_url'};
!    };
----------------------- end of patch -------------------------------

   Create anew sub makeerrorpage to issue an error page instead of 'Die'ing silently.
   Since LXR is badly initialized, use as few features as possible.
  For instance, if you don't trust a smart lxr.conf, don't use 'stylesheet' in your error page template. Anyway, if 'stylesheet' does not exist, the page is still built without substitution; it might do funny things to the UA but anyhow the page is displayed with default styles.
  Added a 'treeextract' parameter to provide a pattern (regexp) to extract the source tree name from URL since where to put it is a matter of personal taste.
  If 'treeextract' does not exist, extract the second to last part of SCRIPT_NAME.

!-895 110226 & 110301   (after line 895, insert:)
!sub makeerrorpage {
!	my $who = shift;
!	my $tmplname;
!	my $template = "<html><body><hr>\n"
!		      . "<div align='center'>\n"
!		      . "<h1>Unrecoverable Error</h1><br>\n"
!		      . "\$tree unknown\n"
!		      . "</div>\n</body></html>\n";
!
!	$tmplname = $who;
!
!	if ($config->value($tmplname)) {
!		if (open(TEMPL, $config->value($tmplname))) {
!			local ($/) = undef;
!			$template = <TEMPL>;
!			close(TEMPL);
!		} else {
!			warning("Template " . $config->value($tmplname) . " does not exist in ".`pwd`);
!		}
!	}
!
!	print("Content-Type: text/html; charset=iso-8859-1\n");
!	print("\n");
!
!	my $treeextract = '([^/]*)/[^/]*$'; # default: capture second to last fragment
!	if (exists ($config->{'treeextract'})) {
!		$treeextract = $config->treeextract;
!	}
!
!	print(
!		expandtemplate(
!			$template,
!			(
!				'tree'    => sub { $_ = $ENV{'SCRIPT_NAME' }; m!$treeextract!; return $1; },
!				'stylesheet' => sub { stylesheet(@_) },
!			)
!		)
!	);
!	$config = undef;
!	$files  = undef;
!	$index  = undef;
!}
!
----------------- end of patch ----------------

2- Config.pm

    in sub initialize
  In case something goes wrong with initialization (config read in), LXR silently dies (from the user point of view).
  Create 'configerror' to tell caller something went screwy. Value explains why.
  Don't do it for genxref, since it is executed from the console and STDERR is
  displayed normally. There, it is better to stop the script.
!-111,112 110226   (replace lines 111 and 112 with:)
!    if(!exists $self->{baseurl}) {
!        if("genxref" ne ($0 =~ /([^\/]*)$/)) {
!            $$self{'configerror'} = "nobaseurl"; return 1;
!        }  elsif($url =~ m!http://.+\.!) {
----------------- end of patch -----------------


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204145&group_id=27350

[Lxr-dev] [ lxr-Bugs-3204285 ] HTML code not enough XML compliant (cosmetic)

[SourceForge.net <no...@so...>]

Bugs item #3204285, was opened at 2011-03-09 15:51
Message generated for change (Settings changed) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204285&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Browsing
Group: None
Status: Open
Resolution: Works For Me
Priority: 2
Private: No
Submitted By: Andre-Littoz (ajlittoz)
>Assigned to: Andre-Littoz (ajlittoz)
Summary: HTML code not enough XML compliant (cosmetic)

Initial Comment:
LXR v0.9.8

File source generates HTML page which may contain informative data structured in paragraphs. These paragraphs are opened with <p> tag but never closed. This is OK for HTML but is not in the spirit of XML.

The proposed patch add a </p> tag at the end of the concerned paragraphs.

File: source

110225 direxpand

!-154,154
!              . " does not exist.</i>\n</p>\n");
!-156,156
!            "\<p align=\"center\">\n<i>This directory might exist in other versions, try 'Show attic files' or select a different Version.</i>\n</p>\n"
-------------------------- and of patch ------------------------

110225 printfile
!-293,293
!                "\<p align=\"center\">\n<i>The file $pathname does not exist.</i>\n</p>\n"
!-296,296
!                "\<p align=\"center\">\n<i>This file might exist in other versions, try 'Show attic files' or select a different Version.</i>\n</p>\n"
---------------- end of patch -----------------------

110225 source init code
!-308,308
!    print("\<p align=\"center\">\n<i>The file $pathname does not exist.</i>\n</p>\n");
------------------ end of patch ----------------------


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204285&group_id=27350

[Lxr-dev] [ lxr-Bugs-3204266 ] Fragment recognition too rudimentary (parsing)

[SourceForge.net <no...@so...>]

Bugs item #3204266, was opened at 2011-03-09 15:45
Message generated for change (Settings changed) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204266&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Lang support
Group: None
Status: Open
Resolution: Works For Me
>Priority: 8
Private: No
Submitted By: Andre-Littoz (ajlittoz)
Assigned to: Nobody/Anonymous (nobody)
Summary: Fragment recognition too rudimentary (parsing)

Initial Comment:
LXR uses a very simple parser to isolate homogeneous fragment of source file to ease recognition of keywords and variables. It tries to wipe out strings, comments and include constructs, so that what is left is composed only of operators and variables.

In some circumstances, the parser leaves strings before seeing the end of the string, e.g. in "A string with \" inside". The parser thinks "A string with \" is the string, the inside may be a variable and a new string is then opened out of sync. The cause is a simplistic parse based on pattern-matching with opening and closing delimiters for context. There is a need for an other kind of objects which maintain the parser in the same state when detected. What happens is data represented by the regexp associated to these objects is "swallowed" and nothing happens because they cannot be classified as opening or closing delimiters (provided it is possible, i.e. they do not appear also as these delimiters).

For that, 'spec' of generic.conf is modified to have 4 parameters insted of 3:

'spec' =>  [ context_name, open_pattern, close_pattern, stay_pattern,  ... ]

(which, by the way, means a huge rewrite of generic.conf)

In the case of C strings, we can now have (extract only):
               'string',  '"',  '"',  '\\\\"',

But it is not enough; we must take care that this stay_pattern is considered BEFORE end_pattern because the latter is a prefix of the former and the expected match leng of stay_pattern is longer than that of end_pattern. Unhappily, there is no way to guarantee that all patterns of a 'spec' can be sorted such that they'll match from the most specific to the less specific.

The proposed patch tries to solve that, but there will always be situations where it will fail. It is a consequence of pattern-matching parsing. Using a finite state automaton only to find candidate identifiers is not really worth it.

Other bug loosely related:
When the line is split according to all patterns of a spec, it must be determined if one element is an opening delimiter. The answer is positive if there is an exact match, that is the pattern extends from the start ^ to the end $. These anchors are added to the ends of the merged regexp containing all the patterns as a sequence of alternatives. It means that only the first delimiter is constrained to start at the beginning of the string and the last delimiter to end at the end of the string. This leads to false detection of short delimiters in the middle of strings.

This is also addressed in the proposed patch.


!-37 110307   (after line 37 in sub init, add:)
!my @stay;        # Fragment maintaining current context
!-49 110307   (after line 49, add:)
!	@stay     = ();
!-58,58 110307   (replace line 58 with:)
!	while (@_ = splice(@blksep, 0, 4)) {
!-61 110307   (after line 61, add:)
! 		push(@stay,   $_[3]);
!-74 110307   (after line 74, add:)
!
!	foreach (@stay) {
!		next if $_ eq '';
!		$split = "$_|" . $split;
!	}
------------------- end of patch -------------

in sub  init & nextfrag
  nextfrag sometimes matches erroneously delimiters because ^ and $ surround
  the merged patterns of all alternatives. ^ and $ should be put arround each
  alternative to ensure that the delimiter will match as a wholle and not as
  a part of the fragment.
!-68,68 110307 init
!		$open  .= "^($_)\$|";
!-138,138 110307 nextfrag
!				if ($frags[0] =~ /$open/) {
!-150,150 110307 nextfrag
!			if (defined($frag) && (@_ = $frag =~ /$open/)) {
------------------------ end of patch ---------------------------



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204266&group_id=27350