[Lxr-dev] [ lxr-Bugs-3204171 ] Garbled output for include files

[SourceForge.net <no...@so...>]

Bugs item #3204171, was opened at 2011-03-09 14:23
Message generated for change (Settings changed) made by ajlittoz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204171&group_id=27350

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Browsing
Group: current cvs
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Andre-Littoz (ajlittoz)
Assigned to: Andre-Littoz (ajlittoz)
Summary: Garbled output for include files

Initial Comment:
LXR v0.9.8

If include filename is <span or contains class or reserved, HTML output is garbled with <span > block embedded in <pan > block.

This is the same bug as that related to variable using HTML keywords and attributes (span, class, reserved).

Its cause is the same: a 2-stage process where substitutions from the first are not protected against substitutions of the second.

Proposed patch:

1- Lang.pm

110304 processinclude
  Same design flaw as in processcode (Generic.pm): all replacement/markings should be processed simultaneously. Since we are supposed to find a single include in the fragment, it can be dealt with without loop.
  Beware! It has been very tedious to debug.

!-77,80 110304   (replace lines 77 to 80 with:)
!    my $source = $$frag;
!
!	$source =~ s/^					# reminder: no initial space in the grammar
!				([\w\#]\s*[\w]*)  	# reserved keyword for include construct
!				(\s+)				# space
!				(?|	(\")(.+?)(\")	# C syntax
!				|	(\0<)(.+?)(\0>)	# C alternate syntax
!				|	()([\w:]+)(\b)	# Perl and others
!				)
!				//sx ;
!	$$frag =	( $self->isreserved($1)
!				? "<span class='reserved'>$1</span>"
!				: "$1"
!				)
!			.	"$2$3"
!			.	&LXR::Common::incref($4, "include" ,$4 ,$dir)
!			.	"$5"
!			. $source;		# tail if any (e.g. in Perl)
--------------------- end of patch -----------------------

2- Generic.pm

   Since this bug is closely related to the one for variables span class and reserved, I give the associated patch here for sub processcode

  This sub HTML-marks the sequences of character equal to a reserved word, then does the same for identifiers. If it happens that one identifier looks
  the same as an HTML tag, attribute or attribute value, the HTML marking gets marked itself. W3C says HTML sentences cannot be embedded inside a <...>. The net result is the browser gets confused and displays garbage.
  HTML-markings must be protected from rescan of file fragment. The only way to do it safely, whatever the other processings, is to do all replacements simultaneously, so that they are mutually exclusive.
  It has been chosen to scan the fragment from left to right without backtracking: candidate prefixes are removed from the fragment and processed for replacement. The replacement result is then appended to a string. This string is returned as the value of the sub when scanning is complete.

  NOTE: lines containing $identdef are related language dependent recognition of identifiers (patch dated 110304)

!155,155 110303	(replace line 155 with the following, effectively removing processreserved)
!    my $source = $$code;
!    my $answer = ''; 
!    my $identdef = $self->langinfo('identdef');
!-158,165 110303 replace regexp substitution
!    while ( $source =~ s/^(.*?)($identdef)\b//s){
!      $answer .= "$1" .
!                 ( $self->isreserved($2)
!                 ? "<span class='reserved'>$2</span>"
!                 :
!                   ( $index->issymbol($2, $$self{'releaseid'})
!                   ? join($2, @{$$self{'itag'}})
!                   : $2
!                   )
!                 );
!    }
!    $$code = $answer . $source; 
--------------------------- end of patch ------------------------


----------------------------------------------------------------------

>Comment By: Andre-Littoz (ajlittoz)
Date: 2011-03-12 09:42

Message:
Same cause as 3089456, same treatment

To allow for more future processing on identifiers, transformed two-pass
process (where result of first process gets rescanned by second process,
leading to eventual marking of HTML tags) in single-pass process with
switch-like routing to appropriate transformation.

----------------------------------------------------------------------

Comment By: Andre-Littoz (ajlittoz)
Date: 2011-03-12 09:42

Message:
This has now been fixed in CVS.

If you can install the new version and check that it solves your
problem, then it would be very useful.

Thanks for reporting this defect and helping to make LXR better.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=390117&aid=3204171&group_id=27350