[Tcl-bugs] [ tcl-Bugs-3173086 ] large list segfault

[SourceForge.net <no...@so...>]

Bugs item #3173086, was opened at 2011-02-04 17:54
Message generated for change (Comment added) made by dgp
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110894&aid=3173086&group_id=10894

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: 14. List Object
Group: None
Status: Open
Resolution: None
Priority: 7
Private: No
Submitted By: Jeff Rogers (dvrsn)
Assigned to: Don Porter (dgp)
Summary: large list segfault

Initial Comment:
The following code segfaults for me on tcl 8.6b1.2, x86_64 Linux:

set a [string repeat a 102400] ; puts [string length $a]
set b [string repeat $a 10583] ; puts [string length $b]
set c [lrepeat 10485 $a]
puts [string length [list $c $b]]

I suspect it is a hole in the fix for bug 1267380.

----------------------------------------------------------------------

>Comment By: Don Porter (dgp)
Date: 2011-02-17 16:11

Message:

Attached a new patch which is a significant
rewrite of the Tcl*(Scan|Convert)*Element()
system, and revisions to the callers.

This is not in a state ready to commit at the moment,
as I need to go through and add/update the comments,
but I wanted to post it so folks can test it, especially to
examine whether there's any significant performance
impact in either direction.


----------------------------------------------------------------------

Comment By: Don Porter (dgp)
Date: 2011-02-14 14:48

Message:
Attached a patch to fix the TCCE bug, and
completely eliminate the unnecessary
BRACES_UNMATCHED flag.

This doesn't fix the reported bug.  Just
provides a better starting point.

----------------------------------------------------------------------

Comment By: Don Porter (dgp)
Date: 2011-02-14 14:14

Message:
Yes, so a demo of the problem is:

% entry .e -vcmd {string is list {%s}}
.e
% .e insert 0 \\{
% .e validate
0


----------------------------------------------------------------------

Comment By: Donal K. Fellows (dkf)
Date: 2011-02-10 18:20

Message:
Tk uses that combination in the entry validation code.

----------------------------------------------------------------------

Comment By: Don Porter (dgp)
Date: 2011-02-10 13:56

Message:

There's a (somewhat) related bug in Tcl_ConvertCountedElement()
that can only hit those who make use of the documented ability to
pass in the TCL_DONT_USE_BRACES flag.  It's a bit more work
than I really want to take on to create a demo test/script because
we do not yet have a testing command to exercise Tcl_Scan* and
Tcl_Convert* .  But a text desciption should be clear.

The problem is that TCCE is using solely the BRACES_UNMATCHED
flag to determine whether or not braces need to be escaped.  This is
not good enough.  Consider the example where the string rep of the
list element is the two character sequence \{ ( that is BACKSLASH
OPENBRACE, 
or \u005C\u007B ).

Then Tcl_ScanCountedElement will return back the flags value USE_BRACES.
Because the open brace is escaped, it contributes no nesting, so there's
no
problem in that string with unmatched braces.  In the usual list building,
then,
The formatted element would be {\{} .  See this at work:

% list \\{
{\{}

But Tcl_ConvertCountedElement() is supposed to support
callers who pass in TCL_DONT_USE_BRACES.  When that
flag is passed in, the formatted element would be \\{ .  The backslash
gets paired with an escaping backslash due to the flag.  However,
the open brace does not get escaped becuase TCCE controls that
choice solely with the BRACES_UNMATCHED flag, which is *not* set.

The problem now is this result fails to pass the validity test spelled
out
in the comments and tested by test util-3.1 that when the formatted
version of
a list element is enclosed in braces, the result must be a valid list. 
Various
list building routines rely on this property.    The string {\\{} is not a
valid Tcl
list.  TCCE fails its mission.

This is of very little importance, becuase only callers passing the TDUB
flag
can trigger the bug, and near as I can tell no such creatures exist.

However, the fix is to make the escaping of braces by TCCE happen
whenever the TDUB flag is set, no matter whether or not BU is set as
well.

But then, we discover that the BU flag is never used for anything
anywhere,
and its existence in the properly functioning code is an unnecessary
complication.


----------------------------------------------------------------------

Comment By: Don Porter (dgp)
Date: 2011-02-07 12:52

Message:
Cause of the problem is that Tcl_ScanCountedElement()
is returning a negative value, which it is never supposed to do.


----------------------------------------------------------------------

Comment By: Don Porter (dgp)
Date: 2011-02-07 11:43

Message:
reproduced on core-8-5-branch


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110894&aid=3173086&group_id=10894