[Tcl-bugs] [ tcl-Bugs-3603557 ] couldn't compile regular expression pattern: out of memory

[SourceForge.net <no...@so...>]

Bugs item #3603557, was opened at 2013-02-06 05:23
Message generated for change (Comment added) made by dkf
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110894&aid=3603557&group_id=10894

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: 43. Regexp
Group: obsolete: 8.6b2
Status: Open
Resolution: None
Priority: 7
Private: No
Submitted By: Jonathan Mills (jomills)
Assigned to: Donal K. Fellows (dkf)
Summary: couldn't compile regular expression pattern: out of memory

Initial Comment:
When I run the following test using tcl8.5 on Debian Squeeze (8.5.8-2) it works just fine, but when I try the same test using 8.6.0~b2-1 on the same PC I get the following error.

couldn't compile regular expression pattern: out of memory

I've looked to see if this bug had already been reported but I cound't find it, so apologies if I have raised a duplicate,
I've checked whether a newer version of 8.6 is available in Sid but it doens't look like it to me.

jmills@jo:~$ tclsh8.5
tclsh8.5 [~]source jotest.tcl 
tclsh8.5 [~]jotest [list TETRA_MODE_CMD ETS_2_2 CONTINUOUS 32 1 FALSE FALSE  512 8192 BOTH UNKNOWN TRUE 300 NORMAL NONE 10 1 0 25  -125 -33 720 -100 0 3 1111111111111111 110101110000  IMMEDIATE 5 15 0 0 0 FALSE 2 FALSE DISABLED]
1
tclsh8.5 [~]exit


jmills@jo:~$ tclsh8.6
tclsh8.6 [~]source jotest.tcl
tclsh8.6 [~]jotest [list TETRA_MODE_CMD ETS_2_2 CONTINUOUS 32 1 FALSE FALSE  512 8192 BOTH UNKNOWN TRUE 300 NORMAL NONE 10 1 0 25  -125 -33 720 -100 0 3 1111111111111111 110101110000  IMMEDIATE 5 15 0 0 0 FALSE 2 FALSE DISABLED]
couldn't compile regular expression pattern: out of memory
while evaluating {jotest [list TETRA_MODE_CMD ETS_2_2 CONTINUOUS 32 1 FALSE FALSE  512 8192 BOTH UNKNOWN TRUE 300 NORMAL NONE 10 1 0 25  -125 -33 720 -100 0 3 1111111111111111 110101110000  IMMEDIATE 5 15 0 0 0 FALSE 2 FALSE DISABLED]}
tclsh8.6 [~]exit


----------------------------------------------------------------------

>Comment By: Donal K. Fellows (dkf)
Date: 2013-02-07 01:34

Message:
Also, I'll expose the limit a bit better so that it can be boosted in
future by defining a symbol in the makefile. We shouldn't remove the depth
check entirely — that'd be an invitation to a stack smash — but we can
make it tunable in 8.6.

----------------------------------------------------------------------

Comment By: Donal K. Fellows (dkf)
Date: 2013-02-07 01:30

Message:
Confirming that raising the limit to 1000 gives enough room.

Can we use that RE as part of our test suite? It might be not very nice
(indeed, you'd find it more efficient to use (?:[[:blank:]]+) instead of
([[:blank:]]+) for the padding, so avoiding capturing it at all) but it
does apparently stress part of the code that we didn't expect to get hit
that way; it's worth something *as part of a test*. Testing of edge cases
*isn't* nice in the first place!

Won't commit anything until I've got a confirmed test case in place; we
want to ensure that we don't hit unexpected problems in the future too. ;-)

----------------------------------------------------------------------

Comment By: Jonathan Mills (jomills)
Date: 2013-02-06 12:14

Message:
>
> but *dude*, the input is still morally wrong.
>
Apologies, I've started to read up on how I should be parsing such
messages.

----------------------------------------------------------------------

Comment By: Donal K. Fellows (dkf)
Date: 2013-02-06 11:50

Message:
That RE is 8163 characters long. With 68 capturing sub-REs.

...
...
...

I guess we can fix this by raising the limit, especially given that Tcl no
longer makes the stack bloat up hugely, but *dude*, the input is still
morally wrong.

----------------------------------------------------------------------

Comment By: Don Porter (dgp)
Date: 2013-02-06 09:49

Message:
2008-07-01 14:29:11 1ab85d8c7df2ba97 BAD CURRENT
2008-07-01 13:24:06 a8de9c95aa018165 GOOD

$ fossil status
...
comment:      Add focussed stack limiting to the RE compiler. Tuning might
not
              yet be right but it passes everything normally checked in
the
              test suite. [Bug 1905562] (user: dkf)



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110894&aid=3603557&group_id=10894