Moving to a developer to-do. Good idea, but it's also one
of those "if it ain't broke..." kinds of deals. I looked at
it a release or two ago of PCRE and it didn't look
insumountable.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I had this running under the 3.0.3 code, was stable for me
under Windows XP. Only required changing the file names in
the build process, almost all the files are preceded
by "pcre_" now.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
As follow-up to tracker ID 2966602 (nested <div> matching) I would strongly appreciate if the internal PCRE lib would be updated to the latest version. I use WinXP and the suggestion offered (compile Privoxy yourself) seems to be out of my scope.
Moreover I did some further tests with <div> matching using regex with negative lookahead which caused Privoxy (the PCRE I assume) to crash.
This shows that the included PCRE lib on Windows is outdated regarding features and also buggy. It is nearly 11 years old!
Maybe another solution would be to _document_ the dynamic linking to a current PCRE lib on Windows in detail (if possible at all).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I suspect that the crashes were cause by PCRE running out of stack-space which is not a real PCRE bug and still possible with the latest version if the pattern causes a lot of back-tracking. Increasing the stack limits enforced by the operating system or configuring PCRE to use less stack-space should solve the problem.
Note that patterns with enough recursion to hit the default stack limit are usually performing poorly anyway, so rewriting them may be an even better solution.
I agree that documentation for dynamically linking Privoxy with PCRE on Windows would be useful and patches for this are always appreciated, just like the patches for the build system are.
At the moment my impression is that nobody cares enough about this to investigate how it's done, though, which explains why the default still hasn't changed.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have found great script to have updated user.action file https://gist.github.com/0x46616c6b/3585720 using frequently updated filters from adblock. It does fine job replacing adblock related language into privoxy accepted version. However for some reason and I suspect that privoxy might still use old PCRE version as this bug is still open not all those ^ that are replaced into [/&:?=_] work.
According to https://adblockplus.org/filters "Separator character is anything but a letter, a digit, or one of the following: _ - . %." I saw underscore as unaccepted separator there so I removed it, but it did not help and privoxy would still crash in the same filters
Now, script created huge number of lines including [/&:?=_] and going thout all of them on every start too use only those working one would be next to impossible so I simply do not use any of those lines including [/&:?=_] , but updating PCRE and probably fixing this issue at the same time would be a welcomed change.
Thanks!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Logged In: YES
user_id=249980
Moving to a developer to-do. Good idea, but it's also one
of those "if it ain't broke..." kinds of deals. I looked at
it a release or two ago of PCRE and it didn't look
insumountable.
Logged In: YES
user_id=249980
Uh, I guess that would be "not _insurmountable_."
Logged In: YES
user_id=322640
Yea, that's pretty old. Maybe after we get past the 3.0.5
release we should take a closer look at it.
Logged In: YES
user_id=163368
I had this running under the 3.0.3 code, was stable for me
under Windows XP. Only required changing the file names in
the build process, almost all the files are preceded
by "pcre_" now.
Logged In: YES
user_id=875547
Originator: NO
Another option would be to simply remove the outdated PCRE
code, forcing the user to dynamically link to PCRE.
On most platforms that's already done anyway
and we don't include zlib either.
As follow-up to tracker ID 2966602 (nested <div> matching) I would strongly appreciate if the internal PCRE lib would be updated to the latest version. I use WinXP and the suggestion offered (compile Privoxy yourself) seems to be out of my scope.
Moreover I did some further tests with <div> matching using regex with negative lookahead which caused Privoxy (the PCRE I assume) to crash.
This shows that the included PCRE lib on Windows is outdated regarding features and also buggy. It is nearly 11 years old!
Maybe another solution would be to _document_ the dynamic linking to a current PCRE lib on Windows in detail (if possible at all).
I suspect that the crashes were cause by PCRE running out of stack-space which is not a real PCRE bug and still possible with the latest version if the pattern causes a lot of back-tracking. Increasing the stack limits enforced by the operating system or configuring PCRE to use less stack-space should solve the problem.
Note that patterns with enough recursion to hit the default stack limit are usually performing poorly anyway, so rewriting them may be an even better solution.
I agree that documentation for dynamically linking Privoxy with PCRE on Windows would be useful and patches for this are always appreciated, just like the patches for the build system are.
At the moment my impression is that nobody cares enough about this to investigate how it's done, though, which explains why the default still hasn't changed.
I have found great script to have updated user.action file https://gist.github.com/0x46616c6b/3585720 using frequently updated filters from adblock. It does fine job replacing adblock related language into privoxy accepted version. However for some reason and I suspect that privoxy might still use old PCRE version as this bug is still open not all those ^ that are replaced into [/&:?=_] work.
According to https://adblockplus.org/filters "Separator character is anything but a letter, a digit, or one of the following: _ - . %." I saw underscore as unaccepted separator there so I removed it, but it did not help and privoxy would still crash in the same filters
eg. of log from failed filters
2018-04-20 11:01:54.392 7f6a7267e0c0 Error: error compiling &pltype=adhost[ from &pltype=adhost[/&:?=_]: unbalanced [] at offset 19
2018-04-20 11:01:54.392 7f6a7267e0c0 Fatal error: can't load actions file '/etc/privoxy/user2.action': line 65: cannot create URL or TAG pattern from: &pltype=adhost
2018-04-20 11:04:21.189 7fa8bc0570c0 Error: error compiling &pltype=adhost[ from &pltype=adhost[/&:?=_: unbalanced [] at offset 19
2018-04-20 11:04:21.190 7fa8bc0570c0 Fatal error: can't load actions file '/etc/privoxy/user2.action': line 65: cannot create URL or TAG pattern from: &pltype=adhost
2018-04-20 11:07:58.420 7f906fae50c0 Error: error compiling +advertorial. from +advertorial.: ? * + invalid at offset 0
2018-04-20 11:07:58.420 7f906fae50c0 Fatal error: can't load actions file '/etc/privoxy/user2.action': line 81: cannot create URL or TAG pattern from: +advertorial.
2018-04-20 11:08:32.412 7f2d6296c0c0 Error: error compiling +adverts from +adverts/: ? * + invalid at offset 0
2018-04-20 11:08:32.412 7f2d6296c0c0 Fatal error: can't load actions file '/etc/privoxy/user2.action': line 82: cannot create URL or TAG pattern from: +adverts
2018-04-20 11:09:20.079 7fb6fca880c0 Error: error compiling [ from [/&:?=_fp=.&prvtof=: unbalanced [] at offset 5
2018-04-20 11:09:20.079 7fb6fca880c0 Fatal error: can't load actions file '/etc/privoxy/user2.action': line 6565: cannot create URL or TAG pattern from:
2018-04-20 11:10:16.567 7f518d7e90c0 Error: error compiling [ from [/&:?=_mod=wms&do=view_.&zone=: unbalanced [] at offset 5
2018-04-20 11:10:16.567 7f518d7e90c0 Fatal error: can't load actions file '/etc/privoxy/user2.action': line 6566: cannot create URL or TAG pattern from:
2018-04-20 11:10:41.669 7fce5fa220c0 Error: error compiling [ from [/&:?=_pid=Ads[/&:?=_]: unbalanced [] at offset 5
2018-04-20 11:10:41.669 7fce5fa220c0 Fatal error: can't load actions file '/etc/privoxy/user2.action': line 6567: cannot create URL or TAG pattern from:
2018-04-20 11:11:10.327 7f39ae16d0c0 Error: error compiling WebAd[ from _WebAd[/&:?=: unbalanced [] at offset 11
2018-04-20 11:11:10.327 7f39ae16d0c0 Fatal error: can't load actions file '/etc/privoxy/user2.action': line 7118: cannot create URL or TAG pattern from: WebAd
2018-04-20 11:11:44.346 7fa630aeb0c0 Error: error compiling .ox-d..[ from .ox-d..[/&:?=_auid=: unbalanced [] at offset 14
2018-04-20 11:11:44.346 7fa630aeb0c0 Fatal error: can't load actions file '/etc/privoxy/user2.action': line 7130: cannot create URL or TAG pattern from: .ox-d..
2018-04-20 11:12:17.060 7f62737670c0 Error: error compiling .46.165.197.153[ from .46.165.197.153[/&:?=_: unbalanced [] at offset 23
2018-04-20 11:12:17.060 7f62737670c0 Fatal error: can't load actions file '/etc/privoxy/user2.action': line 7804: cannot create URL or TAG pattern from: .46.165.197.153
2018-04-20 11:13:03.948 7f4ddc1270c0 Error: error compiling .46.165.197.231[ from .46.165.197.231[/&:?=_: unbalanced [] at offset 23
2018-04-20 11:13:03.949 7f4ddc1270c0 Fatal error: can't load actions file '/etc/privoxy/user2.action': line 7805: cannot create URL or TAG pattern from: .46.165.197.231
2018-04-20 11:13:36.707 7fa7c63640c0 Error: error compiling .covertarget.com[ from .covertarget.com[/&:?=_...php: unbalanced [] at offset 22
2018-04-20 11:13:36.707 7fa7c63640c0 Fatal error: can't load actions file '/etc/privoxy/user2.action': line 7809: cannot create URL or TAG pattern from: .covertarget.com
2018-04-20 11:16:37.385 7fedce4060c0 Error: error compiling .metaffiliation.com[ from .metaffiliation.com[/&:?=_.[/&:?=_]maff=: unbalanced [] at offset 25
2018-04-20 11:16:37.385 7fedce4060c0 Fatal error: can't load actions file '/etc/privoxy/user2.action': line 7814: cannot create URL or TAG pattern from: .metaffiliation.com
2018-04-20 15:49:18.911 7fa9e80fb0c0 Error: error compiling &pltype=adhost[ from &pltype=adhost[/&:?=: unbalanced [] at offset 19
2018-04-20 15:49:18.911 7fa9e80fb0c0 Fatal error: can't load actions file '/etc/privoxy/user5.action': line 65: cannot create URL or TAG pattern from: &pltype=adhost[
Now, script created huge number of lines including [/&:?=_] and going thout all of them on every start too use only those working one would be next to impossible so I simply do not use any of those lines including [/&:?=_] , but updating PCRE and probably fixing this issue at the same time would be a welcomed change.
Thanks!
I think we can close this developer-todo, since it refers to the packaged pcre library, which was removed in 3.0.33.
I think we can close this developer-todo, since it refers to the packaged pcre library, which was removed in 3.0.33.