phpwiki-talk

[Phpwiki-talk] Problem with big archives

[Sébastien Le C. <sl...@ya...>]

Hi,


I have been looking into problems with some dumped archives which could 
not be imported back into phpwiki.

After some investigation, I have isolated the problem to 
ParseMimeMultipart in ziplib.php; the preg_match was exiting too early. 
  The regexp looks ok, but for big archives (whether it is a medium-size 
page with lots of revisions, or a huge file with a couple of revisions), 
preg_match seems to be failing.

It turns out it looks like I'm hitting the pcre backtrack/recursion 
limit.  Increasing the values of pcre.backtrack_limit and 
pcre.recursion_limit in php.ini as per 
http://ie.php.net/manual/en/pcre.configuration.php#ini.pcre.recursion-limit 
seems to do the trick, but I was wondering if anybody has hit this issue 
before?  I have quite large archives to import, and I am not sure 
whether preg_match is able to cope with them, even with the increased 
values.


Thanks,
Sébastien.

Re: [Phpwiki-talk] Problem with big archives

[Sébastien Le C. <sl...@ya...>]

I've finally been able to import all my pages!!

The last hurdle was that I was trying to export iso-8859-1 pages and 
import them into a utf-8 phpwiki setup: all the pages with non-ASCII 
characters were not being imported because the pagename was not 
converted into utf-8 after the urldecode when importing.  A quick change 
to force the pagename to utf-8, and that's it!

One remaining question, though (probably a FAQ, actually): is there a 
way of not importing the PhpWiki “core” pages?  When exporting, I 
usually do a full dump, which contains PhpWikiAdministration and all the 
Help pages, etc. but I don't to import them back, as I may be losing the 
changes done to these pages as part of the upgrade.

Any obvious way of achieving this?

Thanks,
Seb



On 14/05/2010 08:22, Sébastien Le Callonnec wrote:
> Hi,
>
>
> I have been looking into problems with some dumped archives which could
> not be imported back into phpwiki.
>
> After some investigation, I have isolated the problem to
> ParseMimeMultipart in ziplib.php; the preg_match was exiting too early.
>    The regexp looks ok, but for big archives (whether it is a medium-size
> page with lots of revisions, or a huge file with a couple of revisions),
> preg_match seems to be failing.
>
> It turns out it looks like I'm hitting the pcre backtrack/recursion
> limit.  Increasing the values of pcre.backtrack_limit and
> pcre.recursion_limit in php.ini as per
> http://ie.php.net/manual/en/pcre.configuration.php#ini.pcre.recursion-limit
> seems to do the trick, but I was wondering if anybody has hit this issue
> before?  I have quite large archives to import, and I am not sure
> whether preg_match is able to cope with them, even with the increased
> values.
>
>
> Thanks,
> Sébastien.
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Phpwiki-talk mailing list
> Php...@li...
> https://lists.sourceforge.net/lists/listinfo/phpwiki-talk
>

Re: [Phpwiki-talk] Problem with big archives

[Manuel V. <man...@gm...>]

On Mon, May 17, 2010 at 8:53 PM, Sébastien Le Callonnec <sl...@ya...> wrote:
> I've finally been able to import all my pages!!
>
> The last hurdle was that I was trying to export iso-8859-1 pages and
> import them into a utf-8 phpwiki setup: all the pages with non-ASCII
> characters were not being imported because the pagename was not
> converted into utf-8 after the urldecode when importing.  A quick change
> to force the pagename to utf-8, and that's it!
>
> One remaining question, though (probably a FAQ, actually): is there a
> way of not importing the PhpWiki “core” pages?  When exporting, I
> usually do a full dump, which contains PhpWikiAdministration and all the
> Help pages, etc. but I don't to import them back, as I may be losing the
> changes done to these pages as part of the upgrade.
>
> Any obvious way of achieving this?

Hi Sebastien,

To handle this case we had to develop a script that remove from within
this archive all pages with "The PhpWiki team" as author and without
modification (and maybe pages that are only modified by "The PhpWiki
team").

I don't have it right now but I could had a look in my archives sometime,
Manuel

Re: [Phpwiki-talk] Problem with big archives

[Sébastien Le C. <sl...@ya...>]

On 18/05/2010 11:24, Manuel Vacelet wrote:
> On Mon, May 17, 2010 at 8:53 PM, Sébastien Le Callonnec<sl...@ya...>  wrote:
>> One remaining question, though (probably a FAQ, actually): is there a
>> way of not importing the PhpWiki “core” pages?  When exporting, I
>> usually do a full dump, which contains PhpWikiAdministration and all the
>> Help pages, etc. but I don't to import them back, as I may be losing the
>> changes done to these pages as part of the upgrade.
>>
>> Any obvious way of achieving this?
>
> Hi Sebastien,
>
> To handle this case we had to develop a script that remove from within
> this archive all pages with "The PhpWiki team" as author and without
> modification (and maybe pages that are only modified by "The PhpWiki
> team").
>
> I don't have it right now but I could had a look in my archives sometime,
> Manuel
>

Hi Manuel,


Good one, this does the trick indeed.  There are still a few “Help” 
files left behind, so I did the following:

rm -f Help*
grep -l "author=The%20PhpWiki%20programming%20team" * | xargs rm -f

Thanks a mil,
Sébastien.