phpwiki-talk

[Phpwiki-talk] Some memory tests

[Reini U. <ru...@x-...>]

http://phpwiki.sourceforge.net/phpwiki/PhpMemoryExhausted/Testresults
-- 
Reini Urban
http://xarch.tu-graz.ac.at/home/rurban/

Re: [Phpwiki-talk] Some memory tests

[Reini U. <ru...@x-...>]

Reini Urban schrieb:
> http://phpwiki.sourceforge.net/phpwiki/PhpMemoryExhausted/Testresults

Ok, backporting the unittest suite to 1.3.4 found the memory hog:

// test	                nr      USECACHE (t/f)
beforePEAR		1	6644	6640
beforePhpWiki		2	6872	6868
PhpWikiLoaded		3	9064	9064
PhpWikiInitialized	4	9420	9432
testimportoldmarkup	5	10388	10380
testsetupwiki		6	11996	11976
testallpages		7	12200	12172
test02orphanedpages	8	12208	12176
test03textformattingru	9	12208	12176
test99dumphtml		10	12208	12180

// PARAMS:
// PHPWIKI_VERSION=1.3.4
// PHP_SAPI=cgi
// PHP_OS=WINNT
// PHP_VERSION=4.1.2
// test=SetupWiki,AllPagesTest,DumpHtml
// db=dba
// debug=9
// level=2
// pid=1756
// USE_DB_SESSION=false
// CREATED=Thu Oct 21 02:58:52 GMT 2004

against:

// test        nr      USECACHE (t/f)  WIKIDB_NCM(t/f) USERNEW (t/f) 
beforePEAR     1       6636    6636    6636    6636    6636    6636 
beforePhpWiki  2       6864    6864    6880    6880    6876    6876 
PhpWikiLoaded  3       11924   11920   11932   11932   11912   11112 
PhpWikiInitial 4       12376   12364   12368   12368   12376   11544 
testimportoldm 5       14968   14928   14952   14964   14948   14120 
testsetupwiki  6       15264   15104   15240   15260   15248   14424 
test01rateit   7       16496   16196   16504   16508   16480   15672 
test02orphaned 8       16540   16232   16544   16556   16548   15716 
test03oldtextf 9       16592   16256   16540   16564   16568   15728 
test99dumphtml 10      32972   32644   33060   33108   -1      -1

// PARAMS:
// PHPWIKI_VERSION=1.3.11pre-20041020
// PHP_SAPI=cgi
// PHP_OS=WINNT
// PHP_VERSION=4.1.2
// test=SetupWiki,DumpHtml
// db=dba
// debug=9
// level=2
// CREATED=Wed Oct 20 21:16:52 GMT 2004


What do you see?
Well, that phpwiki-1.3.11 is 5068KB large and phpwiki-1.3.4 only 2196KB.
During huge runs (SetupWiki,AllPages) there's a 4MB difference.

But 1.3.4 needed no additional memory at all for our biggest memory hog, 
dumphtml.
Why? clearly because of our new output buffering.
1.3.4 uses no buffering, at 1.3.7 buffering was introduced.
And any ob_clean() doesn't release any memory!
These are the 4.1.2 results, the sf.net platform.
4.3.9 is only 4MB better at dumphtml. (12MB <=> 16MB)

Yes, I've already tried:
       if (!empty($this->_is_buffering_output)) {
           while (@ob_end_flush());
           ob_clean();
           ob_start();
       }
dumphtml always need 16MB on the default pageset, even with ob restarts 
in the loop.

So it looks like we have to turn this ob buffering off.
At least add a constant to do without, so that it can work on servers 
with memory-limit = 8MB.
At least I've found the real culprit. no stupid refs or object tricks.

And I've found tons of otherwise undetected bugs.
-- 
Reini Urban
http://xarch.tu-graz.ac.at/home/rurban/

Re: [Phpwiki-talk] Some memory tests

[Reini U. <ru...@x-...>]

Oliver Betz schrieb:
> Reini Urban <ru...@x-...> wrote:
> 
>>What do you see?
>>Well, that phpwiki-1.3.11 is 5068KB large and phpwiki-1.3.4 only 2196KB.

WikiUserNew (with groups and perms) costs about 600kb. I'll try now to 
seperate the possibly unneeded parts into seperate (optionally loaded) 
modules. Maybe this will bring some more 300kb.

But turning off ob_buffers will bring 16MB!

> BTW: also the needed disk space for a PhpWiki installation increased 
> rapidly (6.5MB for 1.3.10 IIRC), which will limit PhpWiki's usability 
> on small hosting services.

Most of the new stuff is optional. ADODB is much larger now.
Much more internal docs: pgsrc pages.
Someone (me) should write a wiki page, what files pages can be safely 
deleted, under certain conditions.

>>At least I've found the real culprit. no stupid refs or object tricks.
> great!
> 
>>And I've found tons of otherwise undetected bugs.
> 
> critical?

Just loading and dumping errored. And only with rare combinations, 
otherwise it shoud have got detected before.
   plugincached, wikiuser, httpclient, wikicallback.
But no apache/php crash so far. (php 4.1.2 + 4.3.9)

BTW: With the new unittests for 1.3.4 I got a lot of crashes on 1.3.4,
so don't say it got worse.
-- 
Reini Urban
http://phpwiki.org/

Re: [Phpwiki-talk] Some memory tests

[Dan F. <dfr...@cs...>]

Reini,

I am watching this with happiness. Reproducible tests that beat on the 
system mean fewer bugs. That's a good thing.

Dan

Reini Urban wrote:

> Oliver Betz schrieb:
>
>> Reini Urban <ru...@x-...> wrote:
>>
>>> What do you see?
>>> Well, that phpwiki-1.3.11 is 5068KB large and phpwiki-1.3.4 only 
>>> 2196KB.
>>
>
> WikiUserNew (with groups and perms) costs about 600kb. I'll try now to 
> seperate the possibly unneeded parts into seperate (optionally loaded) 
> modules. Maybe this will bring some more 300kb.
>
> But turning off ob_buffers will bring 16MB!
>
>> BTW: also the needed disk space for a PhpWiki installation increased 
>> rapidly (6.5MB for 1.3.10 IIRC), which will limit PhpWiki's usability 
>> on small hosting services.
>
>
> Most of the new stuff is optional. ADODB is much larger now.
> Much more internal docs: pgsrc pages.
> Someone (me) should write a wiki page, what files pages can be safely 
> deleted, under certain conditions.
>
>>> At least I've found the real culprit. no stupid refs or object tricks.
>>
>> great!
>>
>>> And I've found tons of otherwise undetected bugs.
>>
>>
>> critical?
>
>
> Just loading and dumping errored. And only with rare combinations, 
> otherwise it shoud have got detected before.
>   plugincached, wikiuser, httpclient, wikicallback.
> But no apache/php crash so far. (php 4.1.2 + 4.3.9)
>
> BTW: With the new unittests for 1.3.4 I got a lot of crashes on 1.3.4,
> so don't say it got worse.