Menu
▾
▴
Re: [Pfspd-users@sf.net] performance problems
|
From: Rene v. d. V. <ren...@ph...> - 2006-06-09 11:00:27
|
<br><font size=3D2 face=3D"sans-serif">Hi!</font> <br> <br><font size=3D2 face=3D"sans-serif">Without being an expert (and without having looked at the code), I would expect you could use append mode while writing data and then open in update mode when the file is closed in order to correct the header (i.e. I would not update the header after each frame has been written, assuming that is what is currently happening).</font> <br> <br><font size=3D2 face=3D"sans-serif">Regards,</font> <br> <br><font size=3D2 face=3D"sans-serif">Ren=E9.</font> <br><font size=3D2 face=3D"sans-serif"><br> --------------------------------------------------------------------------<= br> Dr. Rene J. van der Vleuten<br> Philips Research Laboratories<br> High Tech Campus 36<br> 5656 AE Eindhoven<br> The Netherlands<br> <br> Phone: +31 40 2742941<br> Fax: +31 40 2742630<br> mailto:Ren...@ph...</font> <br> <br> <br> <table width=3D100%> <tr valign=3Dtop> <td width=3D33%> <br> <br> <br> <br> <br><font size=3D1 face=3D"sans-serif"><b>Bram Riemens</b> </font> <p><font size=3D1 face=3D"sans-serif">Sent by:</font> <br><font size=3D1 face=3D"sans-serif">pfs...@li...urceforg= e.net</font> <p><font size=3D1 face=3D"sans-serif">08-06-2006 17:06</font> <td width=3D66%> <table width=3D100%> <tr valign=3Dtop> <td> <div align=3Dright><font size=3D1 face=3D"sans-serif">To</font></div> <td><font size=3D1 face=3D"sans-serif">pfs...@li...</f= ont> <tr valign=3Dtop> <td> <div align=3Dright><font size=3D1 face=3D"sans-serif">cc</font></div> <td><font size=3D1 face=3D"sans-serif">Gerard Bloemen/EHV/RESEARCH/PHILIPS@= PHILIPS</font> <tr valign=3Dtop> <td> <div align=3Dright><font size=3D1 face=3D"sans-serif">Subject</font></div> <td><font size=3D1 face=3D"sans-serif">Re: [Pfs...@sf...] performance problems</font> <tr> <td> <div align=3Dright><font size=3D1 face=3D"sans-serif">Classification</font>= </div> <td></table> <br> <table> <tr valign=3Dtop> <td> <td></table> <div align=3Dright> <br></div></table> <br> <br> <br><font size=3D2 face=3D"sans-serif"><br> Rene, others,</font><font size=3D3> <br> </font><font size=3D2 face=3D"sans-serif"><br> After obtaining a test case from Rene, I did some investigations on the current development</font><font size=3D3> </font><font size=3D2 face=3D"san= s-serif"><br> repository of cpfspd and pts.</font><font size=3D3> </font><font size=3D2 f= ace=3D"sans-serif"><br> Test conditions:</font><font size=3D3> </font><font size=3D2 face=3D"sans-s= erif"><br> - linux executable, running on dimholt</font><font size=3D3> </font><font s= ize=3D2 face=3D"sans-serif"><br> - output to scratch disk on muscules file server</font><font size=3D3> </fo= nt><font size=3D2 face=3D"sans-serif"><br> - command "pts translate corvette=5F8mbps.m2v corvette=5F8mbps.yuv&quo= t;</font><font size=3D3> </font><font size=3D2 face=3D"sans-serif"><br> - result is file of 3.6Gbyte; 1198 frames 1920x1080</font><font size=3D3> <br> </font><font size=3D2 face=3D"sans-serif"><br> Currently (development version of cpfspd), file is opened in mode "wb&= quot; for allocation,</font><font size=3D3> </font><font size=3D2 face=3D"sans-se= rif"><br> then it is closed. With pts translate on mpeg, the file size is not known in advance,</font><font size=3D3> </font><font size=3D2 face=3D"sans-serif"= ><br> so it is initially created with size 22528 bytes.</font><font size=3D3> </f= ont><font size=3D2 face=3D"sans-serif"><br> After that, it is reopened in "rb+" mode. Then images are written into the file.</font><font size=3D3> </font><font size=3D2 face=3D"sans-ser= if"><br> Result of "time":</font><font size=3D3> </font><font size=3D2 fac= e=3D"sans-serif"><br> real 18m11.415s</font><font size=3D3> </font><font size=3D2 fa= ce=3D"sans-serif"><br> user 1m10.260s</font><font size=3D3> </font><font size=3D2 fac= e=3D"sans-serif"><br> sys 1m2.590s</font><font size=3D3> <br> </font><font size=3D2 face=3D"sans-serif"><br> As a special test, I forced opening the file in "wb" mode the 2nd time; result:</font><font size=3D3> </font><font size=3D2 face=3D"sans-= serif"><br> real 2m23.710s</font><font size=3D3> </font><font size=3D2 fac= e=3D"sans-serif"><br> user 1m5.710s</font><font size=3D3> </font><font size=3D2 face= =3D"sans-serif"><br> sys 0m20.850s</font><font size=3D3> <br> </font><font size=3D2 face=3D"sans-serif"><br> Next test; force mode "wb+" is also very slow.</font><font size= =3D3> <br> </font><font size=3D2 face=3D"sans-serif"><br> I tested opening in append modes (followed by a reset of the file pointer), but that fails to give correct output.</font><font size=3D3> <br> <br> <br> </font><font size=3D2 face=3D"sans-serif"><br> So, just changing the fopen mode results in a performance penalty of a factor 7.5...</font><font size=3D3> </font><font size=3D2 face=3D"sans-seri= f"><br> Interesting observation...</font><font size=3D3> <br> <br> </font><font size=3D2 face=3D"sans-serif"><br> Harold, any suggestion how to keep the file open, and still maintain a simple interface</font><font size=3D3> </font><font size=3D2 face=3D"sans-s= erif"><br> for both *nix and win32/cygwin?</font><font size=3D3> <br> <br> </font><font size=3D2 face=3D"sans-serif"><br> Regards, Bram.</font><font size=3D3> <br> </font><font size=3D2 face=3D"sans-serif"><br> PS. Gerard, maybe this is related to your observations?</font><font size=3D= 3> </font><font size=3D2 face=3D"sans-serif"><br> What version did you use?</font><font size=3D3> </font><font size=3D2 face=3D"sans-serif"><br> You can subscribe this mailing list at</font><f= ont size=3D3> </font><font size=3D2 face=3D"sans-serif"><br> https://lists.sourc= eforge.net/lists/listinfo/pfspd-users<br> <br> --<br> A.K. (Bram) Riemens<br> Principal Scientist, DSP group, Philips Research<br> Office: WO-p-94, Postbox WO02<br> High Tech Campus 36 (WO), 5656 AE Eindhoven, The Netherlands<br> Tel: +31 40 27 43833, Fax: +31 40 27 44675<br> E-mail: bra...@ph...</font><font size=3D3> <br> <br> </font> <table width=3D100%> <tr valign=3Dtop> <td width=3D43%><font size=3D3><br> <br> <br> <br> </font><font size=3D1 face=3D"sans-serif"><b><br> Rene van der Vleuten</b> </font> <p><font size=3D1 face=3D"sans-serif">Sent by:</font><font size=3D3> </font= ><font size=3D1 face=3D"sans-serif"><br> pfs...@li...</font><font size=3D3> </font> <p><font size=3D1 face=3D"sans-serif">08-06-2006 13:04</font><font size=3D3> </font> <td width=3D56%> <br> <table width=3D100%> <tr valign=3Dtop> <td width=3D23%> <div align=3Dright><font size=3D1 face=3D"sans-serif">To</font></div> <td width=3D76%><font size=3D1 face=3D"sans-serif">pfs...@li...urce= forge.net</font><font size=3D3> </font> <tr valign=3Dtop> <td> <div align=3Dright><font size=3D1 face=3D"sans-serif">cc</font></div> <td> <tr valign=3Dtop> <td> <div align=3Dright><font size=3D1 face=3D"sans-serif">Subject</font></div> <td><font size=3D1 face=3D"sans-serif">[Pfs...@sf...] performance pro= blems</font><font size=3D3> </font> <tr> <td> <div align=3Dright><font size=3D1 face=3D"sans-serif">Classification</font>= </div> <td></table> <br> <br> <table width=3D100%> <tr valign=3Dtop> <td width=3D49%> <td width=3D50%></table> <div align=3Dright> <br></div></table> <br><font size=3D3><br> <br> </font><font size=3D2 face=3D"sans-serif"><br> <br> Dear Bram, Harold,</font><font size=3D3> </font><font size=3D2 face=3D"sans= -serif"><br> <br> I am suffering from huge performance problems with "pts translate file.m2v file.yuv"; processing takes more than 10x longer than with the latest released pts!! I suspect these are caused by your latest updates to cpfspd (since I did not see any obviously related changes in pts).</font= ><font size=3D3> </font><font size=3D2 face=3D"sans-serif"><br> <br> Could you please look into this?</font><font size=3D3> </font><font size=3D= 2 face=3D"sans-serif"><br> <br> Thanks!</font><font size=3D3> </font><font size=3D2 face=3D"sans-serif"><br> <br> Ren=E9 <br> <br> --------------------------------------------------------------------------<= br> Dr. Rene J. van der Vleuten<br> Philips Research Laboratories<br> High Tech Campus 36<br> 5656 AE Eindhoven<br> The Netherlands<br> <br> Phone: +31 40 2742941<br> Fax: +31 40 2742630<br> mailto:Ren...@ph...</font><font size=3D3> </font><font = size=3D2><tt>=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F<br> Pfspd-users mailing list<br> Pfs...@li...<br> https://lists.sourceforge.net/lists/listinfo/pfspd-users</tt></font><font s= ize=3D3><br> </font><font size=3D2><tt>=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F<br> Pfspd-users mailing list<br> Pfs...@li...<br> https://lists.sourceforge.net/lists/listinfo/pfspd-users<br> </tt></font> <br> |
