pfspd-users — For pfspd users: support, suggestions, announcments, etc.

[Pfspd-users@sf.net] Std I/O redirection and pipes now supported on

[Bram R. <bra...@nx...>]

Dear all,

Erik Kathman provided a solution to enable the use of stdio and pipes
in a pure windows environment.

This has been put in the cpfspd development version.

>From the Readme file:
- Use of standard I/O for the win32 target implemented.
  Up till now, the use of stdio (and pipes) was not working on the win3=
2
  build. This was caused by the standard windows treatment of text file=
s.
  In the windows environent the standard input or standard output are
  opened as text files and therefore subject to end-of-line substitutio=
n.
  I.e. at input, \cr\lf is substituted by \cr and at output,
  a \cr is substituted by \cr\lf. The cpfspd_low module contains
  platform dependent settings to solve this.
  Also the test script is adapted to incorporate testing of
  stdio when selection the win32 target.
  Thanks to Erik Kathman for providing the solution.

Regards, Bram.
--
Bram Riemens, Senior Principal
NXP Semiconductors / Corporate I&T / Research
Email: bra...@nx...
<--- New address and phone since June 2007 -->
Phone: +31 40 27 25910; Fax: +31 40 27 44639
High Tech Campus 32 (floor 1, office 140)
5656 AE Eindhoven, The Netherlands

The information contained in this message is confidential and may be
legally privileged. The message is intended solely for the addressee(s)=
. If
you are not the intended recipient, you are hereby notified that any us=
e,
dissemination, or reproduction is strictly prohibited and may be unlawf=
ul.
If you are not the intended recipient, please contact the sender by ret=
urn
e-mail and destroy all copies of the original message.=

[Pfspd-users@sf.net] MTK and CPFSPD release

[Bram R. <bra...@nx...>]

Dear all,

New releases are available:
MTK version 5.3.2
CPFSPD version 1.10.2

The releases are installed at their regular location within NXP in
eindhoven:
/home/nlv21431/utils/...
\\wbi.nxp.com\htchome\nlv21431\utils\...

They are available for download from http://sourceforge.net/projects/pf=
spd



=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
CPFSPD release notes
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

Done in release 1.10.2 (dd. Dec. 18, 2007)
------------------------------------------

Highlights
      - Some bugfixes,
      - When writing a file header, the whole file is now allocated
        on disk in a very efficient way. Therefore, we recommend
        application to open the file with the correct number of frames.=

        This results in 1) less file fragmentation on NTFS and
        2) performance improvement on network disks and win32/cygwin
        local disks.
      - Performance improvement of >8 bit files.

Bug fixes
- Application segmentation fault on Linux.
  Following calling sequence failed at the p_read_header() routine:
      p_create_header(&out_header, P_COLOR_444_PL, P_50HZ);
      p_write_header("test.yuv", &out_header);
      p_close_file("test.yuv");
      p_read_header("test.yuv", &out_header);
  More general: the program may crash when accessing a closed file
  in a different mode than it was previously used.
  Underlaying cause: when accessing a file in a different mode,
  it is first closed, then opened in the appropriate mode.
  This test was incomplete: erreonously, a closed file is closed
  again, effectively calling fclose(NULL). This is implementation
  specific; some platforms ignore fclose(NULL).
- Bug #1506617: Growing network file with large chunk on win fails.
  Situation: application executing on win32/cygwin, writing a file
  on a network share; a frame is written a couple of Gbytes beyond
  the higest written location in the file. The operation failes
  with a timeout error.
  A workaround is implemented, catching the timeout error and
  reissuing the failed operation. The workaround is transparant
  to the application.
- In the previous release, only writing files in big-endian
  mode was supported. So, executing p_rewrite_header()
  on a little-endian file was not supported. Unfortunately
  cpfspd failed to signal this condition as an error.
  Instead, the operation was carried out, silently modifying
  the endian bit in the header to big-endian. This resulted
  in incorrect data interpretation of files containing components
  > 8 bits. This problem is resolved by supporting the creation
  and modification of both big- and little-endian files.

Functionality
- The display aspect ratio setting can now have value P_AS_WH,
  indicating that the intended aspect ratio of the diplay
  is equal to the width/height ratio in pixels. This value
  is an indication of square pixels as long as the width/height
  ratio remains the same.
- Function p_create_ext_header() can now be called with
  value P_UNKNOWN_ASPECT_RATIO in the ratio parameter.
  This causes an automatic 16:9/4:3 selection.
  See cpfspd.h for details on the selection algorithm.

Miscellaneous
- Module cpfspd_fio.c uses assembly code for cpu type checks
  and speed optimization that is now also activated on cygwin
  (formerly, this was only possible on win32).
- Files written on the windows (win32 or cygwin) platforms
  contained unused space at the end due to the unbuffered
  file I/O. As a consequence, the file length depended on
  the platform. This is no longer the case. On win32/cygwin,
  the file is truncated to the proper length.
- Due to best-fit allocation strategy of NTFS, pfspd files could become=

  severely fragmented, which interferes with playback performance.
  This is fixed in p_write_hdr() by seeking to the required file size,
  then writing a few bytes. NTFS now allocates a large extent,
  so fragmentation is not a problem (unless the disk is almost full).
  As a side benefit, performance on network disks is also improved.
  Further, write performance on win32/cygwin takes full advantage
  of asynchronous file I/O if the file is allocated in advance.
- Cpfspd will write 10-16 bit files according to the system endianness.=

  Prevously, all files were written with bit-endian data organization.
  This allows optimization of the 16-bit read/write functions.
- The performance to access >8 bit files is improved.
  In the cpfspd_low module, a copy operation to an intermediate buffer
  is avoided if both:
  + memory format is equal to the file format
  + system endianness is equal to the file endianness
  (for 8 bit files, this optimization was already available).
- Any output parameter of p_get_comp_2() may be NULL.
  In case you need only a subset of the values (so you do not need
  to allocate an output variable).
- New errors are introduced to catch header values that cannot be
  represented in the pfspd file format: P_TOO_MANY_IMAGES,
  P_ILLEGAL_INTERLACE, P_ILLEGAL_COMP_SIZE, P_ILLEGAL_LIN_SBSMPL,
  P_ILLEGAL_PIX_SBSMPL, P_ILLEGAL_PHSHFT.
  In order to support the new errors, all defines related to
  header formatting are moved from cpfspd_low.c to cpfspd.h:
  P_SNR_IMAGES, P_SNR_COMPON, P_SNR_FD_RECS, P_SNR_AUXDAT_RECS,
  P_SBYTES_REC, P_SENDIAN_CODE, P_SAUX_HDR_RECS, P_LEN_GLOB_STR,
  P_SIMA_FREQ, P_SLIN_FREQ, P_SPIX_FREQ, P_SACT_LINES, P_SACT_PIXEL,
  P_SINTERLACE, P_SH_PP_SIZE, P_SV_PP_SIZE, P_LEN_GLOB_ATT,
  P_NUM_GLOB_RECS, P_SLIN_IMAGE, P_SPIX_LINE, P_LEN_COMP_STR,
  P_STEM_SBSMPL, P_SLIN_SBSMPL, P_SPIX_SBSMPL, P_LEN_COMP_ATT,
  P_STEM_PHSHFT, P_SLIN_PHSHFT, P_SPIX_PHSHFT.
- Always use large file I/O when compiling with Visual C++.
  It appears that many users compile the cpfspd library by simply inclu=
ding
  the source code in their VC++ project, without defining FIO_WIN32_FIL=
E in
  the project properties. Hence, no large file support was compiled.
  The FIO_WIN32_FILE define is now checked when __MSVC__ is defined.
- Speed test program uses HD image size (not SD anymore) as default.
- The speed test program puts the proper number of frames in the
  header before writing the header to disk. This triggers the
  new feature of fast disk allocation.



=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
MTK release notes
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

Done in release 5.3.2 (dd. Dec. 18, 2007)
-----------------------------------------

Highlights
        MTK: Support of compiler specific flags.
        MTK: Support for MS Visual Studio version 8.
        MTK: Simplification to avoid use of recursive make for dependen=
cy
build.
        MTK: New TM cores tm3271 and tm3182.
        Setup: new site nxpehv
        Misc: Modules magic and parse_options under LGPL licence.

Bug fixes
- The perl script to modify the tmsim/ttisim stat file for use by
  proview did not work on cygwin, due to line-end and '\n' issues.
  This has been resolved.

Functionality
- Microsoft Visual Studio version 8 is supported.
  The makefile system automatically determines which version
  is installed. Currently version 7 (.NET) and version 8 are recognized=
.
  Depending on the version, proper debug and optimization options
  are set.
- New settings in Makefile: DFLAGS_UNIX and DFLAGS_WINDOWS
  Environment specific DFLAGS used to overrule both the generic
  DFLAGS value and the MTK default flags.
  DFLAGS_WINDOWS is used when building on cygwin
  DFLAGS_UNIX is used when building on other platforms
- New settings in Makefile: CFLAGS_<COMP>, CXXFLAGS_<COMP>,
  LFLAGS_<COMP> and CLFLAGS_<COMP>
  Compiler specific values used to overrule both the generic
  values CFLAGS, CXXFLAGS, LFLAGS, CLFLAGS, and the MTK default
  flags.
  <COMP> is GCC, ICC, MSVC, SGI or TM.
  Use of these compiler specific variables is preferred over the
  use of the generic variables in view of the differences of
  available options and syntax.
- New settings in Makefile: CFLAGS2_<COMP>, CXXFLAGS2_<COMP>
  LFLAGS2_<COMP> and CLFLAGS2_<COMP>.
  Compiler specific flags appended to the MTK defaults.
  <COMP> is GCC, ICC, MSVC, SGI or TM.
  Use of these compiler specific variables is preferred over the
  use of the generic variables in view of the differences of
  available options and syntax.
- Added LINK_CPP setting. Use LINK_CPP=3D1 in the Makefile
  to allow applications only using C sources with libraries
  that contains C++ sources. This flag forces the choice
  of the link command.
- When using the Trimedia compiler, C++ sources are now compiled with
  the tmc++ command, in a similar fashion as the gcc/g++ distinction.
  Similarly, the linker command is adaptive when C++ sources are part
  of the application. Up till now, mtk did not support C++ compilation
  with the TriMedia compiler.
- The top level make dependency of "all" is changed. Now $(TARGET) is
  mentioned explicitely as file depencency, not just the phony target
"all".
  This allows an application Makefile to use $(TARGET) in further
  depencencies to e.g. execute a program automatically to generate an
  output file.
- New Makefile variable is available: EXTRA_CLEAN. This specifies a lis=
t
  of base directories that are added to the cleanall and mrproper comma=
nds.
  For cleanall, only platform specific subdirectories are removed;
  for mrproper, the entire base directory is removed.
- Trimedia cores tm3271 and tm3182 are now supported.
- The TCS_DIR setting in Make.env may direct to the full path of the
  trimedia comiler installation - similar to the commonly known TCS
setting.
  This is convenient in case trimedia builds are only performed on the
  windows platform.
- The name of the readme file generated when building a release
distribution
  kit is changed from <name>.readme to <name>.txt to simplify use of th=
e
file
  on windows systems and websites.

Miscellaneous
- The licence conditions of the parse_options module changed from GPL t=
o
LGPL.
- The licence conditions of the magic module changed from GPL to LGPL.
- Default TCS release at natlab updated to version 5.1.
- The TMSIMFLAGS variable in the sample Makefile is set to
  "-v -memorydelay 30 -clockfrequency 300000000 -clockratio 2 1 1
-memorysize 134217728"
  to override the "-v" default of Make.mtk. This serves as an applicati=
on
  example.
- For TriMedia compiler, the *.i files are moved from the src dir
  to the object dir, similar to the *.t and *.s files.
- Output of "gmake dump" is restructured to distingush environment
settings,
  makefile settings and mtk internal variables.
- The perl code in the invoke script prog.sh is adapted to comply
  to "good coding habits" by applying "use strict" and "use warnings".
- On cygwin, the file name of the executable in the $(TARGET)
  variable has the .exe extension. This way, the $(TARGET) value
  is consistent with the actual generated filename. Background:
  on cygwin/windows, the gcc compiler generates a xyz.exe file
  even if the .exe extension is not passed to the linker.
- In all mtk releases up till now, the main make invokation
  took care of depencency generation, then a sub make process
  was spawned to perform the actual build. This mechanism
  is changed such that the sub make is not required anymore.
  User impact should be minimal. One consequence is that the
  definition of "INCDEPFILE=3D" is not required anymore to invoke
  a sub-make from the main Makefile. This mechanism is commonly
  used, but it was never well documented. Now, mtk integration
  in large projects and makefile behavior is more straightforward.
- The msvc commands now contain the full pathname, to avoid that their
  use depends on the order of the command search path in $PATH.
  Especially there may be conflicts with the "link" command.
  A similar approach is followed for the commands of the trimedia
  compiler.



Regards, Bram.
--
Bram Riemens, Senior Principal
NXP Semiconductors / Corporate I&T / Research
Email: bra...@nx...
<--- New address and phone since June 2007 -->
Phone: +31 40 27 25910; Fax: +31 40 27 44639
High Tech Campus 32 (floor 1, office 140)
5656 AE Eindhoven, The Netherlands

The information contained in this message is confidential and may be
legally privileged. The message is intended solely for the addressee(s)=
. If
you are not the intended recipient, you are hereby notified that any us=
e,
dissemination, or reproduction is strictly prohibited and may be unlawf=
ul.
If you are not the intended recipient, please contact the sender by ret=
urn
e-mail and destroy all copies of the original message.=

[Pfspd-users@sf.net] Make.mtk - recursion removed

[Bram R. <bra...@nx...>]

Hi all,

There was still a subtile mistake in the update of make.mtk yesterday.
As a result, I digged into the make/submake design to handle the automa=
tic
depencency generation.
Now, the sub-make is entirely removed and we rely on gnu make features =
of
rebuilding makefiles.

>From the readme file:
- In all mtk releases up till now, the main make invokation
  took care of depencency generation, then a sub make process
  was spawned to perform the actual build. This mechanism
  is changed such that the sub make is not required anymore.
  User impact should be minimal. One consequence is that the
  definition of "INCDEPFILE=3D" is not required anymore to invoke
  a sub-make from the main Makefile. This mechanism is commonly
  used, but it was never well documented. Now, mtk integration
  in large projects and makefile behavior is more straightforward.

The change is in the development repository.
The new Make.mtk is also used now in pp and pv.
Volunteers for testing in other projects are welcome.

Regards, Bram.
--
Bram Riemens, Senior Principal
NXP Semiconductors / Corporate I&T / Research
Email: bra...@nx...
<--- New address and phone since June 2007 -->
Phone: +31 40 27 25910; Fax: +31 40 27 44639
High Tech Campus 32 (floor 1, office 140)
5656 AE Eindhoven, The Netherlands

The information contained in this message is confidential and may be
legally privileged. The message is intended solely for the addressee(s)=
. If
you are not the intended recipient, you are hereby notified that any us=
e,
dissemination, or reproduction is strictly prohibited and may be unlawf=
ul.
If you are not the intended recipient, please contact the sender by ret=
urn
e-mail and destroy all copies of the original message.=

[Pfspd-users@sf.net] Make.mtk updates

[Bram R. <bra...@nx...>]

Hi all,

Some updates in Make.mtk to incorporate some improvements contributed b=
y
Koen Meinds and O P Gangwal.

Functionality
- The top level make dependency of "all" is changed. Now $(TARGET) is
  mentioned explicitely as file depencency, not just the phony target
"all".
  This allows an application Makefile to use $(TARGET) in further
  depencencies to e.g. execute a program automatically to generate an
  output file.
- New Makefile variable is available: EXTRA_CLEAN. This specifies a lis=
t
  of base directories that are added to the cleanall and mrproper comma=
nds.
  For cleanall, only platform specific subdirectories are removed;
  for mrproper, the entire base directory is removed.

Miscellaneous
- On cygwin, the file name of the executable in the $(TARGET)
  variable has the .exe extension. This way, the $(TARGET) value
  is consistent with the actual generated filename. Background:
  on cygwin/windows, the gcc compiler generates a xyz.exe file
  even if the .exe extension is not passed to the linker.

The changes are in the development repository.
Developers, please update your sandboxes.

Regards, Bram.
--
Bram Riemens, Senior Principal
NXP Semiconductors / Corporate I&T / Research
Email: bra...@nx...
<--- New address and phone since June 2007 -->
Phone: +31 40 27 25910; Fax: +31 40 27 44639
High Tech Campus 32 (floor 1, office 140)
5656 AE Eindhoven, The Netherlands

The information contained in this message is confidential and may be
legally privileged. The message is intended solely for the addressee(s)=
. If
you are not the intended recipient, you are hereby notified that any us=
e,
dissemination, or reproduction is strictly prohibited and may be unlawf=
ul.
If you are not the intended recipient, please contact the sender by ret=
urn
e-mail and destroy all copies of the original message.=

[Pfspd-users@sf.net] Cpfspd code update

[Bram R. <bra...@nx...>]

Dear pfspd community,

Recently Reinout Verburgh experienced a serious application crash when
trying to use extra components with subsample factors of 240x135.
We found that this is simply not supported by the pfspd file format,
as only two decimal digits are available to store the subsample factors=
.
However, good error checking should have prevented the crash
and simplify debugging.

For this reason, new errors are introduced as stated in the
following fragment of the readme file:

- New errors are introduced to catch header values that cannot be
  represented in the pfspd file format: P_TOO_MANY_IMAGES,
  P_ILLEGAL_INTERLACE, P_ILLEGAL_COMP_SIZE, P_ILLEGAL_LIN_SBSMPL,
  P_ILLEGAL_PIX_SBSMPL, P_ILLEGAL_PHSHFT.
  In order to support the new errors, all defines related to
  header formatting are moved from cpfspd_low.c to cpfspd.h:
  P_SNR_IMAGES, P_SNR_COMPON, P_SNR_FD_RECS, P_SNR_AUXDAT_RECS,
  P_SBYTES_REC, P_SENDIAN_CODE, P_SAUX_HDR_RECS, P_LEN_GLOB_STR,
  P_SIMA_FREQ, P_SLIN_FREQ, P_SPIX_FREQ, P_SACT_LINES, P_SACT_PIXEL,
  P_SINTERLACE, P_SH_PP_SIZE, P_SV_PP_SIZE, P_LEN_GLOB_ATT,
  P_NUM_GLOB_RECS, P_SLIN_IMAGE, P_SPIX_LINE, P_LEN_COMP_STR,
  P_STEM_SBSMPL, P_SLIN_SBSMPL, P_SPIX_SBSMPL, P_LEN_COMP_ATT,
  P_STEM_PHSHFT, P_SLIN_PHSHFT, P_SPIX_PHSHFT.

The code is committed into the repository.
Developers, please update your sandboxes.

Regards, Bram.
--
Bram Riemens, Senior Principal
NXP Semiconductors / Corporate I&T / Research
Email: bra...@nx...
<--- New address and phone since June 2007 -->
Phone: +31 40 27 25910; Fax: +31 40 27 44639
High Tech Campus 32 (floor 1, office 140)
5656 AE Eindhoven, The Netherlands

The information contained in this message is confidential and may be
legally privileged. The message is intended solely for the addressee(s)=
. If
you are not the intended recipient, you are hereby notified that any us=
e,
dissemination, or reproduction is strictly prohibited and may be unlawf=
ul.
If you are not the intended recipient, please contact the sender by ret=
urn
e-mail and destroy all copies of the original message.=

[Pfspd-users@sf.net] MTK updates

[Bram R. <bra...@nx...>]

Dear all,

The development version of MTK is enriched with some new features.
>From the Readme.txt file:
- Microsoft Visual Studio version 8 is supported.
  The makefile system automatically determines which version
  is installed. Currently version 7 (.NET) and version 8 are recognized=
.
  Depending on the version, proper debug and optimization options
  are set.
- New settings in Makefile: DFLAGS_UNIX and DFLAGS_WINDOWS
  Environment specific DFLAGS used to overrule both the generic
  DFLAGS value and the MTK default flags.
  DFLAGS_WINDOWS is used when building on cygwin
  DFLAGS_UNIX is used when building on other platforms
- New settings in Makefile: CFLAGS_<COMP>, CXXFLAGS_<COMP>,
  LFLAGS_<COMP> and CLFLAGS_<COMP>
  Compiler specific values used to overrule both the generic
  values CFLAGS, CXXFLAGS, LFLAGS, CLFLAGS, and the MTK default
  flags.
  <COMP> is GCC, ICC, MSVC, SGI or TM.
  Use of these compiler specific variables is preferred over the
  use of the generic variables in view of the differences of
  available options and syntax.
- New settings in Makefile: CFLAGS2_<COMP>, CXXFLAGS2_<COMP>
  LFLAGS2_<COMP> and CLFLAGS2_<COMP>.
  Compiler specific flags appended to the MTK defaults.
  <COMP> is GCC, ICC, MSVC, SGI or TM.
  Use of these compiler specific variables is preferred over the
  use of the generic variables in view of the differences of
  available options and syntax.

Developers, please update your sandboxes.

Regards, Bram.
--
Bram Riemens, Senior Principal
NXP Semiconductors / Corporate I&T / Research
Email: bra...@nx...
<--- New address and phone since June 2007 -->
Phone: +31 40 27 25910; Fax: +31 40 27 44639
High Tech Campus 32 (floor 1, office 140)
5656 AE Eindhoven, The Netherlands

The information contained in this message is confidential and may be
legally privileged. The message is intended solely for the addressee(s)=
. If
you are not the intended recipient, you are hereby notified that any us=
e,
dissemination, or reproduction is strictly prohibited and may be unlawf=
ul.
If you are not the intended recipient, please contact the sender by ret=
urn
e-mail and destroy all copies of the original message.=

[Pfspd-users@sf.net] MTK updates - TM c++ support

[Bram R. <bra...@nx...>]

Hi all,

A few MTK updates in the development tree.
>From the Readme file:

Functionality
- When using the Trimedia compiler, C++ sources are now compiled with
  the tmc++ command, in a similar fashion as the gcc/g++ distinction.
  Similarly, the linker command is adaptive when C++ sources are part
  of the application. Up till now, mtk did not support C++ compilation
  with the TriMedia compiler.

Miscellaneous
- Default TCS release at natlab updated to version 5.0er3.
- The TMSIMFLAGS variable in the sample Makefile is set to
  "-v -memorydelay 30 -clockfrequency 300000000 -clockratio 2 1 1 
-memorysize 134217728"
  to override the "-v" default of Make.mtk. This serves as an application
  example.
- For TriMedia compiler, the *.i files are moved from the src dir
  to the object dir, similar to the *.t and *.s files.


Regards, Bram.

--
Bram Riemens, Principal Scientist

Research, Systems and Algorithms
NXP Semiconductors
Office: WDC 3.41, Postbox WDC-03    <--- New address since Sept. 15, 2006
High Tech Campus 31, 5656 AE Eindhoven, The Netherlands
Tel: +31 40 27 43833, Fax: +31 40 27 44639
E-mail: bra...@nx...     www.nxp.com

The information contained in this message is confidential and may be 
legally privileged. The message is intended solely for the addressee(s). 
If you are not the intended recipient, you are hereby notified that any 
use, dissemination, or reproduction is strictly prohibited and may be 
unlawful. If you are not the intended recipient, please contact the sender 
by return e-mail and destroy all copies of the original message.

[Pfspd-users@sf.net] Fw: [Pfspd-users@philips] pfspd file re-open

[Bram R. <bra...@nx...>]

Dear Jan, others,

Thanks for the excellent code exerpt that showed the problem.
Indeed, you isolated a bug that exists already for years.
It is now resolved in the code repository.

Here is the description from the Readme.txt.file:

- Application segmentation fault on Linux.
  Following calling sequence failed at the p_read_header() routine:
      p_create_header(&out_header, P_COLOR_444_PL, P_50HZ);
      p_write_header("test.yuv", &out_header);
      p_close_file("test.yuv");
      p_read_header("test.yuv", &out_header);
  More general: the program may crash when accessing a closed file
  in a different mode than it was previously used.
  Underlaying cause: when accessing a file in a different mode,
  it is first closed, then opened in the appropriate mode.
  This test was incomplete: erreonously, a closed file is closed
  again, effectively calling fclose(NULL). This is implementation
  specific; some platforms ignore fclose(NULL).


I guess this bug only surfaced now, because:
- use of p_open_file() and p_close_file() is rare
- use of these functions in different modes is even more rare
- the erronous behaviour is platform dependent
Anyhow, it is good that this is now resolved.


I could not find a workaround in the application.
Note that the problem is platform dependent, on HP, I know it will run ok.
I order to continue with your application, you can path a cpfspd release 
as follows.
The modification is in the first "if" statement.

File cpfspd_low.c
Routine p_get_file_pointer()

Old code:
        /* If open file found */
        if (idx != -1) {
            /* Opened as read, write access required? Close it */
            /* Opened as write, read access required? Close it */
            if ( ((mode != p_mode_read) && (p_files[idx].mode == 
p_mode_read)) ||
                 ((mode == p_mode_read) && (p_files[idx].mode == 
p_mode_write)) ) {
                p_close_idx(idx); /* Ignore status - we'll access the same 
file later */
            }
        }

New code:
        /* If open file found */
        if ((idx != -1) && (p_files[idx].fp != NULL)) {
            /* Opened as read, write access required? Close it */
            /* Opened as write, read access required? Close it */
            if ( ((mode != p_mode_read) && (p_files[idx].mode == 
p_mode_read)) ||
                 ((mode == p_mode_read) && (p_files[idx].mode == 
p_mode_write)) ) {
                p_close_idx(idx); /* Ignore status - we'll access the same 
file later */
            }
        }

Regards, Bram

--
Bram Riemens, Principal Scientist

Research, Systems and Algorithms
NXP Semiconductors
Office: WDC 3.41, Postbox WDC-03    <--- New address since Sept. 15, 2006
High Tech Campus 31, 5656 AE Eindhoven, The Netherlands
Tel: +31 40 27 43833, Fax: +31 40 27 44639
E-mail: bra...@nx...     www.nxp.com

The information contained in this message is confidential and may be 
legally privileged. The message is intended solely for the addressee(s). 
If you are not the intended recipient, you are hereby notified that any 
use, dissemination, or reproduction is strictly prohibited and may be 
unlawful. If you are not the intended recipient, please contact the sender 
by return e-mail and destroy all copies of the original message.








Jan Caussyn 
Sent by:
pfs...@gf...
27-10-2006 17:14

To
pfs...@gf...
cc

Subject
[Pfspd-users@philips] pfspd file re-open
Classification








Hi, 

Who has a solution to this one ?  In a certain case, I want to make a 
clean start with a certain file, so I close it using p_close_file. 
However, this causes an unhandled exception in a subsequent p_read_header. 

An example you find below : 

This crashes : 

#include "cpfspd.h" 
#include <stdlib.h> 

/* main declaration */ 
/************************************************************************/ 

int main(int argc,char *argv[]) 
{ 
        pT_header                out_header ; 
        pT_status                status ; 
 
 
          int                        xdim, ydim, nframes ; 
          int                        xcount, ycount, framecount ; 
 
    status = p_create_header (&out_header, P_COLOR_444_PL, P_50HZ) ; 
         status = p_write_header ("test.yuv", &out_header ); 

    p_close_file ("test.yuv") ; 

        status = p_read_header ("test.yuv", &out_header) ; 
        p_fatal_error_fileio (status, "test.yuv", stderr) ; 

        exit(0) ; 
}        

If you put the p_close_file line in comment, it works fine, but that is 
not what I am looking for. 
I tried 1.10.1, 1.9.2 and 1.8.2, it's all the same. 

Any comments appreciated, 
Regards, 
Jan 

__________________________________________________________

 Jan Caussyn                                             Philips 
Innovative Applications NV Brugge
 Phone : +32 (0)50 455 750                     Product Development 
 Fax      : +32 (0)50 455 722                     Entrance 2
 Pathoekeweg 11                                     B8000 Brugge, Belgium  
 
 mailto : jan...@ph...
_______________________________________________
Pfspd-users mailing list
Pfs...@gf...
http://ithilien.natlab.research.philips.com/mailman/listinfo/pfspd-users

Re: [Pfspd-users@sf.net] PV rgb conversion

[Harold S. <har...@ph...>]

Hi Bram,

I was aware that the current code may overflow with 16 bit data,
and had already converted the code to 12 bit after the dot (but not 
committed yet).
I'll modify the precision to be a compile-time constant, and commit to 
cvs.

On a related note:
I intend to optimize the convert_line_3x3 functions with SSE2 
instructions,
but this is not compatible with the precompiled wxWindows library in 
/home/sofftv/
I guess wxWindows needs to be recompiled ?

Regards,
Harold
___________________________________
Ing. H.A.W. Schmeitz
Video Processing Systems, Philips Research Laboratories

High Tech Campus 36, 5656 AE Eindhoven, The Netherlands
Tel. +31 (0)40 27 46489
har...@ph...








Bram Riemens 
Sent by:
pfs...@li...
28-09-2006 15:01

To
pfs...@li...
cc

Subject
[Pfs...@sf...] PV rgb conversion
Classification








Hi Harold, 

Recently, you changed / generalized the RGB conversion of PV module 
'format_convert'. 

Currently, only 8 bits input data is handled; however, we plan for the 
near future to handle up to 16 bit files wherein the 16 bits are retained 
in all calculations. 
Calculation is done in fixed point. 
Matrixes are specified in floating point. 
In the conversion from float to fixed point, 16 bits after the "dot" are 
used. 

This means for 16 bit data: 16 data + 16 after-dot = 32 bits 
Thus, 32 bits are really required, there is no headroom. 
Therefore, clipping will not work, as the to-be clipped data is wrapped 
around by the 32 bit C arithmetic. 

Proposal: change the calculations to 8 bits after-dot precision to prepare 
for 16 bits input data. 

Regards, Bram. 

--
A.K. (Bram) Riemens
Principal Scientist, Systems & Algorithms, NXP Corporate Research
Office: WDC 3.41, Postbox WDC-03    <--- New address
High Tech Campus 31 (WDC), 5656 AE Eindhoven, The Netherlands
Tel: +31 40 27 43833, Fax: +31 40 27 44639
E-mail: bra...@ph...
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share 
your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Pfspd-users mailing list
Pfs...@li...
https://lists.sourceforge.net/lists/listinfo/pfspd-users

[Pfspd-users@sf.net] PV rgb conversion

[Bram R. <bra...@ph...>]

Hi Harold,

Recently, you changed / generalized the RGB conversion of PV module 
'format_convert'.

Currently, only 8 bits input data is handled; however, we plan for the 
near future to handle up to 16 bit files wherein the 16 bits are retained 
in all calculations.
Calculation is done in fixed point.
Matrixes are specified in floating point.
In the conversion from float to fixed point, 16 bits after the "dot" are 
used.

This means for 16 bit data: 16 data + 16 after-dot = 32 bits
Thus, 32 bits are really required, there is no headroom.
Therefore, clipping will not work, as the to-be clipped data is wrapped 
around by the 32 bit C arithmetic.

Proposal: change the calculations to 8 bits after-dot precision to prepare 
for 16 bits input data.

Regards, Bram.

--
A.K. (Bram) Riemens
Principal Scientist, Systems & Algorithms, NXP Corporate Research
Office: WDC 3.41, Postbox WDC-03    <--- New address
High Tech Campus 31 (WDC), 5656 AE Eindhoven, The Netherlands
Tel: +31 40 27 43833, Fax: +31 40 27 44639
E-mail: bra...@ph...

Re: [Pfspd-users@sf.net] multilayer extension to pfspd

[Bram R. <bra...@ph...>]

Hi Chris,

A features like "component mapping" to allow the user to select which
component to use for what purpose is already proposed for quite
some time. Although discussed in several occasions, the
ideas as not completely settled down.

This is an attempt to describe the current ideas, as they levelled
from discussions with Harold, Rene, etc.

Currently, we can select the "color display method" (cycles through
the options by hitting the 'c' key). Depending on the method, the
software automagically selects some components from the file...
This needs to be decoupled. So here is the proposal.

For each entry of the "color display method", some
inputs are required as follows:
full color   R G B   |   Y U V   |   Y
grayscale    R G B   |   Y
Y & vector   Y Vx Vy
only vector  Vx Vy
Y & scalar   Y S
only scalar  S

Further, one needs to specify which component from the input file serves
as input to the selected the color display method.
This is called "component mapping".
For compatibility with current mode of operation and for conveniency,
proper default component mapping can be implemented.

The components read from the file are processed through a matrix
(if appropriate). Currently there is only the ITU601 matrix.
This will be generalized to offer selections for
- ITU601
- ITU709
- custom
Again, proper defaults are possible.

Further, there is a choice for "normal or full range".
This is an independent choice, which can be extended with
properties like offset and gain to facilitate high dynamic range video.

So, the selection order is:
- color display method
which determines what kind of components are required, then
- component mapping (to determines the components to be accessed from the 
file)
- matrix choice

Independent of this, several display settings can be chosen
(like currently normal/full range).

The processing order is:
- select components from file
- execute matrix
- execute color display method
- handle display settings


Currently, Harold is working on the underlaying calculations, which 
requires a
major code refactoring to generalize and streamline many of the existing 
functions.
Most of the individual steps can be implemented by a single pixel loop
(by first cascading the operations on the generalized matrix definition),
so performance is  expected to remain very good, even though the code
is significantly generalized.

In a next stage, the user interface will be adapted to enable the wider
and more generic functionality.

Your further inputs and suggestions are welcome.
Support and manhours even more...

Regards, Bram.

--
A.K. (Bram) Riemens
Principal Scientist, DSP group, Philips Research
Office: WO-p-94, Postbox WO02
High Tech Campus 36 (WO), 5656 AE Eindhoven, The Netherlands
Tel: +31 40 27 43833, Fax: +31 40 27 44675
E-mail: bra...@ph...








Chris Bartels 
Sent by:
pfs...@li...
17-08-2006 13:51

To
pfs...@li...
cc

Subject
[Pfs...@sf...] multilayer extension to pfspd
Classification








As a feature request for new pfspd toolset versions: is it possible to add 
a multiple layers extension to the format / viewer, i.e. such that the 
user can add an arbitrary number of color (RGBA) / vector (float,float) 
overlays. The viewer already contains a feature for one vector overlay but 
this is not always sufficient. 

Has anybody had similar problems, or maybe solutions? 

Kind regards, 
Chris Bartels
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job 
easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Pfspd-users mailing list
Pfs...@li...
https://lists.sourceforge.net/lists/listinfo/pfspd-users

[Pfspd-users@sf.net] multilayer extension to pfspd

[Chris B. <chr...@ph...>]

As a feature request for new pfspd toolset versions: is it possible to add 
a multiple layers extension to the format / viewer, i.e. such that the 
user can add an arbitrary number of color (RGBA) / vector (float,float) 
overlays. The viewer already contains a feature for one vector overlay but 
this is not always sufficient.

Has anybody had similar problems, or maybe solutions?

Kind regards,
Chris Bartels

Re: [Pfspd-users@sf.net] Serious bug causing data corruption

[Harold S. <har...@ph...>]

Bug fixed:
The endianness mode of the file was incorrectly set by p=5Frewrite=5Fheader=
().

Developers: please update your cpfspd sandbox, and recompile cpfspd and=20
pts.

Regards,
Harold
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
Ing. H.A.W. Schmeitz
Video Processing Systems, Philips Research Laboratories

High Tech Campus 36, 5656 AE Eindhoven, The Netherlands
Tel. +31 (0)40 27 46489
har...@ph...








Rene van der Vleuten=20
Sent by:
pfs...@li...
16-08-2006 15:50

To
pfs...@li...
cc

Subject
[Pfs...@sf...] Serious bug causing data corruption
Classification








Dear developers:=20

Today I managed to corrupt 175GB of data in a few seconds by simply using=20
"pts header". The problem fortunately does not occur in the latest=20
released pts version, but only in the development version. The problem=20
seems to be invoked by using pts header on Linux on a file that was=20
created on Windows (because the problem does not appear when I first pts=20
cp the file under Linux). I suspect the problem is in cpfspd and not in=20
the recent modification of pts header. Please contact me directly if you=20
need test data.=20

Regards,=20

Ren=E9.=20

--------------------------------------------------------------------------
Dr. Rene J. van der Vleuten
Philips Research Laboratories
High Tech Campus 36
5656 AE  Eindhoven
The Netherlands

Phone: +31 40 2742941
Fax:   +31 40 2742630
mailto:Ren...@ph...=20
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job=20
easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D120709&bid=3D263057&dat=3D1=
21642
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
Pfspd-users mailing list
Pfs...@li...
https://lists.sourceforge.net/lists/listinfo/pfspd-users

[Pfspd-users@sf.net] Serious bug causing data corruption

[Rene v. d. V. <ren...@ph...>]

<br><font size=3D2 face=3D"sans-serif">Dear developers:</font>
<br>
<br><font size=3D2 face=3D"sans-serif">Today I managed to corrupt 175GB of
data in a few seconds by simply using &quot;pts header&quot;. The problem
fortunately does not occur in the latest released pts version, but only
in the development version. The problem seems to be invoked by using pts
header on Linux on a file that was created on Windows (because the problem
does not appear when I first pts cp the file under Linux). I suspect the
problem is in cpfspd and not in the recent modification of pts header.
Please contact me directly if you need test data.</font>
<br>
<br><font size=3D2 face=3D"sans-serif">Regards,</font>
<br>
<br><font size=3D2 face=3D"sans-serif">Ren=E9.</font>
<br><font size=3D2 face=3D"sans-serif"><br>
--------------------------------------------------------------------------<=
br>
Dr. Rene J. van der Vleuten<br>
Philips Research Laboratories<br>
High Tech Campus 36<br>
5656 AE &nbsp;Eindhoven<br>
The Netherlands<br>
<br>
Phone: +31 40 2742941<br>
Fax: &nbsp; +31 40 2742630<br>
mailto:Ren...@ph...</font>

Re: [Pfspd-users@sf.net] Cpfspd change proposal on aspect ratiobehaviour

[Bram R. <bra...@ph...>]

Dear all,

The changes as described below have been implemented.
Note however, that the behaviour of p_create_header() is not modified
since the SD image size still causes the default 4:3 aspect ratio.

>From the readme file:

- The display aspect ratio setting can now have value P_AS_WH,
  indicating that the intended aspect ratio of the diplay
  is equal to the width/height ratio in pixels. This value
  is an indication of square pixels as long as the width/height
  ratio remains the same.
- Function p_create_ext_header() can now be called with
  value P_UNKNOWN_ASPECT_RATIO in the ratio parameter.
  This causes an automatic 16:9/4:3 selection.
  See cpfspd.h for details on the selection algorithm.

>From the cpfspd.h file:

  The pfspd file format supports a display aspect ratio setting.
  This setting defines the intended shape of the physical display
  used by the image.
  Function p_create_ext_header() accepts the aspect ratio as input
  parameter. The default aspect ratio setting is 4:3 when using
  function p_create_header().
  When using function p_create_ext_header() with ratio argument
  P_UNKNOWN_ASPECT_RATIO, then an automatic aspect ratio is choosen,
  according to:
  - For image size HDi and HDp: the aspect ratio is always 16:9.
  - For other image sizes (SD, CIF, QCIF):
    If the number of pixels per line is larger than the default value,
    then the aspect ratio is 16:9; otherwise, the aspect ratio is 4:3.

  Notes:
  1) Aspect ratio setting P_AS_WH indicates that the aspect ratio
  of the display equals the aspect ratio of the image width/height.
  This is equal to a "square pixel" setting. However, after scaling or
  cropping (i.e. changing the width and/or height of the image) then
  the application is responsible to maintain a square pixel setting,
  since the pfspd file stores the display aspect ratio (without
  any knowledge on the pixel aspect ratio).
  This can be achieved by calling p_mod_aspect_ratio().


Developers, please update your sandboxes.

Regards, Bram.

--
A.K. (Bram) Riemens
Principal Scientist, DSP group, Philips Research
Office: WO-p-94, Postbox WO02
High Tech Campus 36 (WO), 5656 AE Eindhoven, The Netherlands
Tel: +31 40 27 43833, Fax: +31 40 27 44675
E-mail: bra...@ph...








Bram Riemens 
Sent by:
pfs...@li...
26-06-2006 11:50

To
pfs...@li...
cc
Frank van Heesch/EHV/RESEARCH/PHILIPS@PHILIPS
Subject
[Pfs...@sf...] Cpfspd change proposal on aspect   ratiobehaviour
Classification








Hi all, 

Frank van Heesch triggered a disussion on the use of the display 
aspect ratio in cpfspd and related tools. 
This email outlines a CPFSPD change proposal. 


Current situation 

Cpfspd supports two display aspect ratios: 4:3 and 16:9. 
These are both commonly used in current TV standards. 
For several reasons, current tools and files do not use the 16:9 
values properly - the 4:3 default is almost always used. 
Consequently, almost every file is tagged as 4:3. 
Up till now, we did not really pay attention to the aspect ratio. 
Frank pointed out some good reasons to pay attention: 
- Frank is working on an application that needs to use the information 
- Wrong values are confusing 


Requirements 

- Appropriate defaults for aspect ration 
  (so most tools will "automatically" set correct aspect ratio values). 
- Support for "square pixel" aspect ratio 
  (since many pfspd files contain a graphics image) 


Boundary conditions 

- Minimal or no change in API. 
- Consistency with current API semantics. 
- No change in pfspd file format. 
  The file format describes only a __display__ aspect ratio (or image 
aspect ratio). 
  Hence, a "square pixel" aspect ratio is only valid for a certain image 
width/height 
  and cannot be preserved automatically over scaling operations. 


Propsed solution 

Both PV and PP will NOT be adapted. Main focus of these tools is 
visulisation 
of the pixels. A deformation of the image aspect ratio is acceptable and 
clearly 
favourable over scaling (requiering filtering with two main disadvantages: 
original 
pixel values are lost and slowdown due to the extra calculations). 

cpfspd.h 
- Add P_AS_WH to pT_asp_rat. 

cpfspd_get.c / p_get_aspect_ratio() 
- Add P_AS_WH; use largest common divider from image width/height to check 
this value. 

cpfspd_mod.c / p_mod_aspect_ratio() 
- Add P_AS_WH; use largest common divider from image width/height to 
calculate this value. 
Note that this is only "square pixel" for the current width/height 
setting. 

cpfspd_hdr.c / p_set_header_values()                internal function 
- Add P_AS_WH as parameter value; use largest common divider from image 
width/height 
to calculate this value. 
- Add P_UNKNOWN_ASPECT_RATIO to serve as "default" parameter value 
to enable automatic setting depending on pT_size values: 
P_4_3  for   P_SD, P_CIF,  P_QCIF 
P_16_9 for P_HDp,  P_HDi 

cpfspd_hdr.c / p_set_stream_header_values()                internal 
function 
As p_set_header_values(). 

cpfspd_hdr.c / p_create_header() 
- Pass P_UNKNOWN_ASPECT_RATIO (not P_4_3) to p_create_ext_header(). 

cpfspd_hdr.c / p_create_ext_header() 
- No code change, funcionality changed by underlaying 
p_set_header_values() / p_set_stream_header_values(). 


Non-cpfspd issues 

Cpfspd application/tools that change image size (scale, border, crop, 
trim) need to handle the 
aspect ratio explicitely. If input file has P_AS_WH then output must be 
explicitely set 
at P_AS_WH to retain the "square pixel" property. 


Your comments / suggestions / improvements are appreciated. 

Regards, Bram. 

PS: Frank, please subscribe to the relevant mailing list via 
https://sourceforge.net/mail/?group_id=137382

--
A.K. (Bram) Riemens
Principal Scientist, DSP group, Philips Research
Office: WO-p-94, Postbox WO02
High Tech Campus 36 (WO), 5656 AE Eindhoven, The Netherlands
Tel: +31 40 27 43833, Fax: +31 40 27 44675
E-mail: bra...@ph...Using Tomcat but need to do more? Need to 
support web services, security?
Get stuff done quickly with pre-integrated technology to make your job 
easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Pfspd-users mailing list
Pfs...@li...
https://lists.sourceforge.net/lists/listinfo/pfspd-users

[Pfspd-users@sf.net] Cpfspd change proposal on aspect ratio behaviour

[Bram R. <bra...@ph...>]

Hi all,

Frank van Heesch triggered a disussion on the use of the display
aspect ratio in cpfspd and related tools.
This email outlines a CPFSPD change proposal.


Current situation

Cpfspd supports two display aspect ratios: 4:3 and 16:9.
These are both commonly used in current TV standards.
For several reasons, current tools and files do not use the 16:9
values properly - the 4:3 default is almost always used.
Consequently, almost every file is tagged as 4:3.
Up till now, we did not really pay attention to the aspect ratio.
Frank pointed out some good reasons to pay attention:
- Frank is working on an application that needs to use the information
- Wrong values are confusing


Requirements

- Appropriate defaults for aspect ration
  (so most tools will "automatically" set correct aspect ratio values).
- Support for "square pixel" aspect ratio
  (since many pfspd files contain a graphics image)


Boundary conditions

- Minimal or no change in API.
- Consistency with current API semantics.
- No change in pfspd file format.
  The file format describes only a __display__ aspect ratio (or image 
aspect ratio).
  Hence, a "square pixel" aspect ratio is only valid for a certain image 
width/height
  and cannot be preserved automatically over scaling operations.


Propsed solution

Both PV and PP will NOT be adapted. Main focus of these tools is 
visulisation
of the pixels. A deformation of the image aspect ratio is acceptable and 
clearly
favourable over scaling (requiering filtering with two main disadvantages: 
original
pixel values are lost and slowdown due to the extra calculations).

cpfspd.h
- Add P_AS_WH to pT_asp_rat.

cpfspd_get.c / p_get_aspect_ratio()
- Add P_AS_WH; use largest common divider from image width/height to check 
this value.

cpfspd_mod.c / p_mod_aspect_ratio()
- Add P_AS_WH; use largest common divider from image width/height to 
calculate this value.
Note that this is only "square pixel" for the current width/height 
setting.

cpfspd_hdr.c / p_set_header_values()                internal function
- Add P_AS_WH as parameter value; use largest common divider from image 
width/height
to calculate this value.
- Add P_UNKNOWN_ASPECT_RATIO to serve as "default" parameter value
to enable automatic setting depending on pT_size values:
P_4_3  for   P_SD, P_CIF,  P_QCIF
P_16_9 for P_HDp,  P_HDi

cpfspd_hdr.c / p_set_stream_header_values()                internal 
function
As p_set_header_values().

cpfspd_hdr.c / p_create_header()
- Pass P_UNKNOWN_ASPECT_RATIO (not P_4_3) to p_create_ext_header().

cpfspd_hdr.c / p_create_ext_header()
- No code change, funcionality changed by underlaying
p_set_header_values() / p_set_stream_header_values().


Non-cpfspd issues

Cpfspd application/tools that change image size (scale, border, crop, 
trim) need to handle the
aspect ratio explicitely. If input file has P_AS_WH then output must be 
explicitely set
at P_AS_WH to retain the "square pixel" property.


Your comments / suggestions / improvements are appreciated.

Regards, Bram.

PS: Frank, please subscribe to the relevant mailing list via 
https://sourceforge.net/mail/?group_id=137382

--
A.K. (Bram) Riemens
Principal Scientist, DSP group, Philips Research
Office: WO-p-94, Postbox WO02
High Tech Campus 36 (WO), 5656 AE Eindhoven, The Netherlands
Tel: +31 40 27 43833, Fax: +31 40 27 44675
E-mail: bra...@ph...

Re: [Pfspd-users@sf.net] Workaround for bug # 1506617 (crash whengrowing large network files)

[Bram R. <bra...@ph...>]

Hi all,

After some suggestions by Harold, I limited the repetition of the failing 
operation to 2000 times.
Further, special file allocation tests for these cases are added to the 
cpfspd test script.

Developers, please update your sandboxes.

Regards, Bram.

--
A.K. (Bram) Riemens
Principal Scientist, DSP group, Philips Research
Office: WO-p-94, Postbox WO02
High Tech Campus 36 (WO), 5656 AE Eindhoven, The Netherlands
Tel: +31 40 27 43833, Fax: +31 40 27 44675
E-mail: bra...@ph...








Bram Riemens 
Sent by:
pfs...@li...
19-06-2006 17:44

To
pfs...@li...
cc

Subject
[Pfs...@sf...] Workaround for bug # 1506617 (crash whengrowing large 
network files)
Classification








Hi all, 

Today, I implemented a workaround for this bug. 
https://sourceforge.net/tracker/index.php?func=detail&aid=1506617&group_id=137382&atid=738511 


Workaround implemented in cpfspd_fio.c: 
The workaround involves catching the error code and reissuing the failing 
operation, involving: the steps of: close file, reopen file, write data. 
In function fio_fp_write_buf() for synchronous operation and in function 
fio_fp_wait() for asynchronous operation. 

The fix is in the sourceforge.net CVS repository. 
Developers, please update your sandboxes. 

Thanks to Harold for the discussions and suggestions. 

Regards, Bram.

--
A.K. (Bram) Riemens
Principal Scientist, DSP group, Philips Research
Office: WO-p-94, Postbox WO02
High Tech Campus 36 (WO), 5656 AE Eindhoven, The Netherlands
Tel: +31 40 27 43833, Fax: +31 40 27 44675
E-mail: bra...@ph...
_______________________________________________
Pfspd-users mailing list
Pfs...@li...
https://lists.sourceforge.net/lists/listinfo/pfspd-users

Re: [Pfspd-users@sf.net] [Pfspd-users@philips] PFSPD aspect ratios

[Rene v. d. V. <ren...@ph...>]

<br><font size=3D2 face=3D"sans-serif">Hi Frank!</font>
<br>
<br><font size=3D2 face=3D"sans-serif">In addition to changing pts header,
I think it would be good to also change cpfspd such that the 16:9 (and
4:3) aspect ratio is correctly set during file creation and width/height
modification (therefore I am cross-posting to the cpfspd discussion list).
In addition to 16:9 and 4:3 aspect ratios, I think we also need a &quot;squ=
are
pixel&quot; aspect ratio indication (which simply means the aspect ratio
is determined by the frame width and height). I think these three aspect
ratio choices would be sufficient for the moment, but comments from others
on this would be most welcome.</font>
<br>
<br><font size=3D2 face=3D"sans-serif">Regards,</font>
<br>
<br><font size=3D2 face=3D"sans-serif">Ren=E9.</font>
<br><font size=3D2 face=3D"sans-serif"><br>
--------------------------------------------------------------------------<=
br>
Dr. Rene J. van der Vleuten<br>
Philips Research Laboratories<br>
High Tech Campus 36<br>
5656 AE &nbsp;Eindhoven<br>
The Netherlands<br>
<br>
Phone: +31 40 2742941<br>
Fax: &nbsp; +31 40 2742630<br>
mailto:Ren...@ph...</font>
<br>

[Pfspd-users@sf.net] Workaround for bug # 1506617 (crash when growing large network files)

[Bram R. <bra...@ph...>]

Hi all,

Today, I implemented a workaround for this bug.
https://sourceforge.net/tracker/index.php?func=detail&aid=1506617&group_id=137382&atid=738511

Workaround implemented in cpfspd_fio.c:
The workaround involves catching the error code and reissuing the failing 
operation, involving: the steps of: close file, reopen file, write data. 
In function fio_fp_write_buf() for synchronous operation and in function 
fio_fp_wait() for asynchronous operation.

The fix is in the sourceforge.net CVS repository.
Developers, please update your sandboxes.

Thanks to Harold for the discussions and suggestions.

Regards, Bram.

--
A.K. (Bram) Riemens
Principal Scientist, DSP group, Philips Research
Office: WO-p-94, Postbox WO02
High Tech Campus 36 (WO), 5656 AE Eindhoven, The Netherlands
Tel: +31 40 27 43833, Fax: +31 40 27 44675
E-mail: bra...@ph...

[Pfspd-users@sf.net] New bug found in file allocation

[Bram R. <bra...@ph...>]

Hi all,

This is to inform you that a new cpfspd bug has been identified.
https://sourceforge.net/tracker/index.php?func=detail&aid=1506617&group_id=137382&atid=738511
When executing a win32/cygwin exec, and growing a network file with a very 
large chunk (say 5 Gbyte), then the write operation will fail.

This is related to unbuffered/asynchronous file I/O.

We will work on this issue.

Regards, Bram.

--
A.K. (Bram) Riemens
Principal Scientist, DSP group, Philips Research
Office: WO-p-94, Postbox WO02
High Tech Campus 36 (WO), 5656 AE Eindhoven, The Netherlands
Tel: +31 40 27 43833, Fax: +31 40 27 44675
E-mail: bra...@ph...

Re: [Pfspd-users@sf.net] performance problems

[Harold S. <har...@ph...>]

Hi Rene, Bram,

Write performance seems to be dependent on access mode.
I think "rb+" implies read-modify-write of chunks of the file, hence=20
slower speed.

The performance problem is due to the brain-dead win32 API, where we have
to open a file in non-overlapped mode for allocation, close it, then open=20
in overlapped mode for performance.
For simplicity, we did the same for Linux (open, allocate, close, open),=20
but this meant the file was reopened in
update mode (otherwise it would be truncated to 0 bytes).

I fixed the issue with conditional compilation (#ifdef FIO=5FWIN32=5FFILE)
For Linux, I put the file allocation to the regular spot (after file=20
open), so the access
mode can be "wb" (write) or "rb+" (update), as specified by the rewrite=20
flag.
For Win32, allocation is done first, followed by a re-open in update mode.

Please verify if this solves your performance problems.
Regards,
Harold
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
Ing. H.A.W. Schmeitz
Video Processing Systems, Philips Research Laboratories

High Tech Campus 36, 5656 AE Eindhoven, The Netherlands
Tel. +31 (0)40 27 46489
har...@ph...








Rene van der Vleuten=20
Sent by:
pfs...@li...
09-06-2006 12:59

To
Bram Riemens/EHV/RESEARCH/PHILIPS@PHILIPS
cc
pfs...@li...
Subject
Re: [Pfs...@sf...] performance problems
Classification








Hi!=20

Without being an expert (and without having looked at the code), I would=20
expect you could use append mode while writing data and then open in=20
update mode when the file is closed in order to correct the header (i.e. I =

would not update the header after each frame has been written, assuming=20
that is what is currently happening).=20

Regards,=20

Ren=E9.=20

--------------------------------------------------------------------------
Dr. Rene J. van der Vleuten
Philips Research Laboratories
High Tech Campus 36
5656 AE  Eindhoven
The Netherlands

Phone: +31 40 2742941
Fax:   +31 40 2742630
mailto:Ren...@ph...=20







Bram Riemens=20
Sent by:=20
pfs...@li...=20
08-06-2006 17:06=20


To
pfs...@li...=20
cc
Gerard Bloemen/EHV/RESEARCH/PHILIPS@PHILIPS=20
Subject
Re: [Pfs...@sf...] performance problems=20
Classification










Rene, others,=20

After obtaining a test case from Rene, I did some investigations on the=20
current development=20
repository of cpfspd and pts.=20
Test conditions:=20
- linux executable, running on dimholt=20
- output to scratch disk on muscules file server=20
- command "pts translate corvette=5F8mbps.m2v corvette=5F8mbps.yuv"=20
- result is file of 3.6Gbyte; 1198 frames 1920x1080=20

Currently (development version of cpfspd), file is opened in mode "wb" for =

allocation,=20
then it is closed. With pts translate on mpeg, the file size is not known=20
in advance,=20
so it is initially created with size 22528 bytes.=20
After that, it is reopened in "rb+" mode. Then images are written into the =

file.=20
Result of "time":=20
real    18m11.415s=20
user    1m10.260s=20
sys     1m2.590s=20

As a special test, I forced opening the file in "wb" mode the 2nd time;=20
result:=20
real    2m23.710s=20
user    1m5.710s=20
sys     0m20.850s=20

Next test; force mode "wb+" is also very slow.=20

I tested opening in append modes (followed by a reset of the file=20
pointer), but that fails to give correct output.=20



So, just changing the fopen mode results in a performance penalty of a=20
factor 7.5...=20
Interesting observation...=20


Harold, any suggestion how to keep the file open, and still maintain a=20
simple interface=20
for both *nix and win32/cygwin?=20


Regards, Bram.=20

PS. Gerard, maybe this is related to your observations?=20
       What version did you use?=20
       You can subscribe this mailing list at=20
               https://lists.sourceforge.net/lists/listinfo/pfspd-users

--
A.K. (Bram) Riemens
Principal Scientist, DSP group, Philips Research
Office: WO-p-94, Postbox WO02
High Tech Campus 36 (WO), 5656 AE Eindhoven, The Netherlands
Tel: +31 40 27 43833, Fax: +31 40 27 44675
E-mail: bra...@ph...=20






Rene van der Vleuten=20
Sent by:=20
pfs...@li...=20
08-06-2006 13:04=20


To
pfs...@li...=20
cc

Subject
[Pfs...@sf...] performance problems=20
Classification












Dear Bram, Harold,=20

I am suffering from huge performance problems with "pts translate file.m2v =

file.yuv"; processing takes more than 10x longer than with the latest=20
released pts!! I suspect these are caused by your latest updates to cpfspd =

(since I did not see any obviously related changes in pts).=20

Could you please look into this?=20

Thanks!=20

Ren=E9=20

--------------------------------------------------------------------------
Dr. Rene J. van der Vleuten
Philips Research Laboratories
High Tech Campus 36
5656 AE  Eindhoven
The Netherlands

Phone: +31 40 2742941
Fax:   +31 40 2742630
mailto:Ren...@ph...=20
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
Pfspd-users mailing list
Pfs...@li...
https://lists.sourceforge.net/lists/listinfo/pfspd-users
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
Pfspd-users mailing list
Pfs...@li...
https://lists.sourceforge.net/lists/listinfo/pfspd-users
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
Pfspd-users mailing list
Pfs...@li...
https://lists.sourceforge.net/lists/listinfo/pfspd-users

Re: [Pfspd-users@sf.net] performance problems

[Rene v. d. V. <ren...@ph...>]

<br><font size=3D2 face=3D"sans-serif">Hi!</font>
<br>
<br><font size=3D2 face=3D"sans-serif">Without being an expert (and without
having looked at the code), I would expect you could use append mode while
writing data and then open in update mode when the file is closed in order
to correct the header (i.e. I would not update the header after each frame
has been written, assuming that is what is currently happening).</font>
<br>
<br><font size=3D2 face=3D"sans-serif">Regards,</font>
<br>
<br><font size=3D2 face=3D"sans-serif">Ren=E9.</font>
<br><font size=3D2 face=3D"sans-serif"><br>
--------------------------------------------------------------------------<=
br>
Dr. Rene J. van der Vleuten<br>
Philips Research Laboratories<br>
High Tech Campus 36<br>
5656 AE &nbsp;Eindhoven<br>
The Netherlands<br>
<br>
Phone: +31 40 2742941<br>
Fax: &nbsp; +31 40 2742630<br>
mailto:Ren...@ph...</font>
<br>
<br>
<br>
<table width=3D100%>
<tr valign=3Dtop>
<td width=3D33%>
<br>
<br>
<br>
<br>
<br><font size=3D1 face=3D"sans-serif"><b>Bram Riemens</b> </font>
<p><font size=3D1 face=3D"sans-serif">Sent by:</font>
<br><font size=3D1 face=3D"sans-serif">pfs...@li...urceforg=
e.net</font>
<p><font size=3D1 face=3D"sans-serif">08-06-2006 17:06</font>
<td width=3D66%>
<table width=3D100%>
<tr valign=3Dtop>
<td>
<div align=3Dright><font size=3D1 face=3D"sans-serif">To</font></div>
<td><font size=3D1 face=3D"sans-serif">pfs...@li...</f=
ont>
<tr valign=3Dtop>
<td>
<div align=3Dright><font size=3D1 face=3D"sans-serif">cc</font></div>
<td><font size=3D1 face=3D"sans-serif">Gerard Bloemen/EHV/RESEARCH/PHILIPS@=
PHILIPS</font>
<tr valign=3Dtop>
<td>
<div align=3Dright><font size=3D1 face=3D"sans-serif">Subject</font></div>
<td><font size=3D1 face=3D"sans-serif">Re: [Pfs...@sf...] performance
problems</font>
<tr>
<td>
<div align=3Dright><font size=3D1 face=3D"sans-serif">Classification</font>=
</div>
<td></table>
<br>
<table>
<tr valign=3Dtop>
<td>
<td></table>
<div align=3Dright>
<br></div></table>
<br>
<br>
<br><font size=3D2 face=3D"sans-serif"><br>
Rene, others,</font><font size=3D3> <br>
</font><font size=3D2 face=3D"sans-serif"><br>
After obtaining a test case from Rene, I did some investigations on the
current development</font><font size=3D3> </font><font size=3D2 face=3D"san=
s-serif"><br>
repository of cpfspd and pts.</font><font size=3D3> </font><font size=3D2 f=
ace=3D"sans-serif"><br>
Test conditions:</font><font size=3D3> </font><font size=3D2 face=3D"sans-s=
erif"><br>
- linux executable, running on dimholt</font><font size=3D3> </font><font s=
ize=3D2 face=3D"sans-serif"><br>
- output to scratch disk on muscules file server</font><font size=3D3> </fo=
nt><font size=3D2 face=3D"sans-serif"><br>
- command &quot;pts translate corvette=5F8mbps.m2v corvette=5F8mbps.yuv&quo=
t;</font><font size=3D3>
</font><font size=3D2 face=3D"sans-serif"><br>
- result is file of 3.6Gbyte; 1198 frames 1920x1080</font><font size=3D3>
<br>
</font><font size=3D2 face=3D"sans-serif"><br>
Currently (development version of cpfspd), file is opened in mode &quot;wb&=
quot;
for allocation,</font><font size=3D3> </font><font size=3D2 face=3D"sans-se=
rif"><br>
then it is closed. With pts translate on mpeg, the file size is not known
in advance,</font><font size=3D3> </font><font size=3D2 face=3D"sans-serif"=
><br>
so it is initially created with size 22528 bytes.</font><font size=3D3> </f=
ont><font size=3D2 face=3D"sans-serif"><br>
After that, it is reopened in &quot;rb+&quot; mode. Then images are written
into the file.</font><font size=3D3> </font><font size=3D2 face=3D"sans-ser=
if"><br>
Result of &quot;time&quot;:</font><font size=3D3> </font><font size=3D2 fac=
e=3D"sans-serif"><br>
real &nbsp; &nbsp;18m11.415s</font><font size=3D3> </font><font size=3D2 fa=
ce=3D"sans-serif"><br>
user &nbsp; &nbsp;1m10.260s</font><font size=3D3> </font><font size=3D2 fac=
e=3D"sans-serif"><br>
sys &nbsp; &nbsp; 1m2.590s</font><font size=3D3> <br>
</font><font size=3D2 face=3D"sans-serif"><br>
As a special test, I forced opening the file in &quot;wb&quot; mode the
2nd time; result:</font><font size=3D3> </font><font size=3D2 face=3D"sans-=
serif"><br>
real &nbsp; &nbsp;2m23.710s</font><font size=3D3> </font><font size=3D2 fac=
e=3D"sans-serif"><br>
user &nbsp; &nbsp;1m5.710s</font><font size=3D3> </font><font size=3D2 face=
=3D"sans-serif"><br>
sys &nbsp; &nbsp; 0m20.850s</font><font size=3D3> <br>
</font><font size=3D2 face=3D"sans-serif"><br>
Next test; force mode &quot;wb+&quot; is also very slow.</font><font size=
=3D3>
<br>
</font><font size=3D2 face=3D"sans-serif"><br>
I tested opening in append modes (followed by a reset of the file pointer),
but that fails to give correct output.</font><font size=3D3> <br>
<br>
<br>
</font><font size=3D2 face=3D"sans-serif"><br>
So, just changing the fopen mode results in a performance penalty of a
factor 7.5...</font><font size=3D3> </font><font size=3D2 face=3D"sans-seri=
f"><br>
Interesting observation...</font><font size=3D3> <br>
<br>
</font><font size=3D2 face=3D"sans-serif"><br>
Harold, any suggestion how to keep the file open, and still maintain a
simple interface</font><font size=3D3> </font><font size=3D2 face=3D"sans-s=
erif"><br>
for both *nix and win32/cygwin?</font><font size=3D3> <br>
<br>
</font><font size=3D2 face=3D"sans-serif"><br>
Regards, Bram.</font><font size=3D3> <br>
</font><font size=3D2 face=3D"sans-serif"><br>
PS. Gerard, maybe this is related to your observations?</font><font size=3D=
3>
</font><font size=3D2 face=3D"sans-serif"><br>
 &nbsp; &nbsp; &nbsp; &nbsp;What version did you use?</font><font size=3D3>
</font><font size=3D2 face=3D"sans-serif"><br>
 &nbsp; &nbsp; &nbsp; &nbsp;You can subscribe this mailing list at</font><f=
ont size=3D3>
</font><font size=3D2 face=3D"sans-serif"><br>
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;https://lists.sourc=
eforge.net/lists/listinfo/pfspd-users<br>
<br>
--<br>
A.K. (Bram) Riemens<br>
Principal Scientist, DSP group, Philips Research<br>
Office: WO-p-94, Postbox WO02<br>
High Tech Campus 36 (WO), 5656 AE Eindhoven, The Netherlands<br>
Tel: +31 40 27 43833, Fax: +31 40 27 44675<br>
E-mail: bra...@ph...</font><font size=3D3> <br>
<br>
</font>
<table width=3D100%>
<tr valign=3Dtop>
<td width=3D43%><font size=3D3><br>
<br>
<br>
<br>
</font><font size=3D1 face=3D"sans-serif"><b><br>
Rene van der Vleuten</b> </font>
<p><font size=3D1 face=3D"sans-serif">Sent by:</font><font size=3D3> </font=
><font size=3D1 face=3D"sans-serif"><br>
pfs...@li...</font><font size=3D3> </font>
<p><font size=3D1 face=3D"sans-serif">08-06-2006 13:04</font><font size=3D3>
</font>
<td width=3D56%>
<br>
<table width=3D100%>
<tr valign=3Dtop>
<td width=3D23%>
<div align=3Dright><font size=3D1 face=3D"sans-serif">To</font></div>
<td width=3D76%><font size=3D1 face=3D"sans-serif">pfs...@li...urce=
forge.net</font><font size=3D3>
</font>
<tr valign=3Dtop>
<td>
<div align=3Dright><font size=3D1 face=3D"sans-serif">cc</font></div>
<td>
<tr valign=3Dtop>
<td>
<div align=3Dright><font size=3D1 face=3D"sans-serif">Subject</font></div>
<td><font size=3D1 face=3D"sans-serif">[Pfs...@sf...] performance pro=
blems</font><font size=3D3>
</font>
<tr>
<td>
<div align=3Dright><font size=3D1 face=3D"sans-serif">Classification</font>=
</div>
<td></table>
<br>
<br>
<table width=3D100%>
<tr valign=3Dtop>
<td width=3D49%>
<td width=3D50%></table>
<div align=3Dright>
<br></div></table>
<br><font size=3D3><br>
<br>
</font><font size=3D2 face=3D"sans-serif"><br>
<br>
Dear Bram, Harold,</font><font size=3D3> </font><font size=3D2 face=3D"sans=
-serif"><br>
<br>
I am suffering from huge performance problems with &quot;pts translate
file.m2v file.yuv&quot;; processing takes more than 10x longer than with
the latest released pts!! I suspect these are caused by your latest updates
to cpfspd (since I did not see any obviously related changes in pts).</font=
><font size=3D3>
</font><font size=3D2 face=3D"sans-serif"><br>
<br>
Could you please look into this?</font><font size=3D3> </font><font size=3D=
2 face=3D"sans-serif"><br>
<br>
Thanks!</font><font size=3D3> </font><font size=3D2 face=3D"sans-serif"><br>
<br>
Ren=E9 <br>
<br>
--------------------------------------------------------------------------<=
br>
Dr. Rene J. van der Vleuten<br>
Philips Research Laboratories<br>
High Tech Campus 36<br>
5656 AE &nbsp;Eindhoven<br>
The Netherlands<br>
<br>
Phone: +31 40 2742941<br>
Fax: &nbsp; +31 40 2742630<br>
mailto:Ren...@ph...</font><font size=3D3> </font><font =
size=3D2><tt>=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F<br>
Pfspd-users mailing list<br>
Pfs...@li...<br>
https://lists.sourceforge.net/lists/listinfo/pfspd-users</tt></font><font s=
ize=3D3><br>
</font><font size=3D2><tt>=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F<br>
Pfspd-users mailing list<br>
Pfs...@li...<br>
https://lists.sourceforge.net/lists/listinfo/pfspd-users<br>
</tt></font>
<br>

Re: [Pfspd-users@sf.net] performance problems

[Bram R. <bra...@ph...>]

Rene, others,

After obtaining a test case from Rene, I did some investigations on the=20
current development
repository of cpfspd and pts.
Test conditions:
- linux executable, running on dimholt
- output to scratch disk on muscules file server
- command "pts translate corvette=5F8mbps.m2v corvette=5F8mbps.yuv"
- result is file of 3.6Gbyte; 1198 frames 1920x1080

Currently (development version of cpfspd), file is opened in mode "wb" for =

allocation,
then it is closed. With pts translate on mpeg, the file size is not known=20
in advance,
so it is initially created with size 22528 bytes.
After that, it is reopened in "rb+" mode. Then images are written into the =

file.
Result of "time":
real    18m11.415s
user    1m10.260s
sys     1m2.590s

As a special test, I forced opening the file in "wb" mode the 2nd time;=20
result:
real    2m23.710s
user    1m5.710s
sys     0m20.850s

Next test; force mode "wb+" is also very slow.

I tested opening in append modes (followed by a reset of the file=20
pointer), but that fails to give correct output.



So, just changing the fopen mode results in a performance penalty of a=20
factor 7.5...
Interesting observation...


Harold, any suggestion how to keep the file open, and still maintain a=20
simple interface
for both *nix and win32/cygwin?


Regards, Bram.

PS. Gerard, maybe this is related to your observations?
        What version did you use?
        You can subscribe this mailing list at
                https://lists.sourceforge.net/lists/listinfo/pfspd-users

--
A.K. (Bram) Riemens
Principal Scientist, DSP group, Philips Research
Office: WO-p-94, Postbox WO02
High Tech Campus 36 (WO), 5656 AE Eindhoven, The Netherlands
Tel: +31 40 27 43833, Fax: +31 40 27 44675
E-mail: bra...@ph...








Rene van der Vleuten=20
Sent by:
pfs...@li...
08-06-2006 13:04

To
pfs...@li...
cc

Subject
[Pfs...@sf...] performance problems
Classification








Dear Bram, Harold,=20

I am suffering from huge performance problems with "pts translate file.m2v =

file.yuv"; processing takes more than 10x longer than with the latest=20
released pts!! I suspect these are caused by your latest updates to cpfspd =

(since I did not see any obviously related changes in pts).=20

Could you please look into this?=20

Thanks!=20

Ren=E9=20

--------------------------------------------------------------------------
Dr. Rene J. van der Vleuten
Philips Research Laboratories
High Tech Campus 36
5656 AE  Eindhoven
The Netherlands

Phone: +31 40 2742941
Fax:   +31 40 2742630
mailto:Ren...@ph...=20
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
Pfspd-users mailing list
Pfs...@li...
https://lists.sourceforge.net/lists/listinfo/pfspd-users

[Pfspd-users@sf.net] performance problems

[Rene v. d. V. <ren...@ph...>]

<br><font size=3D2 face=3D"sans-serif">Dear Bram, Harold,</font>
<br>
<br><font size=3D2 face=3D"sans-serif">I am suffering from huge performance
problems with &quot;pts translate file.m2v file.yuv&quot;; processing takes
more than 10x longer than with the latest released pts!! I suspect these
are caused by your latest updates to cpfspd (since I did not see any obviou=
sly
related changes in pts).</font>
<br>
<br><font size=3D2 face=3D"sans-serif">Could you please look into this?</fo=
nt>
<br>
<br><font size=3D2 face=3D"sans-serif">Thanks!</font>
<br>
<br><font size=3D2 face=3D"sans-serif">Ren=E9 </font>
<br><font size=3D2 face=3D"sans-serif"><br>
--------------------------------------------------------------------------<=
br>
Dr. Rene J. van der Vleuten<br>
Philips Research Laboratories<br>
High Tech Campus 36<br>
5656 AE &nbsp;Eindhoven<br>
The Netherlands<br>
<br>
Phone: +31 40 2742941<br>
Fax: &nbsp; +31 40 2742630<br>
mailto:Ren...@ph...</font>

Re: [Pfspd-users@sf.net] File fragmentation

[Bram R. <bra...@ph...>]

Hi Harold, others,

After some more testing ... I found on win32/cygwin:
An existing large file remained large after p_write_header and was only 
truncated when closing the file.
Hence, an existing fragmented file remained fragmented.

This is resolved by a different CreateFile flag in cpfspd_fio.c.
Now, the file is created as if it did not exist, hence also new disk space 
is allocated with p_write_header.

Solution is in CVS repository.

Regards, Bram.

--
A.K. (Bram) Riemens
Principal Scientist, DSP group, Philips Research
Office: WO-p-94, Postbox WO02
High Tech Campus 36 (WO), 5656 AE Eindhoven, The Netherlands
Tel: +31 40 27 43833, Fax: +31 40 27 44675
E-mail: bra...@ph...








Harold Schmeitz 
Sent by:
pfs...@li...
22-05-2006 09:04

To
pfs...@li...
cc

Subject
Re: [Pfs...@sf...] File fragmentation
Classification








Hello, 

Thanks to Bram's rigorous testing, we uncovered a problem with the win32 
file I/O. 

In overlapped (asynchronous) mode, reserving the file space does not 
complete immediately, 
but it appears that some data (zeros ?) is written to disk. On a local 
file system, this takes some time, but works ok. 
However,on a network share (e.g. writing remotely to a streamer pc), there 
is a timeout after 2-3 GB, and the application crashes. 

Workaround is to create a file using regular I/O, reserve disk space, 
close it, then open again in overlapped mode. 
The disk space allocation is moved to p_fio_allocate( ). 

Please note that this only affects you if you are using the development 
version of cpfspd (sf.net), so update your cvs sandbox. 

Regards, 
Harold
___________________________________
Ing. H.A.W. Schmeitz
Video Processing Systems, Philips Research Laboratories

High Tech Campus 36, 5656 AE Eindhoven, The Netherlands
Tel. +31 (0)40 27 46489
har...@ph...