Re: [Pfspd-users@sf.net] Linux, cygwin, and different yuv file sizes

[Bram R. <bra...@ph...>]

Hi Erwin, others,

On windows platforms (cygwin, win32) we use high performance platform 
specific file I/O.
When the OS version (windows NT, 2000, XP) and the file system (ntfs) 
supports it,
unbuffered and asynchronous I/O is used (this is detected at runtime).
As an example, the underlaying I/O system performs the read DMA of a next 
block
while the application is processing the current block.
This I/O is optimized for sequential image access and allows us to process 
video
data with only a single memcpy operation between an cpfspd internal buffer 
and
the application. On win32, this memcpy is optimized using MMX operations.
The result is a data throughput close to the theoretical limits of the 
harddisk
with a very low CPU load overhead.

This is only possible with I/O blocks that are a multiple of the disk 
block size.
We found that a block size of 256 kbyte provides optimal performance in 
most cases.

As a side effect of this approach, the file size is always a multiple of 
256 kbyte
on cygwin/win32. So you are doing nothing wrong.

When we want to perform checks against expected output, we typically use
an internal tool "pts checksum". This provides a checksum of the file 
contents,
thus ignores the unused file space.

Regards, Bram.

PS. The buffer size can be changed with p_set_file_buf_size().

PPS. As a solution, we could adapt the file size when closing the file.
Erwin, are you able to help with this change?

PPPS. I'll make a FAQ page at our website. This question is not new.

--
A.K. (Bram) Riemens
Principal Scientist, DSP group, Philips Research
Office: WO-p-94, Postbox WO02
High Tech Campus 36 (WO), 5656 AE Eindhoven, The Netherlands
Tel: +31 40 27 43833, Fax: +31 40 27 44675
E-mail: bra...@ph...








Erwin de Kock 
Sent by:
pfs...@li...
26-01-2006 19:09

To
pfs...@li...
cc

Subject
[Pfs...@sf...] Linux, cygwin, and different yuv file sizes
Classification








Hi all, 

I noticed that the cpfsp p_write_header function generates files that have 
different sizes under Linux and Cygwin. I use the following code: 

#include <stdlib.h> 
#include "cpfspd.h" 

int main() 
{ 
    pT_header seqInfo; 
    pT_status status; 

    if ((p_create_ext_header(&seqInfo, P_COLOR_420_PL, P_50HZ, P_SD, 720, 
1, P_16_9) != P_OK) 
         || (p_mod_image_size(&seqInfo, 704, 576) != P_OK) 
         || (p_mod_num_frames(&seqInfo, 7) != P_OK) 
       ) 
    { 
      printf("Error, cannot create pfspd header\n"); 
      exit(1); 
    } 
 
    status=p_write_header("./out.yuv", &seqInfo); 
} 

Under Linux this results in a file out.yuv of size 4280320. 
Under Cygwin this results in a file out.yuv of size 4456448. 

If I write yuv data in the specified format to these files, then the first 
4280320 bytes of the files are identical. So it seems that the last bytes 
of the cygwin out.yuv file are not used (or maybe even more if the linux 
file size is too large as well). 

I also tried supported combinations of color format, image frequency, and 
image size such as p_create_ext_header(&seqInfo, P_COLOR_420_PL, P_50HZ, 
P_SD, 720, 1, P_4_3) without changing the image size later on, but this 
also gives different file sizes. 

The different files are a problem because I want to check the output of my 
program against expected output. I prefer that the expected output does 
not depend on the platform. 

What am I doing wrong? 

Thanks for any help, 
Erwin de Kock