Menu
▾
▴
[SoX-devel] [ sox-Bugs-3558019 ] Wav header incorrect when 4 GB file size exceeded
|
From: SourceForge.net <no...@so...> - 2012-09-02 19:28:36
|
Bugs item #3558019, was opened at 2012-08-15 12:08 Message generated for change (Comment added) made by mhartzel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=110706&aid=3558019&group_id=10706 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Mikael Hartzell (mhartzel) Assigned to: Nobody/Anonymous (nobody) Summary: Wav header incorrect when 4 GB file size exceeded Initial Comment: Bug: Wav file header gets incorrect information when 4 GB file size is exceeded -------------------------------------------------------------------------------------------------------------------- OS: Ubuntu 12.04 Version: Sox version v14.4.2 built from latest git commit (dated 2012.08.07): 35eebaf602ffd1477da1b604d345fab37aa8d42c How to reproduce bug --------------------- - Create two 6 channel 48 kHz, 16 bit audio files, one with duration of 2 hours 4 minutes and the other with 2 hours and 5 minutes. - The latter will exeed 4 GB barrier and wav header file duration and sample count information gets mixed up. - Get file duration from both files with sox --i and ffprobe (is included in FFmpeg). - Sox --i will tell file durations are: 02:04:00.00 and 00:00:43.46 though they should both be 02:04:00.00. Also sample count is from in latter file. - ffprobe will tell both files are 02:04:00.00, I don't know how it gets the information correctly since sox --i does not. - VLC will only play 00:00:43.46 of the latter file. - The wav header gets mixed up also when converting from flac to wav with sox if the file size exceeds 4 GB. Commands to reproduce the bug ----------------------------- - Create test files by running both of these commands: sox -V -V -n -b 16 testfile-duration_124min_6_channels.wav trim 0 02:04:00.00 remix 0 0 0 0 0 0 sox -V -V -n -b 16 testfile-duration_125min_6_channels.wav trim 0 02:05:00.00 remix 0 0 0 0 0 0 - Check with ls -l that the file size of both files seems to be correct. - Use sox --i on both files to get file size. - Use ffprobe on both files to get file size. Mikael Hartzell Text output from my test. Shell commands are underlined: sox -V -V -n -b 16 testfile-duration_124min_6_channels.wav trim 0 02:04:00.00 remix 0 0 0 0 0 0 ; sox -V -V -n -b 16 testfile-duration_125min_6_channels.wav trim 0 02:05:00.00 remix 0 0 0 0 0 0 ---------------------------------------------------------------------------------------------------------------------- sox: SoX v14.4.2 time: Aug 15 2012 20:58:12 uname: Linux mika-HP-Compaq-nc6320-EV073AV 3.2.0-29-generic-pae #46-Ubuntu SMP Fri Jul 27 17:25:43 UTC 2012 i686 compiler: gcc 4.6.3 arch: 1248 48 44 L OMP sox INFO nulfile: sample rate not specified; using 48000 Input File : '' (null) Channels : 1 Sample Rate : 48000 Precision : 32-bit sox DBUG wav: Writing Wave file: Microsoft PCM format, 6 channels, 48000 samp/sec sox DBUG wav: 576000 byte/sec, 12 block align, 16 bits/samp Output File : 'testfile-duration_124min_6_channels.wav' Channels : 6 Sample Rate : 48000 Precision : 16-bit Sample Encoding: 16-bit Signed Integer PCM Endian Type : little Reverse Nibbles: no Reverse Bits : no Comment : 'Processed by SoX' sox DBUG remix: 0: sox DBUG remix: 1: sox DBUG remix: 2: sox DBUG remix: 3: sox DBUG remix: 4: sox DBUG remix: 5: sox INFO sox: effects chain: input 48000Hz 1 channels (multi) 32 bits unknown length sox INFO sox: effects chain: trim 48000Hz 1 channels (multi) 32 bits 02:04:00.00 sox INFO sox: effects chain: remix 48000Hz 6 channels (multi) 32 bits 02:04:00.00 sox INFO sox: effects chain: dither 48000Hz 6 channels 16 bits 02:04:00.00 sox INFO sox: effects chain: output 48000Hz 6 channels (multi) 16 bits 02:04:00.00 sox DBUG sox: start-up time = 0.001637 sox DBUG input: output buffer still held 2048 samples; dropped. sox DBUG wav: Finished writing Wave file, 4285440000 data bytes 357120000 samples sox: SoX v14.4.2 time: Aug 15 2012 20:58:12 uname: Linux mika-HP-Compaq-nc6320-EV073AV 3.2.0-29-generic-pae #46-Ubuntu SMP Fri Jul 27 17:25:43 UTC 2012 i686 compiler: gcc 4.6.3 arch: 1248 48 44 L OMP sox INFO nulfile: sample rate not specified; using 48000 Input File : '' (null) Channels : 1 Sample Rate : 48000 Precision : 32-bit sox DBUG wav: Writing Wave file: Microsoft PCM format, 6 channels, 48000 samp/sec sox DBUG wav: 576000 byte/sec, 12 block align, 16 bits/samp Output File : 'testfile-duration_125min_6_channels.wav' Channels : 6 Sample Rate : 48000 Precision : 16-bit Sample Encoding: 16-bit Signed Integer PCM Endian Type : little Reverse Nibbles: no Reverse Bits : no Comment : 'Processed by SoX' sox DBUG remix: 0: sox DBUG remix: 1: sox DBUG remix: 2: sox DBUG remix: 3: sox DBUG remix: 4: sox DBUG remix: 5: sox INFO sox: effects chain: input 48000Hz 1 channels (multi) 32 bits unknown length sox INFO sox: effects chain: trim 48000Hz 1 channels (multi) 32 bits 02:05:00.00 sox INFO sox: effects chain: remix 48000Hz 6 channels (multi) 32 bits 02:05:00.00 sox INFO sox: effects chain: dither 48000Hz 6 channels 16 bits 02:05:00.00 sox INFO sox: effects chain: output 48000Hz 6 channels (multi) 16 bits 02:05:00.00 sox DBUG sox: start-up time = 0.001569 sox DBUG input: output buffer still held 5632 samples; dropped. sox DBUG wav: Finished writing Wave file, 25032704 data bytes 360000000 samples ls -l testfile-duration_12* ---------------------------- -rw-rw-r-- 1 mika mika 4285440080 elo 15 21:10 testfile-duration_124min_6_channels.wav -rw-rw-r-- 1 mika mika 4320000080 elo 15 21:15 testfile-duration_125min_6_channels.wav sox --i testfile-duration_12* ----------------------------- Input File : 'testfile-duration_124min_6_channels.wav' Channels : 6 Sample Rate : 48000 Precision : 16-bit Duration : 02:04:00.00 = 357120000 samples ~ 558000 CDDA sectors File Size : 4.29G Bit Rate : 4.61M Sample Encoding: 16-bit Signed Integer PCM Input File : 'testfile-duration_125min_6_channels.wav' Channels : 6 Sample Rate : 48000 Precision : 16-bit Duration : 00:00:43.46 = 2086058 samples ~ 3259.47 CDDA sectors File Size : 4.32G Bit Rate : 795M Sample Encoding: 16-bit Signed Integer PCM Total Duration of 2 files: 02:04:43.46 ffprobe testfile-duration_124min_6_channels.wav ------------------------------------------------ avprobe version 0.8.3-4:0.8.3-0ubuntu0.12.04.1, Copyright (c) 2007-2012 the Libav developers built on Jun 12 2012 16:37:58 with gcc 4.6.3 [wav @ 0x92e4aa0] max_analyze_duration reached Input #0, wav, from 'testfile-duration_124min_6_channels.wav': Duration: 02:04:00.00, bitrate: 4608 kb/s Stream #0.0: Audio: pcm_s16le, 48000 Hz, 5.1, s16, 4608 kb/s ffprobe testfile-duration_125min_6_channels.wav ------------------------------------------------ avprobe version 0.8.3-4:0.8.3-0ubuntu0.12.04.1, Copyright (c) 2007-2012 the Libav developers built on Jun 12 2012 16:37:58 with gcc 4.6.3 [wav @ 0x8555aa0] max_analyze_duration reached Input #0, wav, from 'testfile-duration_125min_6_channels.wav': Duration: 02:05:00.00, bitrate: 4608 kb/s Stream #0.0: Audio: pcm_s16le, 48000 Hz, 5.1, s16, 4608 kb/s ---------------------------------------------------------------------- Comment By: Mikael Hartzell (mhartzel) Date: 2012-09-02 12:28 Message: Hi :) Thanks for replying :) Yes I am aware that the 'old' wav format has a hard limit of 4 GB, but the way sox handles this limit is in my opinion a little troublesome. Imagine a scenario where you convert a 8 channel flac file to wav. It is very possible that the output file size may exceed wav format max limit of 4 GB. Sox does not enforce the limit but instead lets the header become invalid. Most of the data in the file becomes not playable in standard players. When sox is used in a script to automate file conversion, this error goes unnoticed until somebody tries to open and play the file. May I humbly suggest two possible workarounds :) 1. When writing wav output sox checks for the 4 GB limit, and stops copying any more data to the ouput file when the limit is reached. Sox should display an error message informing the user that the output file was truncated. 2. When writing wav output sox checks for the 4 GB limit and when it is crossed sox writes a w64 header to the file and continues putting more data to the output file. In my opinion it is very important to keep the header in a sane condition and in sync with the data in the file. Suggestion 1 would be fairly easy to implement and should prevent creating invalid files without the user noticing. I built another open source project that uses your program and I have just now become to know sox better and I am very impressed what all it can do :) So Thanks guys for your great work :) ---------------------------------------------------------------------- Comment By: Ulrich Klauer (uklauer) Date: 2012-08-30 08:23 Message: Well, there is no way to do this right. RIFF/WAVE only uses 32 bits to store lengths, so anything >4 GiB won't fit. This is comparable to a paper form that asks for your age, but only offerss two places for the digits; if you're 102 years old, you might write "02" or "99", but you can't specify your true age. We should perhaps choose the equivalent of "99". But the only way to completely avoid the problem is for you to switch to a different file format like W64 or FLAC. ---------------------------------------------------------------------- Comment By: Mikael Hartzell (mhartzel) Date: 2012-08-15 12:13 Message: typos creaped in the bug report, corrected lines: - Sox --i will tell file durations are: 02:04:00.00 and 00:00:43.46 though they should be 02:04:00.00 and 02:05:00.00. Also sample count is wrong in latter file. - ffprobe will tell files are 02:04:00.00 and 02:05:00.00, I don't know how it gets the information correctly since sox --i does not. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=110706&aid=3558019&group_id=10706 |