bmx frames the audio data according to a fixed pattern of audio sample counts for 29.97Hz: 1602, 1601, 1602, 1601, 1602. If the audio sample count is out by 1 sample, e.g. 1602 is required but only 1601 available, then the frame is not written. The audio sample count could be off by 1 if the input audio followed a different pattern or starting point in the pattern.
I'm guessing this is happening in your case. The audio track duration is 173722 and the video track duration is probably 173723. The clip duration in MXF is the minimum track duration and that's why raw2bmx reports 173722.
I suppose raw2bmx could be made smarter by examining the input size to work out the pattern (with the disadvantage that a known output pattern is not guaranteed) or pad with zeros if it is out by a small amount (with the disadvantage that what comes out isn't identical to what came in). I don't know what the common practice is in this situation for 29.97Hz production workflows.
Philip
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have exactly 278234756 samples in the wav for 01:36:36.557 duration or 5796557 ms
So if I follow your explanation, i could try that :
(278234756 / 48000) * (30000 / 1001) and it gives me : 173722.999.... miss 2 samples
So, I guess you're right !
Thank you for your explanation.
Damned 29.97...
Now, is there something i could do to 'trick' bmx ?
We could loose the last frame... it's not a breaking thing... but if we could keep it, it's better for our quality check.
We have 2000 hours of video to compress this way so this kind of decisions is hard to take :)
Grégory
Last edit: Artix 2015-06-04
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
If you calculate it like this, 278234756 % (1602+1601+1602+1601+1602) = 4804, you'll see that 4804 - (1602+1601) = 1601, and because the third count in the sequence is 1602 you'll be missing one sample to complete the frame.
You could 'trick' bmx by extending the input wav files by say 800 samples of silence, which would result in the last frame being included if it is at least 801 or 802 samples long. In your case it would result in one extra audio sample.
I'm thinking that either bmx should allow a short last audio frame (e.g. 1601 in your case) or add padding to complete it (1 sample in your case). Either approach could be selectable by a user option.
Philip
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
This sample count mismatch issue also affects progressive 59.94 files too. The pattern there however is 801, 801, 801, 801, 800.
Besides missing the last frame, another problem is that video and audio can get out of the original sync when concatenating multiple files (say, using bmxtranswrap).
As far as I understand, audio data gets concatenated and then separated into elements as per pattern while video elements simply get copied. In a two file concatenation operation, the second file will be affected by the first file having missing samples at the end. The second file will lose its last frame and existing frames may be up to around 30ms off sync with audio in worst case scenario (i.e. last audio frame contains less than 100 samples; ffmpeg allows such files to be created).
As you can guess the situation will become linearly worse with more "faulty" files introduced and it won't be those bad files that will be necessarily taking the hit.
Overall, I think padding to fix pattern issues is a more stable solution.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I agree that padding is defnitely less bad than dropping a frame but it does feel a bit unsatisfactory that what comes out of an MXF file isn't exactly what went in. However, maybe that's an accepted compromise to enable fixed audio framing patterns.
Philip
btw, the pattern bmx uses for 59.94Hz is 801, 801, 800, 801, 801
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
801,801,801,801,800 starting pattern is used by libav/ffmpeg when encoding the pcm stream for mxf. Stream copy preserves starting pattern (unless file is being split).
800,801,801,801,801 starting pattern is used by Sony's XDCAM mxf writer.
The 29.97 patterns are "normal" for both however (same as bmx).
Actually, even concatenating files with properly implemented 29.97 pattern can result in losing a frame. As the pcm stream is repatternized, at the end of the stream 1602 samples might be expected as per pattern while only 1601 available (the reverse can never come up, i.e. 1602 available, 1601 expected). Padding an additional sample at the end is obviously the only way out of this.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hello,
I'm trying to wrap an avc-intra 100 1080i with 2 stereo stream into an MXF.
Source is mov uncompressed v210 10bit 422 with 2 stereo stream pcm_s24le
Avc-intra is compressed from v210 with ffmpeg.
x264opts=interlaced=1:tune=psnr:force-cfr=1:avcintra-class=100:colorprim=bt709:transfer=bt709:colormatrix=bt709:fullrange=on
ffmpeg-10bit -y -i input.mov -vcodec libx264 -flags +ildct+ilme -tune psnr -x264-params $x264opts -pix_fmt yuv422p10le -an output.h264
stereo stream is extracted from mov with ffmpeg.
Then mxf wrapping :
raw2bmx -y 09:58:03:08 -f 2997 -t op1a --ard-zdf-hdf -o output.mxf --afd 8 --avci100_1080i output.h264 --wave input.01.wav --wave input.02.wav
I successfully have wrapped ~30 files in 25 fps without problem.
Now I try my process on 29.97fps and on the first 4 samples, i have two that are missing a frame.
I have tested with ffprobe all my video streams.
ffprobe -v error -count_frames -select_streams v:0 -show_entries stream=nb_read_frames -of default=nokey=1:noprint_wrappers=1 [input]
mov : 173723 frames
h264 : 173723 frames
mxf : 173722 frames
And the output from raw2bmx is :
Info: Duration: 173722 (01:36:30:22 @30fps)
I have tried with bmx-snapshot-20141125 and bmx-snapshot-20150603
Any idea what's going on ?
Thanks
bmx frames the audio data according to a fixed pattern of audio sample counts for 29.97Hz: 1602, 1601, 1602, 1601, 1602. If the audio sample count is out by 1 sample, e.g. 1602 is required but only 1601 available, then the frame is not written. The audio sample count could be off by 1 if the input audio followed a different pattern or starting point in the pattern.
I'm guessing this is happening in your case. The audio track duration is 173722 and the video track duration is probably 173723. The clip duration in MXF is the minimum track duration and that's why raw2bmx reports 173722.
I suppose raw2bmx could be made smarter by examining the input size to work out the pattern (with the disadvantage that a known output pattern is not guaranteed) or pad with zeros if it is out by a small amount (with the disadvantage that what comes out isn't identical to what came in). I don't know what the common practice is in this situation for 29.97Hz production workflows.
Philip
I have exactly 278234756 samples in the wav for 01:36:36.557 duration or 5796557 ms
So if I follow your explanation, i could try that :
(278234756 / 48000) * (30000 / 1001) and it gives me : 173722.999.... miss 2 samples
So, I guess you're right !
Thank you for your explanation.
Damned 29.97...
Now, is there something i could do to 'trick' bmx ?
We could loose the last frame... it's not a breaking thing... but if we could keep it, it's better for our quality check.
We have 2000 hours of video to compress this way so this kind of decisions is hard to take :)
Grégory
Last edit: Artix 2015-06-04
If you calculate it like this, 278234756 % (1602+1601+1602+1601+1602) = 4804, you'll see that 4804 - (1602+1601) = 1601, and because the third count in the sequence is 1602 you'll be missing one sample to complete the frame.
You could 'trick' bmx by extending the input wav files by say 800 samples of silence, which would result in the last frame being included if it is at least 801 or 802 samples long. In your case it would result in one extra audio sample.
I'm thinking that either bmx should allow a short last audio frame (e.g. 1601 in your case) or add padding to complete it (1 sample in your case). Either approach could be selectable by a user option.
Philip
Ok, we will pad our videos with empty samples.
I have already done a test and it work for our first video.
Thank you very much for your help !
(hopefully I'm not reviving too old of a thread)
This sample count mismatch issue also affects progressive 59.94 files too. The pattern there however is 801, 801, 801, 801, 800.
Besides missing the last frame, another problem is that video and audio can get out of the original sync when concatenating multiple files (say, using bmxtranswrap).
As far as I understand, audio data gets concatenated and then separated into elements as per pattern while video elements simply get copied. In a two file concatenation operation, the second file will be affected by the first file having missing samples at the end. The second file will lose its last frame and existing frames may be up to around 30ms off sync with audio in worst case scenario (i.e. last audio frame contains less than 100 samples; ffmpeg allows such files to be created).
As you can guess the situation will become linearly worse with more "faulty" files introduced and it won't be those bad files that will be necessarily taking the hit.
Overall, I think padding to fix pattern issues is a more stable solution.
I agree that padding is defnitely less bad than dropping a frame but it does feel a bit unsatisfactory that what comes out of an MXF file isn't exactly what went in. However, maybe that's an accepted compromise to enable fixed audio framing patterns.
Philip
btw, the pattern bmx uses for 59.94Hz is 801, 801, 800, 801, 801
801,801,801,801,800 starting pattern is used by libav/ffmpeg when encoding the pcm stream for mxf. Stream copy preserves starting pattern (unless file is being split).
800,801,801,801,801 starting pattern is used by Sony's XDCAM mxf writer.
The 29.97 patterns are "normal" for both however (same as bmx).
Actually, even concatenating files with properly implemented 29.97 pattern can result in losing a frame. As the pcm stream is repatternized, at the end of the stream 1602 samples might be expected as per pattern while only 1601 available (the reverse can never come up, i.e. 1602 available, 1601 expected). Padding an additional sample at the end is obviously the only way out of this.