#329 fmfconv new features (JPEG/PNG/AVI/PIPE output)

[Gergely Szasz]

This patch add some missing (and badly needed ;) features to fmfconv:

• Internal sound resampling
• greyscale output (BW TV)
• pipe output (YUV4MPEG2/WAV) to external program (ffmpeg/avcodec)
• JPEG (M-JPEG and AVI[MJPEG/PCM_S26LE]) output
• PNG output

Details:

• configure.ac:
  • add check to pipe(), fdopen() for pipe output
  • add check to signal() for CTRL+C to stop fmfconv gracefully
  • add check libjpeg and libpng
• Makefile.am:
  • add fmfconv_{jpg,png}.c as conditional source file to fmfconv
• fmfconv.{h,c}:
  • add #defines and variables for pipe, jpeg, png output
  • add variables to sound resampling
  • change law_2_pcm() stereo_2_mono() and fmf_read_sound() to handle sound resampling fragments
  • new subroutine: resample_sound() to do sound resample if needed
  • out_2_rgb() and out_2_yuv() replaced with out_2_pix() and pix_pix() subroutines to remove duplicates and handle grayscale conversion
  • new subroutine: open_pipe() to handle pipe output
  • change open_out() and open_snd() to handle pipe, jpeg and png outputs
  • changes to handle if only a sound file ( .wav, .aiff and .au) given in the command line
  • several changes to handle new command line args
• fmfconv_{jpg,png}.c:
  • add routines for JPEG, PNG, M-JPEG and AVI output. AVI contain M-JPEG video and PCM_S16LE (or u-/a-Law) audio. AVI files automatically splitted up about at 1GB file size limit
• fmfconv_{aiff,au,wav}.c:
  • changes to handle internal resampled sound
• fmfconv_{ppm,yuv}.c:
  • changes to handle greyscale output
• fmfconv_ff.c:
  • use printi() macro instead of fprintf()

I add PIPE output because libffmpeg/libavcodec looks too "mutable". I think if somewhere libavcodec/libffmpeg, etc installed, then avcodec/ffmpeg installed too, so, I prefer PIPE output over FFMPEG...

Some example for PIPE:

bash> fmfconv input.fmf -x ffmpeg output.mkv

convert input.fmf to output.mkv with ffmpeg (default x264 video and 32kHz/stereo vorbis audio)

bash> fmfconv --acodec opus --vcodec mpeg4 -E dat -f movie input.fmf -x avcodec output.mkv

with avcodec create a Matroska file 24 fps MPEG4 video OPUS stereo audio at 48kHz

bash> fmfconv input.fmf -x 'ffmpeg -ar 44100 -r 25 -vf scale=1024x768' test.avi

Now we use ffmpeg to resample and rescale...

bash> fmfconv --greyscale -f movie input.fmf -x 'mpv --fs --audio-file /dev/fd/%a /dev/fd/%v' dummy

Just play fullscreen movie in greyscale with mpv ... hmm, please do not seek ;-)

Other -- PIPE not just for ffmpeg/avcodec:

bash> fmfconv --avi -E cd -f movie test.fmf test.avi

Force to use internal AVI encoder (M-JPEG/PCM_S16LE) instead of FFMPEG libs

[Sergio Baldoví]

Thanks Gergely. I've had a look...

pipe output (YUV4MPEG2/WAV) to external program (ffmpeg/avcodec)

It's a nice feature. With the traditional pipe we can only pass one stream of data (audio or video), e.g.,

fmfconv input.fmf | mencoder output.fmf

So audio and data transcoding might need less steps. This will be also useful in distros that don't distribute libav (e.g., Fedora) and pack fmfconv without ffmpeg support. Unfortunately, it won't work on Windows.

I've done a test:

fmfconv input.fmf -x avconv output.mkv
*INF open_inp(): Input file (input.fmf) opened as FMF file.
*INF open_inp(): Input file (input.fmf) opened as FMF file.
avconv version 9.18-6:9.18-0ubuntu0.14.04.1, Copyright (c) 2000-2014 the Libav developers
  built on Mar 16 2015 13:19:10 with gcc 4.8 (Ubuntu 4.8.2-19ubuntu1)
Unrecognized option '-metadata'.
Error splitting the argument list: Option not found

avconv don't recognize metadata option. It seems another ffmpeg/avconv divergence?

PNG output

At a first glance, it seems a waste of space dumping repeated frames, but it turns to be useful to feed external programs, e.g., we can make animated GIFs with ImageMagick (50 frames/second):

fmfconv input.fmf tmp.png
convert -delay 2 -loop 0 -layers removeDups -layers Optimize tmp*.png output.gif

Compact size and can be used in a web page. Brilliant!

--png-compress

The build fail without zlib. Should PNG output be available if zlib is not available?

MJPEG output

The playback speed is slow (bad frame rate?) and the size is big. I guess duplicate frames are not optimized. Probably a format that I won't use.

AVI[MJPEG/PCM_S26LE output

I guess people may get disappointed with the file size but seems a good fallback format if ffmpeg is not available.

configure.ac

We should probably add --without-png and --without-jpeg options

It is a bit odd to check for png_write_png and jpeg_write_scanlines before checking for the presence of png.h and jpeglib.h.

[Gergely Szasz]

First: thanks for review.

The build fail without zlib. Should PNG output be available if zlib is not available?

Hmm.. libpng requires zlib, but not zlib-dev...

The playback speed is slow (bad frame rate?)

MJPEG not a 'real' video container. It is just a lot of JPEG image concatenated, but only the first have Huffmann tables. This is an "abbreviated jpeg stream". So, there is no any timing 'record' and there is no any way to 'optimize' frames, etc.
mplayer/mpv/ffplay use 25fps default
You can use -frate 25 with fmfconv, or tell the correct frame rate to player:

mplayer -fps 50 xxx.mjpeg
ffplay -framerate 50 xxx.mjpeg
mpv --no-correct-pts --fps=50 --demuxer-lavf-probescore=25 xxx.mjpeg

(to be continued :-)

[Gergely Szasz]

Here is a new patch (with some little improvements)

without ffmpeg support. Unfortunately, it won't work on Windows.

Now AVI output could be an alternative of PIPE (exept the lossy jpeg video stream...).

fmfconv --avi test.fmf | ffmpeg -i - test.mkv

I just cannot test it, (crosscompiled successfully, but wine has some redirect problem...)

avconv don't recognize metadata option. It seems another ffmpeg/avconv divergence?

Hmm: this is interesting..., but i remove the (incomplete) '-metadata' part...

The build fail without zlib. Should PNG output be available if zlib is not available?

fixed

We should probably add --without-png and --without-jpeg options

done

It is a bit odd to check for png_write_png and jpeg_write_scanlines before checking for the presence of png.h and jpeglib.h.

fixed :)

Note: On windows (exactly on wine) fmfconv does not unlink the temporary index file of AVI output.
Note2: Hmm... If we can encode uncompressed video frames into AVI (DIB) it could be a real alternative for PIPE output ...

[Gergely Szasz]

So, here is a new patch:

Now fmfconv can produce uncompressed AVI (free from compression artifacts).
When output to file, default AVI/M-JPEG, when output to stdout (pipe or redirect) by default create AVI/DIB (Device Independent Bitmap - what a nice name ;-) video.

e.g.:

> fmfconv --avi --frate 25 test.fmf | ffmpeg -i - test.mkv

Alternative way to convert FMF->MKV(H264/Vorbis by default)

> fmfconv --avi --frate 25 test.fmf test.avi

Create an 'old' AVI(M-JPEG/PCM_S16LE) file.

> fmfconv --greyscale --avi --frate 25 test.fmf | mplayer -demuxer lavf -fs -

View grayscale video in fullscreen.
Note: mplayer own demuxer fail with top-bottom encoded (frame height <0) DIB frames, so we need to force libavformat demuxer.

> fmfconv -Q 20 --jpg-smooth 25 --avi-mjpeg --frate 25 test.fmf | mpv -

View forced low quality AVI/M-JPEG video. We can see beautiful compression artifacts.

So... we may drop the whole PIPE thing.

[Gergely Szasz]

Here is a new (more polished) patch:

• I removed the PIPE part, because AVI replaces it.
• Now fmfconv refuses to overwrite existing files, you should use -y to force (a'la ffmpeg/avconv)
• for video outputs (mjpeg/avi/yuv4mpeg2) fmfconv set output fps to 25 (pal timing), if we want to preserve the original framerate, we have to use `--frate raw' option

So... we may drop the libavconv/libffmpeg part too...(?)

[Sergio Baldoví]

Here is a new (more polished) patch

fmfconv_avi, fmfconv_jpg and fmfconv_png files are missing in patch 6. I've taken them from patch 5. Are they the same?

fopen_overwr: for detecting an existing file 'access' seems a better way than opening for read. See file.c in Fuse.

fopen_overwr: S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH flags are missing on Windows. From the man page:

Any created files will have mode S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP| S_IROTH | S_IWOTH (0666)

Are they redundant or should we use compat alternatives?

fopen_overwr: on Windows it is needed the binary mode or the output is corrupt.

Test1:

fmfconv --avi | ffmpeg -i - output.mkv
fmfconv: option `--avi' is ambiguous
*INF open_out(): Output file ((-=stdout=-)) opened as YUV4MPEG2 - uncompressed video stream format file.

There is something wrong with the --avi option as YUV4MPEG2 mode is taken. The output file doesn't have audio.

Test2:

fmfconv --avi-uncompr input.fmf | ffmpeg -i - output.mkv
*INF open_out(): Output file ((-=stdout=-)) opened as AVI - Microsoft Audio Video Interleave format file.

That seems OK. The output file has an audio stream.

So... we may drop the libavconv/libffmpeg part too...(?)

The ffmpeg/libav code have been difficult to maintain and there are still some issues, e.g., there are audio encoders that do not accept samples in s16 format. We could kill two birds with one stone and let the hard work to ffmpeg.

fmfconv -p dvd input.fmf output.vob

would become:

fmfconv --avi-uncompr input.fmf| ffmpeg -i - -target pal-dvd output.vob

Moreover, fuse-utils for Windows currently distributes +20Mb of encoders that will likely not be used, so this sounds interesting to me.

Let's wait for other opinions to go on with this matter.

[Gergely Szasz]

fmfconv_avi, fmfconv_jpg and fmfconv_png files are missing in patch 6

Ohh... here is the whole patch...

... ...

[Gergely Szasz]

open_overwr: for detecting an existing file 'access' seems a better way

converted to use access().

Any created files will have mode S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP| S_IROTH | S_IWOTH (0666)

Are they redundant or should we use compat alternatives?

if i use open() O_CREATE|O_EXCL without mode, on linux i get quite astonishing rights ("--sr-sr-T" in the /tmp directory, "r-sr-xr--" in one of my own directory)... so i vote for some "alternative way".
Now i defined S_Ixxxx constans (GRP/OTH) if not exist...

/tmp>ls -l test*.txt
---sr-sr-T 1 gergely users 0 Mar 30 08:13 test1.txt
-rw-r--r-- 1 gergely users 0 Mar 30 08:13 test2.txt

first open() without mode, second with mode

fopen_overwr: on Windows it is needed the binary mode or the output is corrupt.

hmm..? we open binary mode with fdopen()... Uhh.. O_BINARY for open() ??? That binary/text stuff is not just for stream functions??? Ohh... what a mess, what a mess ...
BTW: fdopen()'s mode "wb" is not compatible mode with open()'s flag O_WRONLY? ... O.K. Never mind. I defined O_BINARY if not exists... ;)

There is something wrong with the --avi option as YUV4MPEG2 mode is taken.

>fmfconv --avi | ffmpeg -i - output.mkv
*ERR fmfconv: no input file specified
Try `fmfconv --help' for more information.
ffmpeg version n2.4.2 Copyright (c) 2000-2014 the FFmpeg developers
  built on Oct 27 2014 14:34:48 with gcc 4.6 (Debian 4.6.4-7)
...

Now (at least for me) it looks like o.k.
Or with input file (-:

>fmfconv --avi test2.fmf | ffmpeg -i - output.mkv
*INF open_out(): Output file ((-=stdout=-)) opened as AVI - Microsoft Audio Video Interleave format file.
*INF open_inp(): Input file (test2.fmf) opened as FMF file.
ffmpeg version n2.4.2 Copyright (c) 2000-2014 the FFmpeg developers
  built on Oct 27 2014 14:34:48 with gcc 4.6 (Debian 4.6.4-7)
...

.
.
So, here is a new patch...

Let's wait for other opinions to go on with this matter.

O.K.

---

BTW: open()/access() not conforming to C89 (at least manpage does not say: yes)... do we need to test existence of these functions somewhere? -- does it matter at all?

BTW2: hmm very (un)funny: formatting->code higliting->with shebang or three colon (:::) simply does not work...

[Stuart Brady]

Regarding the mode parameter to open(), I guess you probably know this, but man open states:

mode specifies the permissions to use in case a new file is created. This argument must be supplied when O_CREAT or O_TMPFILE is specified in flags; if neither O_CREAT nor O_TMPFILE is specified, then mode is ignored.

It seems with glibc and Linux, if you don't pass mode and you do use O_CREAT, the value of mode that is used is undefined.

POSIX defines open() as a variadic function (man 3p open if you have manpage-posix-dev on Debian), and states elsewhere (man 7p stdarg.h) that "If there is no actual next [variadic] argument [...] the behavior is undefined", and I take it from this that a conforming implementation of open() could segfault in this case.

BTW, I hadn't quite realised just what signal codes are and how detailed their categorisation of the cause of a signal is. Of course, there isn't a signal code specified by POSIX (man 7p signal.h) that would be appropriate for run-time argument checking, but apparently implementations can add their own. Linux adds a only a few which are of limited interest. So if this were to be caught by the library of some future OS, I can't even say which signal would even be used.

[Gergely Szasz]

Yeah.. the crucial part is: "This argument must be supplied when O_CREAT..." ;-)

[Sergio Baldoví]

Thanks, Stuart. The manpages-posix-dev package seems indeed interesting.

[Sergio Baldoví]

if i use open() O_CREATE|O_EXCL without mode, on linux i get quite astonishing rights ("--sr-sr-T" in the /tmp directory, "r-sr-xr--" in one of my own directory)

Got it. createhdf use similar parameters.

Now i defined S_Ixxxx constans (GRP/OTH) if not exist

They are defined in compat.h so it is better to reuse them.

O_BINARY for open() ??? That binary/text stuff is not just for stream functions???
hmm..? we open binary mode with fdopen()... Uhh.. O_BINARY for open() ??? That binary/text stuff is not just for stream functions??? Ohh... what a mess, what a mess ...

Sorry, it has been badly explained. Windows have this oddity. I've attached a new patch with a simpler fopen_overwr().

fmfconv --avi input.fmf | ffmpeg -i - output.mkv
Now (at least for me) it looks like o.k.
Or with input file (-:

I still get an ambiguous option on Windows with --avi, but --avi-uncompr and --avi-mjpeg are good. It seems to be a bug with the abbreviation of options as --avi matches two longer options. The workaround is defining --avi before --avi-uncompr and --avi-mjpeg. Fixed in the attached patch.

BTW: open()/access() not conforming to C89 (at least manpage does not say: yes)... do we need to test existence of these functions somewhere? -- does it matter at all?

Both open() and access() functions conform to SVr4, 4.3BSD and POSIX.1-2001. We try to conform to C89 but there are deviations, e.g., strdup is not C89, snprintf is C99, inline definitions are C99 and typeof is a compiler extension. The flag "-std=c89" can be passed to restrict these anomalies. IMO conforming to C89 is more what you'd call guideline than actual rule. It is desirable, though.

[Sergio Baldoví]

Please let me know if there are any comments and if not I'll commit next weekend.

[Sergio Baldoví]

Thanks Gergely. Committed in [r5199] with minor style cleanups.

[Gergely Szasz]

Thanks.

May I prepare a patch to drop ffmpeglib/avlib support?

[Sergio Baldoví]

You have just provided a convenient alternative to libav but I think we should not rush the drop, just in case someone has a different view. I'm in favour of dropping libav support before the next release, though.

Do you mind to document alternatives to current fmfconv profiles? There will be people unwilling to deepen ffmpeg/avconv options.

[Gergely Szasz]

Here is:

youtube

• old: -vf scale=w=480:h=360 -r 25 -codec:v mpeg4 -f mov -b:a 128000 -b:v 600000
• based on new recommendations: -vf scale=w=480:h=360 -r 25 -codec:v h264 -codec:a libfaac -f mp4 -b:a 128000 -b:v 600000

dvd

• -vf scale=w=720:h=576 -aspect 48/45 -r 25 -f dvd -ar 48000 -b:a 192000 -b:v 5000000
• alt: -vf scale=w=720:h=576 -aspect 48/45 -r 25 -f dvd -codec:a libfaac -ar 48000 -b:a 192000 -b:v 5000000

svcd

• -vf scale=w=480:h=576 -aspect 48/45 -r 25 -f svcd -ar 44100 -b:a 192000 -b:v 2600000

ipod

• old: -vf scale=w=320:h=240 -r 30 -codec:v h264 -f ipod -ar 44100 -b:a 128000 -b:v 256000
• alt: -vf scale=w=480:h=360 -r 30 -f ipod

We may add [some|a] shell (win?) script, to make this conversions easier...?

(Markdown sucks:
hmm.. what a weird formatting... hmm... i cannot create an unordered list #@$@#$@#$ ... hmm only preview failed ;-)

[Sergio Baldoví]

Thanks Gergely. My plan is to add more examples to the manual page, so they should work with ffmpeg/avconv in as many systems as possible. I've tested these options with ffmpeg 2.5.2 (Windows) and avconv 9.18 (Ubuntu).

For youtube profile, I've found a missing libfaac encoder. There is an experimental aac encoder in avconv and libvo_aacenc encoder in ffmpeg. I suggest using libmp3lame as an alternative or don't specify any AAC encoder. Also, the scale parameter syntax is not valid:

fmfconv --avi input.fmf | ffmpeg -i - -vf scale=480:360 -r 25
        -codec:v h264 -codec:a libmp3lame -f mov -b:a 128000
        -b:v 600000 -pix_fmt yuv420p output.mov

For DVD profile, there is a VBV buffer size that has not been set, and the same problem with AAC. I suggest using the target pal-dvd that set all parameters and choose AC3. What is the reason for the aspect option? I've get a stretched image so I've dropped it:

fmfconv --avi input.fmf | ffmpeg -i - -target pal-dvd output.vob

For SVCD profile, there are similar issues to DVD. The target pal-svcd seems a good choice:

fmfconv --avi input.fmf | ffmpeg -i - -target pal-svcd output.mpg

For ipod profile, I've added yuv420p pixel format for compatibility:

fmfconv --avi input.fmf | ffmpeg -i - -vf scale=320:240 -r 30
        -codec:v h264 -f ipod -ar 44100 -b:a 128000 -b:v 256000
        -pix_fmt yuv420p output.m4v

Any suggestions are welcome.

[Gergely Szasz]

Hello,

For youtube profile, I've found a missing libfaac encoder. There is an experimental aac encoder in avconv and libvo_aacenc encoder in ffmpeg. I suggest using libmp3lame as an alternative

Yeah, ffmpeg/avcodec can use a lot of external libraries, so in some build some of them missing. libmp3lame is the same category, it may missing too (e.g. debian) and the whole MP3 stuff could be problematic. see e.g.
legal issues on audacity.org
The foolproof choice may the mp2 or the experimental aac codec (with -strict -2 switch).

Also, the scale parameter syntax is not valid

Both ffmpeg and avconv documentation say: this syntax is valid... ?

What is the reason for the aspect option? I've get a stretched image

Because 720x576 aspect is 5:4 not 4:3. The current parameter is wrong (sorry )-: the correct parameter is -aspect 4/3.

[Sergio Baldoví]

Yeah, ffmpeg/avcodec can use a lot of external libraries, so in some build some of them missing. libmp3lame is the same category, it may missing too (e.g. debian) and the whole MP3 stuff could be problematic. see e.g. legal issues on audacity.org. The foolproof choice may the mp2 or the experimental aac codec (with -strict -2 switch).

Thanks. Both solutions should work. I will choose aac over mp2 based on this comparison:

libopus > libfdk_aac = libvorbis > libmp3lame >= libfaac >= eac3/ac3 > aac > libtwolame > vorbis > mp2 > wmav2/wmav1 > libvo_aacenc

Also, the scale parameter syntax is not valid

Both ffmpeg and avconv documentation say: this syntax is valid... ?

It looks like new syntax, with avconv 9.18 I get "Undefined constant or missing '(' in 'w=480'", but with avconv 11.3 it is OK. It seems better to stick to the "scale=480:360" syntax.

What is the reason for the aspect option? I've get a stretched image

Because 720x576 aspect is 5:4 not 4:3. The current parameter is wrong (sorry )-: the correct parameter is -aspect 4/3.

That makes sense, but I will use the "aspect 4:3" syntax because "aspect 4/3" gives me 4:1 :-)

If I'm right, the alternatives to the current profiles could be:

fmfconv --avi input.fmf | ffmpeg -i - -vf scale=480:360 -r 25
        -codec:v h264 -codec:a aac -f mov -b:a 128000
        -b:v 600000 -pix_fmt yuv420p -strict -2 output.mov

fmfconv --avi input.fmf | ffmpeg -i - -aspect 4:3 -target pal-dvd output.vob

fmfconv --avi input.fmf | ffmpeg -i - -aspect 4:3 -target pal-svcd output.mpg

fmfconv --avi input.fmf | ffmpeg -i - -vf scale=320:240 -r 30
        -codec:v h264 -codec:a aac -f ipod -ar 44100 -b:a 128000
        -b:v 256000 -pix_fmt yuv420p -strict -2 output.m4v

We may add [some|a] shell (win?) script, to make this conversions easier...?

I'm not sure about that. Shell scripts are not portable and ffmpeg/avconv could be not available. IMHO, it is more simple to point the user in the right direction.

[Gergely Szasz]

I attach a patch:

• it changes the default output type to AVI instead of YUV4MPEG2 (so we do not need to explicitly specify --avi if pipe output to other application)
• moves the --force-resample option in a #ifdef USE_FFMPEG block
• moves the 'E:' option in getopt_long() out of #ifdef USE_FFMPEG_VARS, because this option now activate the internal resampling code, so always present
• moves the case 'E' part out of the #ifdef USE_FFMPEG_VARS block
• moves the E option to the Audio options section in the man page

[Sergio Baldoví]

Thanks, committed in [r5203].

[Gergely Szasz]

Here is a patch to implement -V --version option...

BTW: I cannot manage to compile fmfconv with a new ffmpeg (version n2.6.2) installed into a nonstandard directory. Configure done without any notice, but compile time I got a lot of warnings: implicit declaration of function (av_new_stream, avcodec_open, av_set_parameters, ...) and an error: fmfconv_ff.c:1141:14: error: 'AVCodec' has no member named 'encode' ...

[Sergio Baldoví]

Thanks, committed in [r5207]. I've tweaked libjpeg version info as libjpeg-turbo uses different definitions than libjpeg.