So yeah, having a tricky ffmpeg issue here.

In my app I'm encoding a video based on image frames and add a WAV file for the sound from the same source but through a different web socket. It can happen that the WAV file doesn't always have exactly the same length as the video based on those images + fps (plus minus few seconds).

How can I tell ffmpeg to "stretch" the input WAV file to have it the same length as the video?

In short: images (i) + WAV (w) = final video (v)

length(w) != length (v) ... which is causing problems.

Here an example ffmpeg command I am currently using:

ffmpeg -f image2 -thread_queue_size 512 -framerate 21.5 -i /frames/%d.webp -i videomail_preview.wav -y -acodec aac -strict experimental -ac 1 -vcodec libx264 -crf 16 -preset fast -profile:v baseline -pix_fmt yuv420p -loglevel warning -movflags +faststart videomail_good.mp4 -acodec libvorbis -ac 1 -vcodec libvpx -crf 8 -deadline good -cpu-used 1 -pix_fmt yuv420p -loglevel warning -movflags +faststart videomail_good.webm

And here the ffmpeg version + its extensions

ffmpeg version 2.8.6-1ubuntu2 Copyright (c) 2000-2016 the FFmpeg developers built with gcc 5.3.1 (Ubuntu 5.3.1-11ubuntu1) 20160311 configuration: --prefix=/usr --extra-version=1ubuntu2 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv libavutil 54. 31.100 / 54. 31.100 libavcodec 56. 60.100 / 56. 60.100 libavformat 56. 40.101 / 56. 40.101 libavdevice 56. 4.100 / 56. 4.100 libavfilter 5. 40.101 / 5. 40.101 libavresample 2. 1. 0 / 2. 1. 0 libswscale 3. 1.101 / 3. 1.101 libswresample 1. 2.101 / 1. 2.101 libpostproc 53. 3.100 / 53. 3.100

Any suggestions/trick I am unaware of?

2 Answers

rubberband will probably sound better than atempo; especially for larger ratio differences. If you want to use rubberband but don't have support for this filter in your ffmpeg (and don't want to compile ffmpeg), then you can use the standalone rubberband tool.

rubberband example

rubberband -D 12 input.wav output.wav

• -D<X>, --duration <X> - Stretch or squash to make output file X seconds long

See rubberband -h or rubberband command line utility help text for more info.

Step by step process

Unfortunately this won't be a single command. Using this method you will have to:

1. Encode the video.
2. Get the video duration with ffprobe.
3. Generate the audio with rubberband.

4. Mux the video and audio:

   ffmpeg -i video -i audio -c copy -shortest output

Additional notes

• -movflags +faststart Is suitable for MP4, but is ignored by the webm muxer.
• FFmpeg version 2.8.6 is old. Users are recommended to use a build from the current git master branch. The easiest solution is to download a static build, but you can compile if you prefer. Compiling will allow you to use the rubberband filter in ffmpeg.

There are 2 option in FFMPEG Audio filters that may help you: in both scenario you must manually calculate the ratio of RATIO=(VIDEO_DURATION/AUDIO_DURATION) and put it in command.

1. atempo : with this filter you can speed up or slow down your audio. for your goal ffmpeg -f image2 -thread_queue_size 512 -framerate 21.5 -i /frames/%d.webp -i videomail_preview.wav -y -acodec aac -strict experimental -ac 1 -vcodec libx264 -crf 16 -preset fast -profile:v baseline -pix_fmt yuv420p -loglevel warning -movflags +faststart -filter:a "atempo=RATIO" videomail_good.mp4 -acodec libvorbis -ac 1 -vcodec libvpx -crf 8 -deadline good -cpu-used 1 -pix_fmt yuv420p -loglevel warning -movflags +faststart -filter:a "atempo=RATIO" videomail_good.webm. IMPORTANT note : The atempo filter is limited to using values between 0.5 and 2.0. If you need to, you can get around this limitation by stringing multiple atempo filters together.

2. rubberband : with this, you can have more control on stretching your audio in compare with atempo filter. one basic command is : ffmpeg -f image2 -thread_queue_size 512 -framerate 21.5 -i /frames/%d.webp -i videomail_preview.wav -y -acodec aac -strict experimental -ac 1 -vcodec libx264 -crf 16 -preset fast -profile:v baseline -pix_fmt yuv420p -loglevel warning -movflags +faststart -filter:a "rubberband=tempo=RATIO" videomail_good.mp4 -acodec libvorbis -ac 1 -vcodec libvpx -crf 8 -deadline good -cpu-used 1 -pix_fmt yuv420p -loglevel warning -movflags +faststart -filter:a "rubberband=tempo=RATIO" videomail_good.webm. this filter has more options such as pitch, transients, channels and more.