FFmpeg for Developers:
Programmatic Video Generation

FFmpeg is arguably the most powerful piece of open-source software in existence. It powers YouTube, Netflix, and practically every video editor on the market. But its command-line syntax is notoriously arcane. When we built the automated pipeline for ExamLoom, we needed to generate question-and-answer videos automatically. Using raw bash commands for complex overlays and concatenations quickly became a nightmare.

The solution? The ffmpeg-python wrapper. It allows you to build complex FFmpeg filter graphs using a clean, object-oriented Python syntax.

1. Setting Up the Toolchain

First, you must have the actual ffmpeg binary installed on your system. ffmpeg-python is merely a wrapper that generates the command-line string and executes it via the subprocess module.

pip install ffmpeg-python
  

2. The Graph Paradigm

FFmpeg works on the concept of nodes and graphs. You take an input stream (video or audio), pass it through a filter node (like scale or trim), and output a new stream. In ffmpeg-python, you chain these together.

3. Basic Concatenation and Audio Overlay

Let's say you have a 10-second intro video, a dynamic middle clip, and an outro. You want to stitch them together and add a background music track.

import ffmpeg

# 1. Define inputs
intro = ffmpeg.input('intro.mp4')
middle = ffmpeg.input('middle_clip.mp4')
outro = ffmpeg.input('outro.mp4')
bgm = ffmpeg.input('background_music.mp3')

# 2. Concatenate the video streams
joined = ffmpeg.concat(
    intro.video, intro.audio,
    middle.video, middle.audio,
    outro.video, outro.audio,
    v=1, a=1 # Output 1 video stream, 1 audio stream
)

video_stream = joined.video
voice_stream = joined.audio

# 3. Mix the background music with the voice
# We use amix to blend the two audio streams, dropping bgm volume to 10%
mixed_audio = ffmpeg.filter([voice_stream, bgm.audio.filter('volume', 0.1)], 'amix', inputs=2)

# 4. Output the result
out = ffmpeg.output(video_stream, mixed_audio, 'final_video.mp4')
ffmpeg.run(out, overwrite_output=True)
  

4. Adding Dynamic Text Overlays

For ExamLoom, we needed to overlay the actual exam question text onto a template background. We use the drawtext filter. Note that you need a TTF font file available in your directory.

import ffmpeg

background = ffmpeg.input('template_bg.mp4')

# Overlay text at X=100, Y=200
video_with_text = background.filter(
    'drawtext',
    fontfile='Roboto-Bold.ttf',
    text='What is the derivative of e^x?',
    fontcolor='white',
    fontsize=48,
    x=100,
    y=200,
    box=1, # Add a background box behind the text
    boxcolor='black@0.5', # 50% opacity black
    boxborderw=10 # Padding inside the box
)

out = ffmpeg.output(video_with_text, background.audio, 'question_video.mp4')
ffmpeg.run(out, overwrite_output=True)
  

5. Debugging Complex Graphs

If your graph gets too complex, FFmpeg might throw a cryptic error about stream mapping. You can visualize the exact graph ffmpeg-python generated by viewing the compile command:

print(ffmpeg.compile(out))
# Outputs the raw bash command array that you can inspect manually
  

Conclusion

By moving from raw bash scripts to ffmpeg-python, you gain access to variables, loops, and conditional logic. This is the foundation of building a scalable programmatic video generation pipeline that can churn out hundreds of marketing assets per day.

— Ankit Kumar