Generate audio waveform videos using FFmpeg

Audio waveforms, or audiograms, are a visual representation of sound displayed as a video. It is very common to see a podcast video or podcast ad on social media that includes an animated waveform to make the video more visually appealing or add animation to an otherwise static scene.

In this tutorial, you will learn how to create the kind of waveform videos you might see in a podcast video using a tool called FFmpeg.

About FFmpeg

FFmpeg is a free and open-source command line tool for manipulating video, audio, and other forms of multimedia. You can use it for tasks such as file conversion, editing, decoding, and more. Its versatile nature makes it a popular choice among developers and multimedia enthusiasts.

How to generate waveforms using FFmpeg

FFmpeg gives you a quick and easy way to generate waveforms to visually represent the dynamics and rhythm of the audio in your projects.

The following is a step-by-step walkthrough of how to generate a waveform using FFmpeg.

Install FFmpeg

First, run the command below to check if you have FFmpeg already installed on your system.

ffmpeg --version

You should see a version number and build information if it's installed.

If not, visit the official FFmpeg download page, select the appropriate version for your operating system, and follow the installation instructions.

Usually, it will include downloading the FFmpeg package, extracting its contents, and adding the FFmpeg bin folder to your system's PATH environment variables.

Run the FFmpeg waveform command

For this demo we will use the following audio

To create audio waveforms with FFmpeg, open your terminal and navigate to the directory with your audio file.

Run the FFmpeg command below. Be sure to replace input.mp4 with the name of the audio file you're generating the waveforms for. And replace output.mov with the name you want the output file.

ffmpeg -y -i input.mp4 -loop 1 -r 25 -filter_complex "[0:a]showwaves=size=1280x200:mode=line:colors=white:draw=full[v];[v]colorchannelmixer=1:0:0:0:0:1:0:0:0:0:1:0:0.3:0.3:0.3:0[vout]" -map "[vout]" -c:v qtrle output.mov

This command instructs FFmpeg to generate a video file that visually depicts the audio waveform from your input file.

Let us break down what each part does:

• The -yflag tells FFmpeg to overwrite any file with the same name as output.mov.
• -i input.mp4 shows FFmpeg the audio filen we want to use to generate the waveform. Replace the input.mp4 with the filename of your audio.
• loop 1 (Optional) ensures the waveform is generated based on a single playback of the input audio.
• -r 25 sets the frame rate to 25 frames per second (fps) for the output video.
• -filter_complex signals the start of a complex filter graph. It allows you to apply multiple filters simultaneously.
• showwaves=size=1280x200:mode=line:colors=white:draw=full generates an audio waveform that is 1280 by 200 pixels, drawn as a line, with a color of white and ensures the full waveform is visible.
• colorchannelmixer=... changes the colour and brightness of the waveform.
• -c:v qtrle uses the QuickTime Animation (RLE) codec, a lossless video compression format that prevent loosing video quality.
• output.mov specifies the name of the output video file.

Review the output video

When you run the command, FFmpeg will generate a video file that visualises your audio waveform. The name of the file will be the output.mov you specified in the command.

Play the video and check if it meets your aesthetic preferences, including color and style. For example, below is an example of a waveform generated with the FFmpeg command.

As you can see, it looks plain and simple. But you can add some color and style to it. Let's do that in the next section.

How to improve the appearance of your audio waveform.

You can customize your audio waveform to meet your brand or project's aesthetics. For example, to change the color from white to blue, you need to edit the value of the colors parameter in the command.

ffmpeg -y -i input.mp4 -loop 1 -r 25 -filter_complex "[0:a]showwaves=size=1280x200:mode=line:colors=blue:draw=full[v];[v]colorchannelmixer=1:0:0:0:0:1:0:0:0:0:1:0:0.3:0.3:0.3:0[vout]" -map "[vout]" -c:v qtrle output.mov

• Initial command: colors=white
• Modified command: colors=blue

This changes the waveform's colour to blue. See the example below.

You can also use multiple colours by separating by a | symbol for a multicoloured effect.

You can also create a multicoloured effect by separating them with the pipe | symbol

For example, colors=blue|yellow would create a waveform with the colours blue and yellow.

Another form of customization is the waveform style. FFmpeg allows you to choose different styles by changing the value of the mode parameter.

Some of the available modes include cline, point, line, and p2p. Each mode offers a unique visualisation of the waveform.

For example, setting the mode to point mode=point shows individual sample points in the waveform without connecting them with lines.

You can customise your waveform in several other ways by customising its style (line, point, p2p, cline), size, and background colours, adding dynamic elements like aphasemeter for circular waveforms, or layering multiple waveforms using FFmpeg's complex filter graphs for rich, textured visual effect.

You can customise your waveform in several other ways by customising the following;

• size
• background colours
• adding dynamic elements like aphasemeter for circular waveforms
• layering multiple waveforms using FFmpeg's complex filter graphs for rich, textured visual effect.

How to add audio waveform to a video

Now, let's look at two quick and easy ways you can include the waveforms in your video projects. You can use an online video editor or video editing API if you want to automate your video editing.

Using an online video editor

To add a waveform you can use the Shotstack Studio, a browser based video editor used to design templates for automation. To overlay a pre-generated waveform on a video using the Shotstack Studio, start by opening a new project and uploading your video to the first track. This will form the base layer of your project. Next, upload the waveform file you've already created with FFmpeg. Place this waveform on the second track, directly above the video track, to overlay it onto your video. Adjust settings like opacity to achieve the desired effect and blend the waveform with your video. Once you're satisfied with how the waveform looks over your video, render your project in Shotstack Studio.

Using a video editing API

For those who prefer precise and flexible control over their edits, here is how to add your waveform to a video project using a base edit JSON:

{
    "timeline": {
        "tracks": [
            {
                "clips": [
                    {
                        "asset": {
                            "type": "video",
                            "src": "< link to waveform >",
                            "fit": "none",
                            "scale": 0.16
                        },
                        "start": 0,
                        "length": 5.34,
                        "offset": {
                            "x": 0.363,
                            "y": 0.153
                        },
                        "scale": 0.218
                    }
                ]
            },
            {
                "clips": [
                    {
                        "asset": {
                            "type": "video",
                            "src": "https://shotstack-ingest-api-v1-sources.s3.ap-southeast-2.amazonaws.com/wddscw6f59/zzy8k6l9-1gnv-zq4f-enlv-4swozl1im3ec/source.mp4",
                        },
                        "start": 0,
                        "length": 5.41
                    }
                ]
            }
        ]
    },
    "output": {
        "format": "mp4",
        "size": {
            "width": 1024,
            "height": 576
        }
    }
}

Common use cases for audio waveforms

Waveforms might seem like cool visuals for your audio, but they offer more than that. Let's look at some of the common applications of audio waveforms.

Marketing and social media

Waveforms add a dynamic visual element to audio/video clips shared on social media. It makes your content more likely to capture the audience's attention. Artists and other creators also use them in teasers for music releases, podcast episodes, or as part of an audio-visual branding strategy.

Educational content

Waveforms are also useful tools for educators. They give a visual representation of sound characteristics which can help improve student understanding of concepts like pitch, amplitude, and frequency. This is useful, especially in subjects like physics, where concepts like wave behavior can be abstract. And also in music theory and linguistics, where understanding pitch and frequency is crucial.

Music production and editing

Music producers also use waveforms to observe audio structure. It helps them visualize the rhythm of beats, verses, and choruses. And make them perform tasks like timing adjustments, looping, and slicing with more precision.

Conclusion

You now know how to generate and customize audio waveforms with FFmpeg. And you've also learned how to add the waveforms to your videos using the Shotstack Studio. But there's more to FFmpeg than just waveforms. You can also use it to do things like trim videos and compress videos.

However, FFmpeg, as a command-line tool, comes with a steep learning curve. And it can be intimidating for commercial projects due to the need to learn many commands and also manage scalability. Shotstack offers a user-friendly solution. It's a cloud-based media automation tool that allows you to programmatically create, edit, and render videos without the added complexities associated with FFmpeg.