How to Use Stable Diffusion to Create Videos with Zapier

In this tutorial, we'll explore generating stock video using generative AI (text-to-video) using Stability AI, Zapier, and Creatomate to auto-generate social media videos.

With OpenAI's Sora making its debut soon, AI-generated videos are getting closer to becoming a reality. Especially when it comes to video automation, text-to-video will open up an entirely new world of possibilities. No more need to rely on Shutterstock, Storyblocks, or Pexels, for finding B-roll footage – just describe what you want the video to portray, and AI will do the rest.

There are several popular APIs for leveraging AI in video automation right now. First, there's ChatGPT for generating video scripts and ideas. Then there's DALL·E for generating images. And for those who want to go further and use AI-based stock footage, there's Stable Video Diffusion:

AI video models are capable of producing clips of about 3 to 5 seconds. Even though that seems too short to be of any practical use, it serves well as backdrop footage for social media videos, particularly for TikTok, Instagram, and YouTube.

In this tutorial, we'll focus on text-to-video AI for social media automation. In order to demonstrate this, let's use a video format that is popular on social media, a "Quiz" video. We'll set up a Zapier automation to feed in a topic and output a ready-to-post video. This is an example of what we'll end up with:

Here, we asked AI to make an engaging "Questions and Quizzes" video for social media. The AI generated the question ("Why was the cat on the computer?"), the answer ("It wanted to keep an eye on the mouse!"), and the backdrop video.

Let's dive in!

Prerequisites

Here's what we need:

• Creatomate: to create a video template. Sign up for free.
• Stability.ai: to generate video clips using the Stable Diffusion AI.
• Zapier: to set up an automated workflow and further automate the process.

How to auto-generate AI videos through Zapier?

To generate AI videos through Zapier, you'll need Stability AI and Creatomate. Stability AI runs the Stable Video Diffusion model, capable of generating video based on a prompt.

To put the video together into a ready-to-post social media video, we'll use Creatomate, an API for creating videos from a template. Creatomate integrates directly with Stability AI, making it very easy to set up. Plus, Creatomate integrates with Zapier, allowing integration with thousands of other apps.

For this tutorial, I'll assume you're already familiar with Zapier. If not, no worries – it's very easy to learn. Our basic Zapier workflow will take input data from a Zapier Table, pass it through Creatomate (which in turn uses Stability AI), and insert the AI-generated video back into the table. The process is as simple as that:

Let's get started!

1. Create a Stability.ai account and get your API key

Go to Stability.ai, and click the Get Started with API button:

In the top right corner, click on Login. Log in if you already have an account with Stability. If not, you can create an account by clicking Sign up.

Once logged in, click on your profile icon in the top right corner. Go to the API Keys page.

Copy your API key; you'll need it in the next step.

2. Create a video template in Creatomate

Log in to your Creatomate account or create a free account if you haven't already.

Let's connect to our Stability.ai account first. To do so, click ... on the left, then choose Project Settings. Within the Integration section, enable Stability AI. You'll then be prompted to enter your API key. Once you're done, click Confirm, and close the project settings menu:

Now, let's create a design for our videos. Navigate to the Templates page, and click the New button. For this tutorial, we'll use the Question & Quizzes template from the Social Media category. Select the 9:16 Vertical size, then click the Create Template button to open it in the editor:

The template editor lets you completely customize the video. This is out of scope of this tutorial, but before we get started with Zapier, let's go over a few things.

First, check out the left side panel, where you can see all the elements that make this template. Most of them are marked Dynamic. This means we'll be able to replace them in our Zapier workflow later.

Now select the Video element. This is the element that'll be generated by Stability AI. To set this up, we need to change its Provider from "Uploaded File" to "Stability AI" in the right hand property panel, like so:

For Model, you can choose between "Stable Image Core" and "Stable Diffusion 3.0". I'd suggest going with the Stable Image Core model. It is a model based on Stable Diffusion with an additional parameter, the Style, which lets you instruct the AI to generate in a specific theme, like Comic Book, Fantasy Art, and Photographic. It requires the least prompt engineering to generate images in a particular setting, which is perfect for the purpose of this tutorial.

Perhaps you're wondering why we're using an AI image model here. That's because this process works in two steps; first, an image is generated based on the prompt using the selected AI model (text-to-image). This image is then converted into a video with Stable Video Diffusion (image-to-video). All of that is handled automatically by Creatomate and Stability AI.

One more thing. Because we're creating vertical videos, set the Aspect Ratio to 9:16. You can leave the Prompt field blank.

And that's all you need to do. Your template is now ready for use. Let's move on to the next step: setting up the automated workflow.

3. Set up a Zapier trigger

Log in to your Zapier account or sign up for free if you haven't already.

From the left menu, navigate to the Tables page. Click the Create blank table button, enter a name, and click Create table:

Let's create a table with these three fields:

• A text field: Question
• A text field: Answer
• A link field: Video URL

Then, insert the following data for the first record:

• Question -> Why was the cat sitting on the computer?
• Answer -> It wanted to keep an eye on the mouse! 🐁😂
• Video URL -> Leave this cell empty for now. In step 5, we'll update it with the link to the generated video.

Our table is all set up and ready to use. Head back to your Zapier dashboard, click Create, and select Zaps. Once you're in the Zap editor, click the Trigger block. Search for and select the Zapier Tables app, and choose the New Record event. Then, click Continue.

Under Trigger, select the table you've just created:

Then, click Continue.

Under Test, click Test trigger to make sure Zapier can find a record from the table. When the test is successful, click "Continue with selected record", and move on to the next step:

4. Generate the video

Search for the Creatomate app, and choose the Create Single Render event. Continue by selecting your account or signing in with your project's API key, which you can find under Project Settings in your Creatomate dashboard. Then, click Continue.

Under Action, select the Question & Quizzes template first:

Now, let's map the text to the template. We'll use the question as the text-to-video prompt:

• Set Video to Zapier Tables -> Question
• Set Question to Zapier Tables -> Question
• Set Answer to Zapier Tables -> Answer

When done, click Continue.

Now that everything is set up correctly, click Test step. What happens is as follows; Zapier instructs Creatomate to create a video using the selected template and input data. In turn, Creatomate uses Stability AI to generate the background video, then puts it in the template, generates the final video, and finally sends it back to Zapier.

You can view the generated video by visiting the given URL. But before you do, let's wait until the video finishes; it might take a minute. If you visit the URL too soon, you'll see "Not Found," which means that the video creation process is still in progress. If this is the case, just wait another minute and then try again.

5. Use the video

The video is now complete and ready for use. For instance, it could be posted to YouTube, TikTok, Instagram, or Facebook. Or, you could send it to your inbox so that you can review it first.

In this tutorial, we just put the generated video back in the Zapier Table.

Click + to add another step to your Zap. Search for the Zapier Tables app, and select the Update Record event. Then, click Continue.

On the Action page, select your table in the Table ID field first:

Next, in the Record ID field, select Zapier Tables -> Record ID, which you can find under the Custom tab:

Then, select Creatomate -> Url in the Video url field, and click Continue:

Click Test step on the Test page to make sure Zapier is able to update the record with a link to the video. After that, go back to your table. You'll see that the video link is there:

What's next for video automation using AI?

In terms of applying AI to video automation, this tutorial barely scratches the surface. You can go much further if you know how to use the right APIs and techniques.

For example, you can use ChatGPT to generate video ideas and write scripts. By integrating ElevenLabs, you can establish a workflow for producing AI voiceovers. Adding Creatomate to the mix allows you to generate subtitles. Depending on your goals, you could even set up a Zapier workflow to automatically create complete videos from scratch!

To learn how, check out these step-by-step tutorials next:

👉 How to use AI to Automate Videos for Shorts, Stories, and Reels
👉 Use ChatGPT to Create Social Media Content using Zapier
👉 Use AI-generated Images from DALL·E to Create Videos with Zapier
👉 Add Subtitles to Videos using Zapier
👉 Add AI Voice Overs to Videos using Zapier