Making short videos with AI: what actually works, and the pitfalls

The steps of an AI-assisted short-video pipeline, from source to render: faithful script, real images, synthetic voice, captions, checks and duplicate-free publishing.

Adapted from a guide first written for Quebec readers: passages specific to Quebec were removed. Rules in your province may differ.

A short video made with AI is not a “generate” button. It is a chain of small decisions, several of which are made before you even open a generation tool. This guide describes the method we adopted for vertical documentary videos, and above all the mistakes that method tries to avoid.

The pipeline at a glance

1. Source: an article or reference text, frozen at the start.
2. Script: a narration faithful to the source, with at most one automatic correction if it strays.
3. Scene breakdown: 6 to 9 scenes, each with a written visual intent.
4. Image search: all the breakdown’s queries are prepared together, in a single image pool with provenance.
5. Assignment: each scene gets its image based on relevance, accuracy, continuity and variety.
6. Editorial check of the images before the voice: a rejected scene gets one targeted retry.
7. Synthetic voice and captions aligned to the audio.
8. Render in a single pass, then a technical check of the file.
9. Preview ready for review; publishing is a separate, locked subsystem.

1. Start from a source, never from a vague topic

Asking a model for “a video about such-and-such event” invites invention: approximate dates, plausible but false details, mixed-up names. The method that served us is the reverse. First we freeze a story sheet: the source text, the names, the dates and the scope. No later step may replace those elements.

• The narration is generated from the sheet, with a single correction if it contains something the source does not say.
• If the text model is unavailable, a fallback plan reuses complete sentences from the already-approved text and adds no new claim.

2. The scene breakdown

A useful breakdown says, for each scene, what must be seen and why: a specific person, a place, an object, a map, an era. This written “visual intent” is what later allows you to reject an attractive but off-topic image. Preparing all the search queries together, rather than scene by scene, also avoids repeating the same images and limits calls to outside services.

3. Find real images, with their provenance

This is the step where AI-assisted videos fail most often. For a historical or factual subject, we favour archives over generic image banks:

• open collections such as Wikimedia Commons, the Library of Congress, Europeana, the US National Archives, the Smithsonian or the Internet Archive;
• an open-licence image search engine, keeping only licences that allow commercial use and transformation;
• images already attached to the source article.

For each image we keep the source, author and licence: they feed the video’s credits later.

The accuracy rules that prevent real blunders

• A named person requires an image of that person, established by the source’s metadata.
• A named ship or aircraft requires the right ship or aircraft, not a similar model.
• A “map” scene must show an actual map, plan or diagram.
• A relative is not the person: an emperor’s wife does not illustrate the emperor.
• A newspaper clipping from another case does not become evidence for this one.
• A modern warship does not illustrate an ancient battle.
• If no truthful image exists, the video is rejected rather than filled with an unrelated one.

The underlying rule: truth beats variety. Repeating an exact image is better than inserting a false one. Repetition is still penalized when another equally truthful image exists.

4. AI-generated images and video: when to avoid them

For a real subject (a person, an event, a place), a generated image risks being mistaken for a document. Keep it for clearly generic illustrations, never as “proof”, and say that it is synthetic. This caution matches the verification practices explained in our guide on how to verify audio and video that may be fake.

5. Synthetic voice and captions

• Read exactly the approved text. If you switch voice engine midway, the new voice reads the same text without rewriting it.
• Spell out numbers and years before sending them to the voice, especially in French: several engines mispronounce years, ordinals and abbreviations.
• Align captions to the real audio, not to an estimate from the text: measure durations once the voice is generated.
• Check the terms of use. Many free voices rely on unofficial services or limit commercial use; plan a fallback.

6. Render and technical check

• Vertical 1080 × 1920 format, H.264 encoding, in a single render pass: fewer quality losses and one file to check.
• A technical check after rendering (duration, video and audio streams present, file readable) before allowing publishing.
• No search for new images at this stage: the technical check no longer changes the editorial content.

7. Operating safeguards

A pipeline that calls free external services regularly hits outages or quota limits. A few simple rules prevent runaway behaviour:

• A circuit breaker per provider. At the first quota refusal, stop calling that service for the rest of the video instead of retrying in a loop.
• Per-video budgets. A maximum number of targeted searches, visual checks by a model and retries of the script or voice.
• Caching of results already obtained, so you do not spend quota on the same search twice.
• Preview separate from publishing. Publishing reuses the reviewed file; it does not silently re-render a new one.
• Duplicate-free publishing. Compute the file’s fingerprint (SHA-256) and record each successful upload per platform. On a partial failure, only the failed platform is retried.

8. Learn from failures without drawing the wrong lesson

When an image is rejected, the temptation is to blacklist the person, the source or the keyword. That is a mistake: a failure in one context says nothing about the next. Saved observations should stay general (for example “generic image for a named entity”, “wrong era”, “wrong place”) and become a preference only after repeated evidence. The truth rules themselves are not learned: they are fixed in advance.

Common mistakes

• Letting the model choose and describe images without a provenance check.
• Mixing several voice engines without freezing the text read.
• Retrying a service with an exhausted quota without limit.
• Publishing without review because “it was fine last time”.
• Forgetting credits and licences at export time.

Can you make a quality video with AI without touching it?

Not without safeguards. AI speeds up the first draft, but factual errors, images that do not match the subject and mispronounced voices slip through easily if nobody checks. A check before publishing remains the most useful part of the process.

Should the images be generated with AI?

For a factual or historical subject, real archives with their source and licence are better. Generated images suit clearly generic illustrations, with an honest note that they are synthetic.

Can free synthetic voices be used commercially?

It depends on the tool. Some rely on unofficial services or have terms that limit commercial use. Read the terms before monetizing, and keep a plan B.

What video format should I aim for?

For short-video platforms, vertical 1080 × 1920 in H.264 is the reference format. Render in a single pass and check the final file before publishing.

How do I avoid publishing the same video twice?

Compute a fingerprint (for example SHA-256) of the final file and keep a record of each successful upload per platform. On a retry, resend only what failed.

Where can I find free tools to prepare my files?

Open Provinces offers tools that run in your browser, with no upload: crop an image, compress or convert an image, create a QR code, and several PDF tools for working documents.

Free tools to prepare your files

These tools run in your browser: your files are not uploaded.

Sources and limits

This guide summarizes the design and rules of an in-house pipeline for vertical documentary videos built and run by Open Provinces. These are practices adopted for that project, not an industry standard, and they are not based on any audience measurement. Always check the terms of use and licensing of each tool and source before commercial use. For image licences: <a href="https://commons.wikimedia.org/wiki/Commons:Licensing" rel="noopener" target="_blank">Wikimedia Commons licensing</a> and <a href="https://www.loc.gov/free-to-use/" rel="noopener" target="_blank">Library of Congress free-to-use sets</a>.

General information based on federal and national rules; provincial rules may differ. Confirm with the official body before acting. Last reviewed : 2026-11-10.