So wurde ein viraler TikTok AI-Trend zu einer One-Photo-Vorlage

AI video trends on TikTok come and go every few days, so a page for one has to ship while the trend is still rising. This post walks through the pipeline I built to do that: trend config, link resolver, reference-video template, credit reservation, polling, portrait reframe and failure handling, with code from the repo.

The product is simple from the outside. You pick a trend (or paste the TikTok link you saw), add one photo, and get the same clip with you, your pet or your car in it, as a vertical MP4 with sound. Underneath, every one of those steps had at least one way to fail that I only found by shipping it.

The first decision was where a trend lives. I tried keeping trends purely in the database and gave up quickly: each page carries a lot of hand-checked content (which TikTok post is the reference, which second of it the story beats come from, which versions people remake), and I wanted that reviewed in code, with diffs. So the content is a TypeScript object, and whether it is live is one column in a trend table.

A page is public only when both are true: its config exists in TREND_PAGES and its row says page_status = 'published'. Drafts render for admins with a noindex tag and a "Draft" bar; everyone else gets a 404. The status is a small state machine:

/** * none = no page in code; draft = in code, waiting for approval; * held = turned down for now; published = public. */ export type PageStatus = ' none ' | ' draft ' | ' held ' | ' published '; export type WallStatus = ' pending ' | ' approved ' | ' hidden '; Enter fullscreen mode Exit fullscreen mode The home page "wall" reads from the same table with its own wall_status, so a trend can appear as a card before its page exists. There is also a quality floor in code rather than in someone's head: no TikTok post with fewer than 200,000 plays is shown anywhere on the site ( MIN_PLAYS = 200_000 ), and remakes shown under a trend need 100,000. A checker script holds every page to those numbers.

Each page also records the seconds where its story beats happen, and a script cuts a still at each of them for the page's "what happens, in order" strip.

Splitting content (code) from state (database) is what made "ship a page in an afternoon" possible: the content gets reviewed like any other change, and going live is a one-row update. The next problem was getting people from a TikTok link to the right page.

Many visitors arrive having just watched the trend on their phone. The fastest path is "paste the link you saw", so the home page has a single input. What people actually paste is rarely a clean URL. It is the whole share text from the TikTok app, a caption plus emoji plus a vm.tiktok.com short link, often opened inside TikTok's in-app browser.

The resolver does three things: pull the first URL out of whatever was pasted, follow short links by hand, and only ever follow TikTok hosts. That last part matters, because a server that follows any redirect a stranger hands it is an open proxy into your network.

const TIKTOK_HOST = / ( ^| \.) tiktok \. com$/i; const SHORT_HOST = /^ ( vm|vt )\. tiktok \. com$/i; for ( let hop = 0; hop < 3; hop ++ ) { const direct = parseAwemeId ( url ); // /@user/video/<15-22 digits> if ( direct ) return { url, awemeId: direct }; if (! TIKTOK_HOST. test ( new URL ( url ). hostname )) return null; const resp = await fetch ( url, { redirect: ' manual ', // we read Location ourselves signal: AbortSignal. timeout ( 5000 ), }); const next = resp. headers. get ( ' location ' ); if (! next ) break; url = new URL ( next, url ). toString (); } Enter fullscreen mode Exit fullscreen mode Once there is a video id, it is matched against every published page's known post ids (the reference, the remakes, and the ids the trend feed saw). A match returns { slug } and the browser navigates to that page.

Anything else is stored as a copy_request row and the visitor is told we will look at it. No video is fetched or stored at this step. That table doubles as the backlog: a list of trends real people tried to copy and could not.

The endpoint is rate limited to 20 links per 10 minutes per IP.

With the visitor on a trend page, the generator needs a recipe. The rule every template follows: the person is new, the scene is not. So the model gets the scene as a reference video and the visitor's photo as an image reference, and the prompt asks it to keep everything except the subject: the car, the camera move, the beats and the sound.

Here is one template from the Cat Dance AI page, trimmed and translated (the real prompt is longer and names every beat in order):

generation: { mode: ' reference-video ', photos: [{ key: ' cat ', label: { en: ' Your cat ' }, hint: { en: ' One cat, alone, face and ears clear. A dog or another pet works too. ' } }], prompt: ' Replace the white kitten leaning out of the taxi window in @video1 with the animal in @image1. ' + ' Fur, markings, face and build all follow @image1. Keep the scene, camera, movement and rhythm of @video1. ' + ' Keep its sound, add no narration.

No text, captions, logos or watermarks. ', referenceVideoUrl: ' <our 10 s copy of the post, sound kept> ', durationSec: 10, aspectRatio: ' 9:16 ', generateAudio: true, etaMinutes: [ 3, 6 ], } Enter fullscreen mode Exit fullscreen mode @image1, @image2 are the uploaded photos in slot order; @video1 is the reference. The visitor's photos are always sent first, then any template-side images (a costume, a prop), so the numbering in the prompt stays stable. etaMinutes is what the waiting screen shows. There is also an optional note field, capped at 200 characters, which is appended last so it can override background or lighting.

It is stripped of @, braces and newlines before it goes anywhere near the prompt, so nobody can type @video1 into it and point the model at our inputs.

Every template on the page maps to one version people actually remade, in the order the tutorials showed them, which is why the generator above has four. The interesting bugs started when templates got choices.

Some trends are landscape originals, and people want to post them vertically. The Gangnam Style page lets the visitor pick Landscape or Portrait, and one group photo or a photo each. Those two choices change the kind of request I send.

A landscape run is an edit of the reference. On October 16 I found out the hard way that the model treats "replace the player in @video1" as a video edit, and an edit only accepts a duration of -1 (as long as the reference) and an adaptive aspect ratio. Send anything else and the task is accepted, then fails about a minute later with a parameter-constraint error. That is the worst failure mode: the visitor has already waited, and the charge has to be undone. So the plan builder decides up front:

const portrait = picked?. frame === ' portrait ' &&!! gen. choices?. frame; const edit =!! reference &&! portrait; task: { durationSec: edit? - 1: gen. durationSec, aspectRatio: edit? ' adaptive ': portrait? ' 9:16 ': gen. aspectRatio, } Enter fullscreen mode Exit fullscreen mode A portrait run cannot edit a landscape reference (an edit keeps the frame), so it uses a second prompt, portraitPrompt, that makes a new clip following the reference at 9:16. Even then, a model sometimes returns another shape. Every trend video leaves the site portrait, so after download it goes through ffmpeg: anything more than 6% off 9:16 is centered on 720×1280 over a blurred, darkened copy of itself, sound untouched. The blur runs at an eighth of the size, which is cheaper and looks just as soft:

[0:v]split=2[bg][fg]; [bg]scale=90:160:force_original_aspect_ratio=increase,crop=90:160, boxblur=6:2,colorchannelmixer=rr=0.6:gg=0.6:bb=0.6,scale=720:1280[b]; [fg]scale=720:1280:force_original_aspect_ratio=decrease:force_divisible_by=2[f]; [b][f]overlay=(W-w)/2:(H-h)/2,setsar=1,format=yuv420p[v] Enter fullscreen mode Exit fullscreen mode If ffmpeg fails, the video is kept as rendered and I get an alert. A slightly wrong shape is better than no video.

Video renders cost real money per second, so the order of operations is strict, and the comment in the code says it in one line: reserve first, render second: a render that cannot be paid for never starts, and a render that fails gives the reservation back.

Before anything is charged, the run is fully planned and validated. Photos are only accepted if they are HTTPS URLs on our own storage host under uploads/, and each one gets a HEAD request, because a photo from an earlier run may have been deleted with its video. Handing the model a dead link would fail after charging. Billing rounds every started second up, with an epsilon so float noise does not add a whole second:

export function billedSeconds ( seconds: number ) { return Math. min ( Math. ceil ( seconds - 1 e - 6 ), MAX_TEMPLATE_SECONDS ); // 15 } Enter fullscreen mode Exit fullscreen mode After submission, the run is a row that gets polled. Polling is safe from any number of places at once (the open page, the "My videos" list, a cron), because closing a run goes through one conditional update that only one caller wins. The timings in the code today:

The finished file is copied into our own storage right away, because provider links expire and "My videos" has to keep working. The pack page reflects the same rule in plain words: one-time packs, credits never expire, and failed renders give their credits back.

The first version showed one message for every failure: "Something went wrong, try again in a minute." On October 26 one visitor sent the same photos six times in a row after seeing it. The model had refused them for showing a real person, and it was going to refuse them every time. Each failure was refunded, but the message had sent them in a loop.

Now the failure reason from the provider is sorted into three kinds before anything is shown:

const CONTENT_RULES = /sensitive|content (?: policy|filter|safety|moderation|check|risk|violation ) |safety|moderat|nsfw|violen|sexual|nud (?: e|ity ) |porn|inappropriate|prohibited|policy/i; const PHOTO_RULES = /real (?: person|people ) |input image|aspect ratio|too (?: narrow|wide|small|short|tall ) /i; export function failureKind ( reason?: string | null ): ' content ' | ' photo ' | ' other ' { if (! reason ) return ' other '; if ( CONTENT_RULES. test ( reason )) return ' content '; return PHOTO_RULES. test ( reason )? ' photo ': ' other '; } Enter fullscreen mode Exit fullscreen mode content says plainly that the model's rules stopped it. photo says to change the photo (too narrow, wrong shape, or read as a real person), not to retry. Only other keeps "try again". For content and real-person refusals, a background job asks a text model which photo was the likely cause, and the page shows that hint if it arrives.

The rest of the run never waits on it, so a slow hint can't hold anything up.

Separately, every uploaded photo goes through an automatic check for nudity and sexual content. That check runs after the render has started and only raises an alert, so it never blocks a legitimate video. The video model's own rules are the hard gate.

Group templates were the last hard part. With three people in a scene, the video model has to keep three faces apart in a single pass, and one wrong face ruins the clip. The fix, added this week, is a keyframe step before the video: for each shot of the reference, an image model draws a still with the visitor's people placed in it, and a text model checks the still against their photos.

The checker's contract is deliberately tiny. It is asked whether each person is clearly the same as their photo, in the right place, shown once, with no text or warped hands, and it must answer with JSON only:

{ "ok": false, "why": "The person on the left has a different hairstyle than their photo." } Enter fullscreen mode Exit fullscreen mode A failed still is drawn again, three tries at most. Only stills that passed are attached, after the photos, and the prompt's {kf-<shot>} placeholders become a pointer like "frame follows @image6" for those shots; with none, the placeholders are empty and the prompt reads as it did before keyframes. The whole step has a 200-second budget so it fits inside the route's 300-second limit with room to submit the video.

Two failure cases needed their own code. Drawn stills can themselves trip the "real person" refusal, so on that error the run is submitted once more without them. And when the keyframe step dies with its function, any run still waiting after six minutes goes ahead without keyframes rather than not at all.

If I started again, I would keep three things. Content in code, state in the database. Money reserved before any work and refunded on every exit path. Failure messages that tell the person what to change. Everything else (the keyframes, the reframe, the link resolver) was added because a real run failed in a specific way, and each fix is a few dozen lines next to a comment with the date it broke.

The other lesson is about scope. Each trend gets its own page with templates that match the versions people actually remade, instead of one generic "upload a photo and describe a video" box. That is more work per trend, but the visitor never has to write a prompt, and the reference clip does the work a long scene description would otherwise have to do. If you want to see the pipeline from the user side, the wall of AI video trends shows which templates are live right now. The newest one, Outstanding Tiny Desk AI, runs on exactly this pipeline: one photo in, the reference clip's framing and sound out, as a vertical MP4.

On the creator side, I keep a short workflow note on what to keep from a trend (the first-second hook, the cut rhythm, the on-screen text pattern) and what to change so the result is yours: my trend-remix notes.

Questions about the reference-video approach, the reframe filter or the credit state machine are welcome in the comments.