When to End a Beta: Our AI Video Feature Stayed Behind a Flag for 77 Days

Signup Write New Story When to End a Beta: Our AI Video Feature Stayed Behind a Flag for 77 Days by Frank Houbre by Frank Houbre | @ screenweaver Founder at Outerframe Studio Founder of Outerframe Studio and creator of ScreenWeaver, an AI workspace for writers and filmmakers.

Founder of Outerframe Studio and creator of ScreenWeaver, an AI workspace for writers and filmmakers.

On July 24, 2026, three days after ScreenWeaver opened its beta, Thibaut added one feature flag to the codebase. It was called video-generation. For the next 77 days, anyone outside our team who clicked the Production tab got a placeholder screen instead of a generate button.

I'm Frank Houbre, co-founder of ScreenWeaver, and I spent those 77 days on the other side of that flag, making an anime series with it. So here is my answer to the question in the title: you end a beta when the feature you held back works for a stranger without you in the room, and when everything left on the list is something you would ship as a normal update anyway. A complete feature list has very little to do with it.

The flag came off on October 9. Between those two dates, the app took 229 commits. I went back through them this week, because the log says more about what "ready" meant than anything I could reconstruct from memory.

A beta should end on criteria you wrote down before it started, with a date attached. That's the boring answer, and it happens to be the right one.

GitLab publishes its own rules for this, and they are blunt. A beta feature should have an end date at which it becomes generally available. If not enough users give feedback in time, the policy says it is better to ship what you have than to stay in beta forever. And one line from it I now quote to anyone building a product: "Be wary of including 'just one more' feature." (GitLab, Experiment, Beta, GA support)

What that policy can't tell you is what your own exit criteria should be. For a generative AI product, I'd argue they look strange compared to a normal SaaS:

During the beta, the public side of ScreenWeaver was writing and storyboarding. You could write a screenplay in the standard format, break it into shots, and plan each one (framing, angle, camera movement, duration). The writing part was free then and is still free now.

Video generation existed in the code from July 28, when the first video node landed. It just sat behind the flag, and the flag only turned it on for our own team.

That meant my series became the test bench. Episode 2 of Lost Garden, The King Beneath the Vault, came out on September 15, 2026, in the middle of the flag window. It runs 21 minutes and took me about 75 hours, made the same way as every episode: script first, then the storyboard, then Seedance 2.5 inside ScreenWeaver.

If the pipeline broke on a Tuesday night, I was the one who lost the evening. Nobody outside the team ever saw it.

That's what a feature flag is for in a beta: the people who wrote the code feel the pain before anyone who paid for it does.

I didn't expect the ratio. When I sorted the 229 commits between July 24 and October 9, 116 of them have "fix" in the message, and only 51 start with "feat". Most of the work went into making existing things behave.

A few of them tell the story better than any roadmap:

Notice what's missing from that list. No new effect, no new button. Almost every fix makes the output easier to predict, and when you sell a pipeline, predictable output is most of what people are paying for.

Because a stranger with a half-working pipeline doesn't blame the pipeline. They blame themselves, then the model, then they leave.

I've watched this for years teaching people AI video. A beginner types a prompt, gets a face that changes between two shots, generates again, and again, and burns through their credits before they understand that the problem was upstream: no reference, no plan, no shot list. If we had opened generation in July with a storyboard prompt that quietly asked for polish, every one of those panels would have looked "finished" and wrong at the same time. People would have paid to learn nothing.

The order we shipped in is the order you should make a film in: write first, plan the shots, generate last.

There's a selfish reason too. A film is a chain. If the script feeds the breakdown, the breakdown feeds the storyboard and the storyboard feeds the render, a bug at the top multiplies all the way down. Keeping generation behind a flag let us fix the top of the chain with real episodes running through it, before strangers stacked their own projects on it.

The commit that removed the flag is short, and its first line is the whole story: "The flag is on everywhere, so video and the Production tab are now ungated." The placeholder screen was deleted the same day.

For users, ending the beta meant three concrete things:

For me, it meant the tool I'd been using alone at night became the tool anyone can open, and the bugs I tolerated stopped being acceptable. That shift in standard is the real line between beta and launch. The date on the blog post is just when you admit it.

Is ScreenWeaver still in beta? No. The beta ended in October 2026 and the whole pipeline, from script to video, is open.

Do you need a feature flag service to run a beta like this? No. A single boolean checked on the server would have done the job. What mattered was that the gate sat on the server routes too, so the generate endpoint itself refused requests, and that it had one owner and one date to remove it.

What's the one metric that told you it was time? There wasn't one. It was the moment the list of open fixes stopped scaring me. Every item left was something we'd happily ship on a Tuesday as a normal update.

If you want to see the order I'm talking about, the writing side is free: start a script on ScreenWeaver and only move to production once the story holds up. I'm biased, I co-founded it, but the 229 commits are real and so is the 77-day wait.

I Vibe-Coded My Own Screenwriting Tool. Shipping It Was the Easy Part.