← ClaudeAtlas

video-model-bakeofflisted

Use when choosing an AI video or image model for a real build — surveys what providers offer right now, runs a scored head-to-head on your actual content, and publishes a comparison page. Trigger with 'which video model', 'compare video models', 'model bake-off', 'evaluate image-to-video', or before any video generation project where the model choice is not already settled.
thepandanlabs/video-gen-demo-booth · ★ 1 · AI & Automation · score 64
Install: claude install-skill thepandanlabs/video-gen-demo-booth
# Video model bake-off Pick a provider and a model with evidence from your own content, not from a leaderboard. ## The core claim **Leaderboards measure preference, not your job.** The highest-ranked image-to-video model on Artificial Analysis will lose your bake-off if your job is "keep this person's face" and the model quietly swaps in someone else's. That has happened: a model ranked #1 for image-to-video produced a visibly different person by the final frame in every cell, while scoring fine on general aesthetics. Run the bake-off. It costs about $10 and takes an hour. ## Flow ``` 1. Frame the job what must be true of the output, in one sentence 2. Survey providers who hosts models, at what price, on what terms 3. Survey models shortlist 4-6 arms, including the incumbent 4. Build the assets test subjects and scenes (IP-safe if archetypal) 5. Run the harness every arm x subject x plate, checkpointed 6. Score by frame pairs first frame vs final frame, not by watching 7. Publish the comparison a page with clips, scores, cost and latency ``` Do not skip 1. Everything else is downstream of it. ## 1. Frame the job Write one sentence: *what must be true of the output for this to work at all?* Examples that produce different winners: - "The attendee must recognise themselves" → identity preservation is the gate - "The product must look exactly like the SKU" → object fidelity is the gate - "It must feel cinematic" → aesthetics is th