xAI Imagine Video Joins Video Lab — Native-Audio Clips With A Safety-Net Fallback Chain

May 19, 2026

Video Lab now talks to three providers: Runway, Higgsfield, and xAI's new Grok Imagine Video. Pick provider: "xai" for short native-audio clips with reference images (dialogue, SFX, ambience baked into the MP4) — or leave it at the default provider: "auto", which now runs a three-stage fallback chain so a single-provider outage degrades gracefully instead of failing the call.

What you can do

• Pin xAI Imagine Video on a video_lab call. provider: "xai" routes through OpenRouter's /videos endpoint on x-ai/grok-imagine-video. Supports create (text-to-video), create_with_image (image-to-video), and a reference-image array (up to seven stills) on a single endpoint.
• Native audio out of the box. Grok Imagine Video renders dialogue, SFX, and ambience into the MP4 directly — no separate mix_audio round-trip needed. The tool ffprobes the output, detects audio, and runs editor-grade analysis automatically so the agent doesn't blindly layer narration on top.
• 1–15 second clips, 480p or 720p, eight aspect ratios. Durations clamp into [1, 15] with a note; aspect ratios 1:1, 16:9, 9:16, 4:3, 3:4, 3:2, 2:3 are passed through directly; ultrawide / ultrawide_portrait fall back to 16:9 / 9:16 with a note. Higher resolution requests (1080p, 2K, 4K) snap down to the model's supported ceiling with a note.
• A real fallback chain on provider: "auto". Auto now runs xAI → Runway → Higgsfield instead of going straight to Runway. Each provider gets one attempt; if xAI and Runway both fail on a text-to-video request, Image Lab generates a still from the same prompt and Higgsfield animates it (image-to-video). The bridge image is generated from your prompt via Image Lab — not pulled from a Google image search — so the clip stays on-prompt.
• fallback_chain block in the response. Every auto call returns which provider succeeded, the failures that came before it, and (when used) the Image Lab bridge metadata. Alfred surfaces "xAI rejected → Runway succeeded on gen4.5" instead of pretending a single provider handled the call.
• The same honest-failure contract you already know. xAI moderation hits map to the provider-neutral nsfw state with the STOP all sibling calls action; non-moderation failures return retry_suggestions with concrete next steps. Pinned providers fail loudly — only provider: "auto" runs the chain.

Where this shows up

• You want a short dialogue clip with voice-over baked in. Before: render a silent MP4 on Runway, then route through video_editor.mix_audio with a TTS track. Now: pin provider: "xai", write the line in the prompt, and Grok Imagine Video returns the MP4 with audio already mixed.
• You want the planner to pick the right provider without thinking about it. Leave provider at auto. xAI usually answers first for short native-audio clips; if it's rate-limited or refuses, Runway picks up the silent rendering; if Runway is down too, Image Lab + Higgsfield animate a still as the last line of defence.
• A provider has a regional outage. Before: the call fails with a single-provider error. Now: the chain tries the next provider with the same prompt and surfaces the chain in fallback_chain.attempts so you can see which one carried the load.
• You want a reference-led video. Pin xAI with input_image. The reference image is hosted on the same GitHub transient host Higgsfield already uses (xAI requires HTTPS URLs, not data URIs) — uploaded for the render, deleted as soon as the task hits a terminal state.

Try it

• "Generate a 6-second clip of a barista calling out an order in a busy café, native audio, on xAI."
• "Animate this portrait with a slow camera push-in and gentle ambient noise, 5 seconds, provider xai."
• "Render a 10-second cinematic scene of waves at sunset with seagulls in the soundscape — provider auto, pick whatever works."
• "I asked for a video and it fell back to Higgsfield — show me the chain." — Alfred reads fallback_chain and explains which provider succeeded and why the earlier ones didn't.

Heads up

• Configure once. Set OPENROUTER_API_KEY in the API environment to enable xAI. If the key is missing, pinned provider: "xai" calls return a clear capability error with the exact env-var name to set; provider: "auto" skips the xAI step and continues to Runway.
• Image-to-video reuses the Higgsfield transient host. xAI's frame_images field only accepts HTTPS URLs. The runtime uploads your reference still to the same public GitHub repo Higgsfield already uses (HIGGSFIELD_GITHUB_TOKEN, HIGGSFIELD_GITHUB_REPO) and deletes the upload after the render finishes — same lifecycle, success or failure.
• The OpenRouter video URL needs the Bearer token to download. That's how the upstream API is set up — every authenticated GET returns the MP4, an unauthenticated GET returns 401. The tool's post-processor handles this for you automatically; nothing for you to configure.
• First+last keyframes are still Runway-only. xAI accepts frame_images with a last_frame slot, but keyframe-style precision morphs (gen3a_turbo / veo3.1 / seedance2 territory) stay on Runway for now. Pin provider: "runway" when you need both ends locked.
• provider: "auto" is now a chain, not a Runway alias. Existing prompts that relied on auto == Runway will now hit xAI first. Pin provider: "runway" to restore the classic single-shot behaviour.
• Auto-chain T2V uses Image Lab for the bridge, not Google Images. If xAI and Runway both fail on a text-to-video call, Image Lab generates a still from your prompt and Higgsfield animates it. We deliberately do not pull a stock image — the bridge stays on-prompt so the final clip matches your intent.
• xAI moderation is its own moderator. A clip Runway accepts can still be blocked by xAI, and vice-versa. Pinning a provider doesn't bypass any safety layer — rewrite the prompt to avoid IP names and loaded terms when you see the STOP all sibling calls action.