Skip to main content

Video & Audio Generation

Libre WebUI 0.18.0 extends generation beyond images: provider plugins can declare video and audio capabilities, and everything generated — images, videos, speech, and sound — lands in one per-user media gallery.

Media generation is available to every authenticated user. The gallery is strictly per-user: every read, content fetch, and delete is scoped to the signed-in account.

Plugin Capability Blocks​

A plugin definition declares each media capability as its own block:

"capabilities": {
  "image": { "endpoint": "...", "model_map": ["..."], "config": { ... } },
  "tts":   { "endpoint": "...", "model_map": ["..."], "config": { ... } },
  "audio": { "endpoint": "...", "model_map": ["..."], "config": { ... } },
  "video": { "endpoint": "...", "model_map": ["..."], "config": { ... } }
}

Every block has an endpoint, a model_map fallback list, an optional models_endpoint for live model discovery, and a config object with capability-specific options — sizes and aspect ratios for images, voices and formats for speech, resolutions, aspect ratios, and durations for video.

Two audio capabilities exist and both end up in the gallery as audio:

  • tts is speech: text is read aloud in a selected voice.
  • audio is sound: a model generates audio content from a prompt.

OpenRouter (plugins/openrouter.json) is currently the only bundled plugin that declares video and audio blocks. When a models_endpoint is present, the model list refreshes on the normal discovery cycle (see Environment Variables for the discovery TTL settings); the model_map remains the fallback.

Generating Media​

Open Imagine (/gallery). The header offers Generate for images (when image generation is enabled in Settings), plus Video and Audio panels.

Speech and sound generation are synchronous: the request runs, the result is saved to the gallery, and the response returns the finished item.

Video Job Lifecycle​

Video generation is asynchronous. Submitting a job (POST /api/media/video/generate) returns 202 with a job record, and the job moves through pending, in_progress, and finally completed or failed.

  • Progress is client-driven. The provider is contacted only when the job is polled (GET /api/media/video/jobs/:jobId); the UI polls every 30 seconds while the panel is open. There is no background worker, so nothing advances a job while nobody is polling it — the provider keeps rendering, and the next poll picks up the result.
  • On completion the backend downloads the video (200 MB cap, HTTP redirects not followed) and saves it to the gallery.
  • The job record stores the plugin, model, options, status, and the prompt (encrypted at rest). Job records older than 30 days are pruned opportunistically.

The gallery lists all media kinds interleaved by creation time, with filter pills for All, Images, Videos, and Audio. Videos and audio play inline; images open in the lightbox; every item can be downloaded or deleted.

Storage and serving are deliberately conservative:

  • Media is stored encrypted inside the application database (under DATA_DIR), not as loose files on disk. Back up the database and ENCRYPTION_KEY together, as with all encrypted data.
  • API responses never embed media payloads; items reference a per-item content URL instead.
  • Served content must match a per-kind MIME allowlist and the stored type, is capped at 200 MB, and is delivered with X-Content-Type-Options: nosniff and a Content-Security-Policy that sandboxes the response.

The previous image-only endpoints and the image generation panel keep working unchanged; they write into the same gallery.

Rate Limits​

The media API is rate-limited per client:

OperationLimit
Generation (video, speech, sound)10 requests per minute
Video job polling60 requests per minute
Gallery listing, content, and deletes120 requests per minute

The 30-second UI poll stays well inside the polling budget.

Metering and Privacy​

Media generation calls are metered in the administrator usage analytics like every other outbound provider call: plugin, model, status, duration, and unit counts. Prompts and generated content are never written to the usage records. The generated media itself and the video job's prompt exist only in the user's own encrypted rows.

As with chat, the configured provider receives the prompt and returns the content — provider pricing, retention, and content policies apply.