Design & media

flickies

Try it

Self-hosted video REST + MCP API. POST JSON, get a video back. Lipsync (LatentSync 1.5 + Wav2Lip/Wav2Lip-GAN) at /v1/video/lipsync, GFPGAN face restore at /v1/video/restore, pure-ffmpeg ops (trim, concat, transcode incl. gif + fps + codec, scale, mux_audio, extract_audio, thumbnail_grid) under /v1/video/*, and ffprobe metadata at /v1/video/info. file_path (staged) xor file_url in; output_path xor output_url out. Fire-and-forget async jobs (async_job=true → 202 → poll /v1/jobs/{id}) with HMAC-signed webhooks. 11 MCP tools at /v1/mcp. Bearer-token auth. CPU + CUDA images. Use when the user wants to lipsync a face to audio, restore faces in footage, trim/concat/transcode/scale/mux/extract/thumbnail video, probe a video's metadata, or drive any of that from an LLM over MCP.

What it does

Self-hosted video toolkit — lipsync, face restore, and ffmpeg ops in one container. POST a JSON body, get a video back. Every video endpoint takes the same input/output contract; drive it from curl, the generated Go/Python clients, or point a function-calling LLM at the MCP endpoint.

The skill document

flickies

Self-hosted video toolkit — lipsync, face restore, and ffmpeg ops in one container. POST a JSON body, get a video back. Every video endpoint takes the same input/output contract; drive it from curl, the generated Go/Python clients, or point a function-calling LLM at the MCP endpoint.

Lipsync (POST /v1/video/lipsync): drive a face video/image from an audio track. Engines: latentsync-1.5 (ByteDance, Apache-2.0, commercial-safe default, CUDA-only) and wav2lip / wav2lip-gan (Rudrabha, fast/low-VRAM, LRS2 non-commercial — refused unless FLICKIES_ENABLE_NONCOMMERCIAL=1 is set in the server env). restore_face=true chains GFPGAN over the result.

Face restore (POST /v1/video/restore): GFPGAN v1.4 (gfpgan, Apache-2.0) — clean up a Wav2Lip mouth crop or old footage, standalone.

ffmpeg ops (pure CPU, no engine): POST /v1/video/trim, /concat, /transcode (mp4/webm/mov/mkv + gif + fps + codec change), /scale, /mux_audio, /extract_audio, /thumbnail_grid. Metadata: POST /v1/video/info (ffprobe).

Extras: async jobs (async_job=true → 202 + job_id → poll GET /v1/jobs/{job_id}), HMAC-signed webhooks on async completion, server-side file staging, engine load/evict control, an MCP endpoint at /v1/mcp with 11 tools, optional bearer-token auth.

For installation, configuration, and container setup, see references/setup.md.

Security & safety

  • Auth is off by default — FLICKIES_AUTH_TOKEN is unset out of the box, so the whole API is open to anyone who can reach the port. Set it for any deployment beyond localhost, pass Authorization: Bearer once it's set, and bind to loopback / behind an authenticating proxy. Never expose an unauthenticated instance on a network.
  • File and engine management — the staging and engine endpoints include remove and evict operations. Like any state change, only run them against a resource the current task created, only when the user asked, and not against a shared instance others depend on.

When To Use

  • Lipsync a face (video or still image) to a driving audio track — latentsync-1.5 (commercial-safe, CUDA) or wav2lip / wav2lip-gan (non-commercial gate).
  • Restore / sharpen faces in a video with GFPGAN, standalone or chained after a Wav2Lip pass (restore_face=true).
  • Trim a clip, concatenate several clips, or transcode to another container/codec (incl. animated GIF).
  • Change frame rate, re-encode with a specific codec/CRF/preset, scale dimensions, mux an audio track in, extract the audio out, or generate a thumbnail sprite-sheet.
  • Probe a video for duration, codec, fps, dimensions, bitrate (/v1/video/info).
  • Run any long operation fire-and-forget: submit with async_job=true, poll /v1/jobs/{id}, or receive an HMAC-signed webhook on completion.
  • Drive the whole pipeline from a function-calling LLM (Claude, LibreChat, Cursor) via MCP at /v1/mcp.

When NOT To Use

  • Real-time / streaming video output — every endpoint is request/response; long jobs go async + poll, not stream.
  • latentsync-1.5 on the CPU image — it's CUDA-only (cuda_only: true) and the CPU image refuses to load it. GFPGAN is also CUDA-only in practice. Use the :latest-cuda image for those.
  • wav2lip / wav2lip-gan anywhere unless the server was started with FLICKIES_ENABLE_NONCOMMERCIAL=1 — otherwise the request returns 403 NONCOMMERCIAL_GATE_REFUSED. This is a server-side env flag; you cannot flip it per-request.
  • Two engines resident at once — one model is resident at a time. A different engine request triggers hot-swap eviction of the current one. If you need two simultaneously, run two containers.
  • output_url via the MCP tools — MCP tools only accept output_path (they write under FILES_DIR). Use the REST endpoint if you need presigned-PUT output_url delivery.
  • Multipart upload on the video endpoints — only PUT /v1/files/{path} accepts a raw-body upload. Video endpoints take JSON with file_path or file_url.

Setup

The container should already be running. Set the base URL:

export FLICKIES_URL=http://localhost:8000

If the server has FLICKIES_AUTH_TOKEN set, export it too:

export FLICKIES_AUTH_TOKEN=
# every request below then needs: -H "Authorization: Bearer $FLICKIES_AUTH_TOKEN"

Verify: curl $FLICKIES_URL/healthz returns {"status": "ok"} (unversioned, always auth-exempt). For the richer discovery payload — device, ffmpeg version, available/enabled/loaded engines, non-commercial flag — hit GET /v1/health:

curl -s $FLICKIES_URL/v1/health | jq
# { "status": "ok", "version": "...", "device": "cuda", "ffmpeg": "...",
#   "available_engines": [...], "enabled_engines": [...],
#   "loaded_engine": null, "noncommercial_enabled": false }

For install / configuration / env vars / CPU vs CUDA images / engine weights, see references/setup.md.

Quick Start

The input/output contract is uniform across every video endpoint:

  • Input — exactly one of file_path (FILES_DIR-relative, staged via /v1/files) or file_url (any HTTP/HTTPS URL the server fetches).
  • Output — exactly one of output_path (server writes to FILES_DIR/, response {path, size, ...}; download via GET /v1/files/) or output_url (server PUTs to a presigned/PUT-accepting URL, response {url, size, ...}). In async mode both are optional — the server auto-stages to jobs/{id}.{ext}.
# Probe a video (stage it first, then reference by path).
curl -s -X PUT --data-binary @clip.mp4 \
  -H "Authorization: Bearer $FLICKIES_AUTH_TOKEN" \
  "$FLICKIES_URL/v1/files/uploads/clip.mp4"

curl -s -X POST "$FLICKIES_URL/v1/video/info" \
  -H "Content-Type: application/json" \
  -d '{"file_path": "uploads/clip.mp4"}' | jq

# Trim seconds 5–12, write the result under FILES_DIR, download it.
curl -s -X POST "$FLICKIES_URL/v1/video/trim" \
  -H "Content-Type: application/json" \
  -d '{
        "file_path": "uploads/clip.mp4",
        "start_sec": 5, "end_sec": 12, "precise": true,
        "output_path": "out/clip-trimmed.mp4"
      }' | jq
curl -s "$FLICKIES_URL/v1/files/out/clip-trimmed.mp4" --output clip-trimmed.mp4

# Lipsync a face to an audio track (LatentSync, commercial-safe default, CUDA).
curl -s -X POST "$FLICKIES_URL/v1/video/lipsync" \
  -H "Content-Type: application/json" \
  -d '{
        "face_url": "https://example.com/face.mp4",
        "audio_url": "https://example.com/voice.wav",
        "engine": "latentsync-1.5",
        "output_path": "out/lipsynced.mp4"
      }' | jq

Add -H "Authorization: Bearer $FLICKIES_AUTH_TOKEN" to every call if the server has a token set. /healthz is the only always-exempt route.

API — POST /v1/video/lipsync

Drive a face from an audio track. JSON body.

Request Fields

FieldRequiredDefaultNotes
face_path / face_urlexactly one—Driving face — a video OR a still image. face_path is FILES_DIR-relative; face_url is an HTTP(S) URL the server fetches.
audio_path / audio_urlexactly one—Driving audio (wav/mp3/m4a). audio_path FILES_DIR-relative; audio_url an HTTP(S) URL.
enginenolatentsync-1.5latentsync-1.5 (Apache-2.0, CUDA-only, commercial-safe) / wav2lip / wav2lip-gan. The two wav2lip* slugs require FLICKIES_ENABLE_NONCOMMERCIAL=1 on the server → else 403.
restore_facenofalseChain GFPGAN over the output to clean up the face region. Recommended after wav2lip* (fixes the soft 96×96 mouth crop).
output_path / output_urlone (sync)—Sync mode requires exactly one. Async mode: both optional (auto-staged).
output_formatnomp4mp4 / webm / mov / mkv.
async_jobnofalsetrue → 202 + job_id; poll /v1/jobs/{id}.
webhook_urlno—Async only. HMAC-signed POST on completion — see Async Job Lifecycle.

Response

200 (sync) — one of:

{ "path": "out/lipsynced.mp4", "size": 4823110 }
{ "url": "https://bucket.example.com/out.mp4?...", "size": 4823110 }

StagedOutputResponse may also carry sha256, duration_sec, width, height. 202 (async) — { "job_id": "", "status": "accepted" }.

Error Contract

StatuscodeWhen
400BAD_REQUESTnot exactly one of face_*, not exactly one of audio_*, output_path+output_url both set, or neither in sync mode
401UNAUTHORIZEDFLICKIES_AUTH_TOKEN set, missing/wrong bearer
403NONCOMMERCIAL_GATE_REFUSEDwav2lip / wav2lip-gan requested but server has no FLICKIES_ENABLE_NONCOMMERCIAL=1
404NOT_FOUNDengine slug or referenced file_path missing
422VALIDATION_FAILEDPydantic validation (missing/wrong-typed fields)

Error body is always { "code": "UPPER_SNAKE", "message": "...", "details"?: {...} }.

API — POST /v1/video/restore

GFPGAN face restoration on a video. Input via file_path / file_url, output via output_path / output_url (same contract).

Request Fields

FieldRequiredDefaultNotes
file_path / file_urlexactly one—Source video.
enginenogfpganOnly gfpgan.
output_path / output_urlone (sync)—Standard output contract.
output_format / async_job / webhook_urlnomp4 / false / —As above.

Response

200 → StagedOutputResponse or UrlOutputResponse (as lipsync). 202 → JobAcceptedResponse.

Error Contract

400 BAD_REQUEST, 401 UNAUTHORIZED, 422 VALIDATION_FAILED. (GFPGAN carries no non-commercial gate.)

API — ffmpeg ops (POST /v1/video/{trim,concat,transcode,scale,mux_audio,extract_audio,thumbnail_grid})

Pure ffmpeg, CPU. Each takes the standard input/output contract plus op-specific fields. All return StagedOutputResponse xor UrlOutputResponse on 200. All support async_job / webhook_url except where noted (the info-shaped ones — extract_audio, thumbnail_grid — carry no BaseVideoOutputRequest; they take output_path / output_url directly and run sync).

trim — cut [start_sec, end_sec]

FieldRequiredDefaultNotes
file_path / file_urlexactly one—Source.
start_secyes—≥ 0.
end_secyes—≥ 0.
precisenofalsefalse: -c copy, fast, but start_sec snaps to the nearest keyframe (can eat up to one GOP of leading content). true: re-encode H.264 + AAC for frame-accurate boundaries (slower, visually transparent).

concat — join ≥2 videos in order

FieldRequiredDefaultNotes
inputs_paths / inputs_urlsexactly one—Array, minItems: 2. FILES_DIR paths or HTTP(S) URLs.
precisenofalsefalse: concat demuxer + -c copy — requires identical codec/timebase/SAR across inputs. true: re-encode to uniform H.264 + AAC so mixed inputs join cleanly (slower).

transcode — universal re-encode

output_format (from the output contract, mp4/webm/mov/mkv) drives the filter graph. Additional:

FieldRequiredDefaultNotes
file_path / file_urlexactly one—Source.
video_codecno—e.g. libx264, libx265, libvpx-vp9, libaom-av1.
audio_codecno—e.g. aac, libopus, copy.
crfno—0–51.
presetno—e.g. ultrafast, fast, medium, slow.
fpsno—1–240. Applies to all output formats.
gif_optionsno—Only consulted for GIF output: { width?, loop? (0 = infinite), palette_mode? (full/diff/single) }.

scale — resize

FieldRequiredDefaultNotes
file_path / file_urlexactly one—Source.
widthyes—≥ 16.
heightyes—≥ 16.
keep_aspectnotruePad/crop to maintain source aspect.

mux_audio — replace / merge the audio track

FieldRequiredDefaultNotes
video_path_or_urlyes—Video source (path or URL — the server sniffs http(s)://).
audio_path_or_urlyes—Audio source (path or URL).
replace_existing_audionotruefalse merges instead of replacing.

extract_audio — pull the audio out

FieldRequiredDefaultNotes
file_path / file_urlexactly one—Source.
audio_formatnowavwav / mp3 / m4a / ogg / flac.
output_path / output_url——Output target (this op has no async/webhook fields).

thumbnail_grid — sprite-sheet PNG

FieldRequiredDefaultNotes
file_path / file_urlexactly one—Source.
rowsyes—1–16.
colsyes—1–16.
cell_widthno320Per-cell width.
cell_heightno180Per-cell height.
output_path / output_url——Output target (no async/webhook fields).

Error Contract (ffmpeg ops)

Same envelope. 400 BAD_REQUEST (bad input xor, constraint violation), 422 VALIDATION_FAILED (missing start_sec/end_sec, rows/cols, width/height, arrays under minItems), 401 when auth is on.

API — POST /v1/video/info

ffprobe metadata. Input via file_path / file_url; no output fields.

curl -s -X POST "$FLICKIES_URL/v1/video/info" \
  -H "Content-Type: application/json" \
  -d '{"file_path": "uploads/clip.mp4"}' | jq

Response

{
  "duration_sec": 12.4, "width": 1920, "height": 1080, "fps": 30.0,
  "video_codec": "h264", "audio_codec": "aac", "bitrate": 4200000,
  "container_format": "mov,mp4,m4a,3gp,3g2,mj2", "size_bytes": 6510022
}

audio_codec, bitrate, container_format are nullable. Errors: 400 BAD_REQUEST, 401 UNAUTHORIZED.

Async Job Lifecycle

Any video-producing endpoint (lipsync, restore, and the ffmpeg ops that carry async_job) runs fire-and-forget when you set async_job: true. Lipsync/restore are the ones worth doing async — they're the slow ones.

1. Submit — POST with async_job: true. Output is optional; if you omit both output_path and output_url, the server auto-stages the result to jobs/{job_id}.{ext} under FILES_DIR. Response is 202:

JOB=$(curl -s -X POST "$FLICKIES_URL/v1/video/lipsync" \
  -H "Content-Type: application/json" \
  -d '{
        "face_url": "https://example.com/face.mp4",
        "audio_url": "https://example.com/voice.wav",
        "engine": "latentsync-1.5",
        "async_job": true
      }' | jq -r .job_id)

2. Poll — GET /v1/jobs/{job_id}:

curl -s "$FLICKIES_URL/v1/jobs/$JOB" | jq
# { "job_id": "...", "status": "running", "result": null, "error": null }

status ∈ pending / running / complete / failed / cancelled. On complete, result holds the output payload ({path, size} or {url, size}). On failed, error holds {code, message}.

3. Fetch — when status: complete, download the auto-staged result:

curl -s "$FLICKIES_URL/v1/files/jobs/$JOB.mp4" --output result.mp4

Webhook alternative — pass webhook_url on the async submit and the server POSTs the final job state ({job_id, status, result, error}) to that URL on completion instead of making you poll:

  • HMAC-SHA256 over timestamp + "." + body, keyed by the server's FLICKIES_WEBHOOK_SECRET.
  • Headers: X-Webhook-Timestamp: , X-Webhook-Signature: t=,v1=.
  • Retried on non-2xx / transport error with exponential backoff (30s, 1m, 5m, 30m, 2h, 12h), then dead-lettered to the server log.
  • Receiver MUST verify the signature and de-dupe on (timestamp, signature).

GET /v1/jobs/{job_id} returns 404 NOT_FOUND for an unknown id. The queue is in-process — jobs don't survive a container restart.

MCP Endpoint

flickies mounts a Model Context Protocol server at /v1/mcp (streamable-HTTP JSON-RPC, same FastAPI process, same auth middleware). Point a function-calling LLM at it and it drives the pipeline.

Eleven tools mirror the REST surface: list_engines, info, lipsync, restore, transcode, trim, concat, scale, mux_audio, extract_audio, thumbnail_grid. Argument shapes match the REST bodies, with one difference: MCP tools accept output_path only (they write under FILES_DIR; output_url presigned-PUT delivery is REST-only). MCP tools run synchronously — there's no async_job on the MCP side.

Wire it into Claude Code:

claude mcp add --transport http flickies $FLICKIES_URL/v1/mcp
# with auth:
claude mcp add --transport http flickies $FLICKIES_URL/v1/mcp \
  --header "Authorization: Bearer $FLICKIES_AUTH_TOKEN"

The transport requires Accept: application/json, text/event-stream. Raw JSON-RPC over HTTP POST for debugging / non-MCP callers:

# tools/list
curl -s "$FLICKIES_URL/v1/mcp/" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -d '{"jsonrpc": "2.0", "id": 1, "method": "tools/list"}'

# tools/call — lipsync
curl -s "$FLICKIES_URL/v1/mcp/" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -d '{
    "jsonrpc": "2.0", "id": 2, "method": "tools/call",
    "params": {
      "name": "lipsync",
      "arguments": {
        "face_url": "https://example.com/face.mp4",
        "audio_url": "https://example.com/voice.wav",
        "engine": "latentsync-1.5",
        "output_path": "out/lipsynced.mp4"
      }
    }
  }'

The canonical mount path carries a trailing slash (/v1/mcp/); bare /v1/mcp redirects to it. With auth on, every MCP call needs the same Authorization: Bearer header.

Bearer-Token Auth

If FLICKIES_AUTH_TOKEN is set on the server, every route except /healthz (and CORS preflight) requires Authorization: Bearer . Wrong/missing token returns 401 UNAUTHORIZED.

curl -H "Authorization: Bearer $FLICKIES_AUTH_TOKEN" $FLICKIES_URL/v1/engines

With FLICKIES_AUTH_TOKEN unset the API/MCP surface is unauthenticated — anyone who can reach it gets full access. Set the token and bind to loopback / behind an authenticating proxy; for untrusted networks add a reverse proxy doing TLS + rate limiting. See references/setup.md.

Typical Workflows

Lipsync + face restore in one shot

curl -s -X POST "$FLICKIES_URL/v1/video/lipsync" \
  -H "Content-Type: application/json" \
  -d '{
        "face_path": "uploads/portrait.png",
        "audio_path": "uploads/line.wav",
        "engine": "wav2lip-gan",
        "restore_face": true,
        "output_path": "out/talking.mp4"
      }' | jq
# NOTE: wav2lip-gan needs FLICKIES_ENABLE_NONCOMMERCIAL=1 on the server.

Stage once, run several ops off the same file

curl -s -X PUT --data-binary @raw.mp4 "$FLICKIES_URL/v1/files/uploads/raw.mp4"

curl -s -X POST "$FLICKIES_URL/v1/video/scale" \
  -H "Content-Type: application/json" \
  -d '{"file_path":"uploads/raw.mp4","width":1280,"height":720,"output_path":"out/720p.mp4"}' | jq

curl -s -X POST "$FLICKIES_URL/v1/video/transcode" \
  -H "Content-Type: application/json" \
  -d '{"file_path":"uploads/raw.mp4","output_format":"gif","fps":12,"gif_options":{"width":480}}' \
  --fail | jq   # async auto-stages if you omit output_path

Concatenate several clips (mixed sources → re-encode)

curl -s -X POST "$FLICKIES_URL/v1/video/concat" \
  -H "Content-Type: application/json" \
  -d '{
        "inputs_urls": ["https://ex.com/a.mp4","https://ex.com/b.mp4"],
        "precise": true,
        "output_path": "out/joined.mp4"
      }' | jq

Long lipsync, async + poll to completion

FLICKIES_URL=$FLICKIES_URL FLICKIES_AUTH_TOKEN=$FLICKIES_AUTH_TOKEN \
  bash scripts/flickies.sh lipsync \
  '{"face_url":"https://ex.com/f.mp4","audio_url":"https://ex.com/v.wav","engine":"latentsync-1.5"}' \
  result.mp4

See scripts/flickies.sh — submits any endpoint async, polls /v1/jobs/{id} to a terminal state, and downloads the staged result.

Free VRAM after a job

Frees the resident engine from VRAM. Rarely needed — engines hot-swap on demand — so only evict one the current task loaded, and remember a shared instance may have another caller using it.

curl -s -X DELETE "$FLICKIES_URL/v1/engines/latentsync-1.5"   # evict from VRAM (204)

Tips

  1. file_url over staging — if the source is already at a URL, pass file_url and skip the upload round-trip.
  2. precise=false is the fast path for trim/concat but snaps to keyframes / needs matching codecs. Flip to precise=true when you need frame accuracy or are joining mismatched inputs — it re-encodes.
  3. One engine resident at a time — a lipsync request after a restore evicts the restore engine (hot-swap). restore_face=true intentionally chains GFPGAN second, evicting the lipsync model to free VRAM.
  4. async_job=true for lipsync/restore — they're slow. Submit, then poll /v1/jobs/{id} or take a webhook. ffmpeg ops are fast enough to run sync.
  5. Async output is auto-staged — omit output_path/output_url on an async submit and fetch from jobs/{job_id}.{ext}.
  6. wav2lip* is gated at the server — 403 NONCOMMERCIAL_GATE_REFUSED means the operator hasn't set FLICKIES_ENABLE_NONCOMMERCIAL=1. You can't override it per-request; latentsync-1.5 is the ungated default.
  7. CPU image can't run latentsync-1.5 or gfpgan — both are CUDA-only. Wav2Lip-CPU works (slow) if the gate is set. Use :latest-cuda for the full engine set.
  8. GET /v1/health shows device, loaded_engine, and noncommercial_enabled — check it before you get a surprise 403 or a CPU-refusal.
  9. Idempotency-Key — pass an Idempotency-Key header on a POST for safe retries; the server replays the cached response for a repeat (key, method, path).
  10. X-Request-Id — send one to correlate logs across hops; the server echoes it back and mints a UUID4 if you don't.

Related skills

Generate and edit Draw.io, Mermaid, and Excalidraw diagrams from natural language using a structured JSON spec.

by nssa.io1.0k installs47 stars

Join a video meeting as an AI bot with voice, avatar, and screenshare across four operating modes.

by johnpatternai22 installs8 stars

Stores durable facts in a categorized, plain-markdown vault on disk, alongside your agent's built-in memory.

by Iván561 installs18 stars

Find why your productivity system keeps failing, then apply the smallest fix — capacity math, bottleneck routing, durable local notes.

by Iván857 installs69 stars

Fetch raw ad creative, app, ranking, and revenue data from AdMapix as structured JSON.

by fly0pants4.3k installs296 stars

More from psyb0t

Browse all skills

Drive a user-deployed mt5-httpapi MetaTrader 5 bridge over HTTP, with per-action confirmation on every real-money call.

by psyb0t108 installs4 stars

Docker-packaged browser automation for QA against anti-bot stacks and authorized detection testing.

by psyb0t138 installs2 stars

Self-hosted OpenAI-compatible speech service — transcription, translation, and synthesis from one container.

by psyb0t13 installs

Run ffmpeg, sox, and ImageMagick jobs against your files in a fixed-allowlist SSH sandbox.

by psyb0t71 installs

Generate speech audio via Qwen3-TTS over SSH with preset voices, voice cloning, and voice design.

by psyb0t55 installs

Drive one or more IMAP/SMTP mailboxes through a single REST + MCP endpoint — read, search, send, mark-seen, and delete across accounts in parallel.

by psyb0t15 installs