movie-subtitler

movie-subtitler

Foreign-language video in → English-subtitled video out. One script does the whole pipeline.

Prerequisites

• ffmpeg — audio extraction and video mux/burn.
  • macOS: brew install ffmpeg
  • Linux: sudo apt install ffmpeg (Debian/Ubuntu) or your distro's package manager
• yt-dlp — only needed if --input is a URL rather than a local file.
  • macOS: brew install yt-dlp
  • Linux: pip install -U yt-dlp or sudo apt install yt-dlp
• whisperx — the transcribe/translate engine, on both macOS and Linux. Install it into a Python virtualenv, e.g.:

  python3 -m venv ~/.venvs/whisperx
  source ~/.venvs/whisperx/bin/activate
  pip install whisperx
  

  (or uv pip install whisperx if you use uv.) A venv install commonly isn't left on PATH, so point the script at it with WHISPERX_BIN (see below) if command -v whisperx doesn't find it — e.g. WHISPERX_BIN=~/.venvs/whisperx/bin/whisperx.
• GPU is optional. The script auto-detects nvidia-smi: if present (Linux/Windows with an NVIDIA GPU) it uses WhisperX's CUDA default (float16); otherwise it falls back to --compute_type float32 for CPU (including Apple silicon) inference. CPU-only transcription is slower but works fine — expect roughly ⅓–½ of the video's runtime (see below).

If any of whisperx/yt-dlp/ffmpeg aren't on PATH, set WHISPERX_BIN, YTDLP_BIN, or FFMPEG_BIN to their full paths before running the script.

TL;DR — how to run it

./subtitle.sh \
  --input "https://www.youtube.com/watch?v=..." \
  --lang tr

• --input — URL (anything yt-dlp handles) or a local file path.
• --lang — source language code (tr, pt, es, ja, fr, ...). Always pass it; check the video first (yt-dlp --skip-download --print "%(title)s") if unsure.
• Output lands in the current directory as .subbed.mp4 + .subbed.eng.srt (srt basename matches the video so VLC auto-pairs the sidecar; override with -o/--outdir and --name).
• Default is a fast soft-mux (subs as a selectable track, no re-encode — Plex/players handle it fine). Pass --burn only if subs must be in the pixels (slow full re-encode).
• --no-translate keeps subs in the original language. --model small trades accuracy for speed.

Known limitation: no speaker labels (diarization)

This skill doesn't label who's speaking. We tried bolting on whisperx --diarize (pyannote) and it didn't hold up in practice, so it was left out. If you want to add it, here's what to know going in:

• Why a naive bolt-on doesn't work: in translate mode, Whisper emits long (20-30s) subtitle cues, since alignment isn't possible on translated text. That means a single [SPEAKER_NN] label ends up stamped across an entire multi-person exchange — wrong, and a wall of text on screen. The speaker clusterer can also over-split a small cast into extra spurious IDs, especially with music or overlapping dialogue in the mix.
• A design that would work better: (1) transcribe in the original language with alignment enabled, giving short, per-word-timed cues with clean speaker turns; (2) translate each cue to English separately (an LLM works well here) while preserving timing and speaker labels — this step can also infer real character names from context instead of generic SPEAKER_NN IDs. Passing --min_speakers/--max_speakers to pyannote helps when you know the cast size. Diarization is the most compute-hungry step in the whole pipeline (roughly +50% runtime on CPU), so a GPU helps a lot if you go this route. It also needs pyannote installed alongside whisperx and an HF_TOKEN environment variable with access to pyannote's gated models on Hugging Face.

Run it in the background — WhisperX on CPU takes roughly ⅓–½ of the video's runtime (a 2h movie ≈ 30–60 min; faster with a GPU). It prints [movie-subtitler] DONE: at the end.

Gotchas

• Whisper only translates into English, not between arbitrary language pairs.
• When translating, the script passes --no_align: WhisperX's per-language alignment models can't align translated English text to foreign audio, so we keep plain Whisper timestamps. They're accurate to ~1s, fine for subtitles.
• If the download 403s or fails, yt-dlp is probably stale: upgrade it (pip install -U yt-dlp or brew upgrade yt-dlp, depending on how you installed it).

Files

• subtitle.sh — the whole pipeline (download → audio extract → whisperx → mux/burn).