Replace In Video

Replace in video

Swap a single on-camera element in a source video (a character, a product, a garment) for one from a reference image, while the motion, timing, camera movement, lighting, audio, and background all carry through unchanged. The lever is a source video plus one to three reference images, and a prompt that names exactly what to replace and what to preserve. This is not re-shooting or re-prompting the scene, it is a targeted swap that leaves everything else frame-faithful.

Inputs to collect

• The source video. The clip whose scene, motion, and audio you want to keep. Bring footage you have rights to (see Technique). (Ask only if none provided.)
• The reference image(s). What the new element should look like: a clean portrait for a character recast, a bare product/garment photo for an object swap. One is enough, up to three.
• What to replace and what to keep. The specific on-camera element being swapped, and whether anything ambiguous nearby (a second character, a held object) must stay.
• Optional: target resolution (720p draft vs 1080p finish), and whether the source audio should play, drive lip sync, or be stripped.

Models

• Default: Pruna P-Video-Replace (prunaai:p-video@replace). The model for swapping one on-camera element in a clip while preserving the rest. Takes the source under inputs.video and up to 3 images under inputs.referenceImages. Capability io:video-to-video + op:edit.
• Reversed direction: if you want the image to be the scene and the video to only lend its motion, that is P-Video-Animate (prunaai:p-video@animate), not this. Replace keeps the video's world, Animate keeps the image's.
• To generate the source first (an iconic shot, a UGC pitch) before recasting it, route the generation to a flagship image-to-video model via product-ad-video, then run replace on the result.
• Confirm the model is live and inspect its exact input fields via runware-models + runware-run before calling. Never hardcode a stale choice.

Workflow

1. Resolve the schema (runware-run) and confirm the input fields (inputs.video, inputs.referenceImages), the resolution and fps options, and that the task is videoInference.
2. Upload the source video and the reference image(s) as URLs or base64 into inputs.video and inputs.referenceImages.
3. Run videoInference asynchronously (deliveryMethod: async). The call returns a taskUUID. Poll getResponse until terminal, then read videoURL. Video is never a blocked sync call.
4. For a localised swap (product, garment, or one character among several), add a positivePrompt that names the element to replace and lists everything to preserve. For a clean single-character recast, the prompt is optional.
5. Review the output for identity/object fidelity and for unwanted changes elsewhere in the frame. Retry drifters with a clearer reference or a tighter preserve list.

See references/examples.md for three worked async videoInference recipes (recast a character, swap a product or garment, drop a subject into a generated scene) with the real request and videoURL result shape.

Technique

• Name what to replace and what to preserve. This is the load-bearing pattern. The reference image carries what the new element should look like, the positivePrompt carries what gets swapped for it and what stays. The closing direction "Only the X should change, everything else stays as the source" is what turns a global appearance match into a localised swap. Be specific about the target: "replace the matte-black earbuds case she's holding" works, "replace the product" is too vague and drifts between runs. Template: Replace [specific source element] with [the X] from the reference image. Preserve [list everything that stays] exactly as they appear in the source. Only [the named element] should change, everything else stays as the source.
• Character recast: clean portrait in, identity out. For a character swap the reference only needs the face, hair, build, and identity. The source carries the costume, the lighting, the setting, and the action, so do not dress the reference to match the scene. A clearly visible front or three-quarter face is the safest reference. For a single-character clip with no ambiguity, no prompt is needed.
• Object swap: bare product/garment photo in. For a product or wardrobe swap the reference should be a clean photograph of the target object alone, no person or hands, plain neutral background. Match the reference framing to how the source presents the element (a vertical held product pairs with a vertical product shot, a torso-on garment pairs with a front-facing flat-lay).
• Three references hold identity through head turns. referenceImages accepts up to 3. Sending a front plus three-quarter left and right tightens a character recast when the source head turns away from camera. The single front reference is enough for static or near-static shots.
• One reference per element for multi-swaps. When the source has two on-camera characters, or you want to swap a garment and a product in one call, send one reference per target and index each in the prompt ("reference image 1", "reference image 2"), mapping each to its position or its element. Without a prompt the model auto-assigns by visual cues, which works when the references are visually distinct.
• Rights and likeness are the caller's responsibility. Replace does not care whether the source was generated, recorded, or licensed, and it will recast a real person's likeness. Bring source footage and reference images you have the rights to. Do not recast a real, identifiable person without their consent.

Parameters that matter

• taskType: videoInference and deliveryMethod: async. Always polled, never a blocked sync call.
• inputs.video is the source clip. inputs.referenceImages takes up to 3 images. Confirm the exact field names against the live schema, never guess.
• positivePrompt is optional. Required for localised object/garment swaps and for multi-character disambiguation. Names the replace target and the preserve list.
• resolution is 720p or 1080p. Output inherits the source's aspect ratio, so this is the only sizing call. 1080p roughly doubles the per-second cost. Draft a variant batch at 720p, re-run only approved variants at 1080p.
• fps accepts 24 or 48 only. Omit it to inherit the source's native frame rate, which is what most workflows want.
• settings.preserveAudio (default true) controls whether the source audio is in the output at all. Set false for a silent output you intend to re-dub.
• settings.sourceAudioSync (default true) controls whether the source audio drives the character's lip and motion sync. Set false when the audio is incidental (room tone, music) and should not pull the face around.
• seed: fix it to hold a take while iterating, vary it for alternates.

Quality bar

• Only the named element changed. The actor (or the rest of the cast), the motion, timing, camera, lighting, background, and audio of the source are intact.
• Identity or object fidelity reads true: a recast character is recognizably the reference, a swapped product matches its shape, color, and material.
• Audio and face align where it matters. Source audio is preserved by default, so a male reference into a female-voiced source yields a mismatched voice unless you gender-match or strip the audio.
• Watch the failure modes: extreme camera motion or back-of-head framing with no face anchor makes the model improvise a new scene, and tiny details (a logo, small print, a jersey number) may drift. For pixel-perfect detail preservation, use an inpainting model with a mask instead. Retry anything that improvised or drifted.

Related skills

runware-run, runware-models, runware-prompting; edit-video (general video editing/restyle of a whole clip), product-ad-video (generate the source spot, or swap a product into shot footage).