Tongyi VideoRetalk lip sync / lip-sync (mouth sync, dubbing) video model — takes a talking-person video plus a voice audio track and regenerates the video so the speaker's mouth/lips match the new audio. Use this for lip syncing a person video to new speech. Optionally provide a reference face image to pick the target person when the video contains multiple faces. 通义声动人像 VideoRetalk 口型同步(对口型、lip sync / lip-sync、配音对嘴)视频模型,输入一段人物讲话视频与一段人声音频,生成讲话口型与音频匹配的新视频;适用于让人物视频的口型对上新的语音。当视频中存在多张人脸时,可额外提供人脸参考图来指定要替换口型的目标人物。
Design & media
Alibaba Cloud AI Video Retalk
Try itUse when replacing lip sync in existing videos with Alibaba Cloud Model Studio VideoRetalk (`videoretalk`). Use when creating dubbed videos, replacing narrat...
What it does
Use when replacing lip sync in existing videos with Alibaba Cloud Model Studio VideoRetalk (`videoretalk`). Use when creating dubbed videos, replacing narrat...
The skill document
Category: provider
Model Studio VideoRetalk
Validation
mkdir -p output/aliyun-videoretalk
python -m py_compile skills/ai/video/aliyun-videoretalk/scripts/prepare_retalk_request.py && echo "py_compile_ok" > output/aliyun-videoretalk/validate.txt
Pass criteria: command exits 0 and output/aliyun-videoretalk/validate.txt is generated.
Output And Evidence
- Save normalized request payloads, target face selection settings, and task polling snapshots under
output/aliyun-videoretalk/. - Record the exact video/audio input URLs and whether
video_extensionwas enabled.
Use VideoRetalk when the input is already a person video and the job is to replace lip sync with a new speech track.
Critical model names
Use this exact model string:
videoretalk
Prerequisites
- This model currently only supports China mainland (Beijing).
- API is HTTP async only; there is no online console experience.
- Set
DASHSCOPE_API_KEYin your environment, or adddashscope_api_keyto~/.alibabacloud/credentials.
Normalized interface (video.retalk)
Request
model(string, optional): defaultvideoretalkvideo_url(string, required)audio_url(string, required)ref_image_url(string, optional): target face when input video contains multiple facesvideo_extension(bool, optional): extend video to match longer audioquery_face_threshold(int, optional):120to200
Response
task_id(string)task_status(string)video_url(string, when finished)usage(object, optional)
Endpoint and execution model
- Submit task:
POST https://dashscope.aliyuncs.com/api/v1/services/aigc/image2video/video-synthesis/ - Poll task:
GET https://dashscope.aliyuncs.com/api/v1/tasks/{task_id} - HTTP calls are async only and must set header
X-DashScope-Async: enable.
Quick start
python skills/ai/video/aliyun-videoretalk/scripts/prepare_retalk_request.py \
--video-url "https://example.com/talking-head.mp4" \
--audio-url "https://example.com/new-voice.wav" \
--video-extension
Operational guidance
- Keep input videos front-facing and close enough for stable face tracking.
- If the video contains multiple faces, provide
ref_image_urlto anchor the intended target. - If the new audio is longer than the input video, decide explicitly whether to extend the picture track or truncate the audio.
- URLs must be public HTTP/HTTPS links; local file paths are not accepted by the API.
Output location
- Default output:
output/aliyun-videoretalk/request.json - Override base dir with
OUTPUT_DIR.
References
references/sources.md
Related skills
Use when generating lightweight talking-head portrait videos with Alibaba Cloud Model Studio LivePortrait (`liveportrait`) from a detected portrait image and...
Use when generating talking, singing, or presentation videos from a single character image and audio with Alibaba Cloud Model Studio digital-human model `wan...
Use when generating expressive portrait videos from a person image and speech audio with Alibaba Cloud Model Studio EMO (`emo-v1`). Use when creating non-Wan...
Generate lip-sync video with Volcengine Lip Sync through RunAPI. Use when the user asks an agent to sync mouth movement in a source video to an audio track. Default to the RunAPI CLI for one-off generation; use SDKs only when integrating RunAPI into an app or backend.
Create and manage Alibaba Cloud IMS video translation jobs via OpenAPI (subtitle/voice/face). Use when you need API-based video translation, status polling,...