- Client
- Zencia.ai
- Industry
- Applied AI
- Role
- Architecture, GPU pipeline, deployment
- Platform
- Web
- Stack
- PyTorch · CUDA · Flask · WebSocket · ffmpeg
- Dated
- 2026-07-15
- Realtime spoken conversation with a lip-synced photoreal face
- Zero-disk frame pipeline — frames stream straight into the encoder
- Segmented replies so speech begins before generation completes
- Seamless idle loop between utterances
- Avatar preparation from a single photo or short video
- Serialised GPU access with resident-avatar eviction
① Challenge
Lip-sync generation is slow enough that the naive approach — wait for the full answer, render the whole clip, then play it — leaves the viewer staring at a frozen face for several seconds. The stock pipeline also wrote hundreds of intermediate frames to disk per sentence, which throttled everything downstream.
② Solution
Raw frames are piped straight into the video encoder instead of being written out as image files, removing the disk bottleneck entirely. Replies are cut into segments — the first few seconds ship as soon as they exist, longer chunks follow — so the avatar starts speaking while the rest of the answer is still generating. A seamless idle loop plays between utterances so the face never freezes, and GPU access is serialised behind a single lock so concurrent requests queue instead of colliding.
③ Result
Video starts playing about a second into a request rather than after the full clip renders. Runs as always-on services on a GPU instance, with new avatars preparable from a single photo or a short video clip.