VedicTHG: Symbolic Vedic Computation for Low-Resource Talking-Head Generation in Educational Avatars

Talking-head avatars are increasingly adopted in educational technology to deliver content with social presence and improved engagement. However, many recent talking-head generation (THG) methods rely on GPU-centric neural rendering, large training sets, or high-capacity diffusion models, which limits deployment in offline or resource-constrained learning environments. A deterministic and CPU-oriented THG framework is described, termed Symbolic Vedic Computation, that converts speech to a time-aligned phoneme stream, maps phonemes to a compact viseme inventory, and produces smooth viseme trajectories through symbolic coarticulation inspired by Vedic sutra Urdhva Tiryakbhyam. A lightweight 2D renderer performs region-of-interest (ROI) warping and mouth compositing with stabilization to support real-time synthesis on commodity CPUs. Experiments report synchronization accuracy, temporal stability, and identity consistency under CPU-only execution, alongside benchmarking against representative CPU-feasible baselines. Results indicate that acceptable lip-sync quality can be achieved while substantially reducing computational load and latency, supporting practical educational avatars on low-end hardware. GitHub: https://vineetkumarrakesh.github.io/vedicthg

💡 Research Summary

The paper introduces VedicTHG, a deterministic, CPU‑only talking‑head generation pipeline designed for low‑resource educational avatars. Recognizing that state‑of‑the‑art neural THG systems demand GPUs, large training corpora, and high‑capacity diffusion models, the authors propose a lightweight alternative that separates audio control from visual synthesis. The system first converts speech into a time‑aligned phoneme stream using either forced alignment with a transcript or a compact MFCC‑based recognizer. Each phoneme is then mapped deterministically to a compact viseme inventory (12–20 classes) via a fixed lookup table, eliminating any learning requirement.

To achieve smooth mouth motion, the authors devise a symbolic co‑articulation operator inspired by the Vedic sutra Urdhva Tiryakbhyam. For a transition between two consecutive visemes a and c, the mouth parameter trajectory is computed as
y(t) = (1‑α)a + αc + λ α(1‑α)(a⊙c),
where α is a linear overlap weight and ⊙ denotes element‑wise multiplication. The cross‑term λ α(1‑α)(a⊙c) provides curvature that peaks mid‑transition, avoiding the “snap” of pure linear interpolation while remaining a simple vectorized operation suitable for CPUs.

The visual front‑end is a 2D ROI renderer. A lightweight landmark detector supplies mouth landmarks; an exponential moving average stabilizes the mouth bounding box. A pre‑prepared mouth texture bank supplies the appropriate viseme patch, which is warped to the current ROI and composited with an alpha mask. A modest affine transform applied only to the head region adds natural head motion without increasing computational load. The rest of the face remains unchanged from the reference image, preserving identity.

Experiments were conducted on public corpora (GRID, TCD‑TIMIT, LRS2/LRS3, VoxCeleb) using a 16‑core Xeon CPU. Metrics included lip‑sync accuracy (percentage of phoneme‑viseme events within ±40 ms), SyncNet distance, runtime (latency, FPS, peak CPU utilization), identity drift (FaceNet/ArcFace cosine distance), and perceptual similarity (LPIPS, SSIM). VedicTHG achieved 30 fps with an average frame latency of 26.7 ms and peak CPU usage of 29 %, while maintaining 100 % sync accuracy (median error 12.5 ms, 95th percentile 23.8 ms). In contrast, a CPU‑only version of Wave2Lip required 957 ms per frame, 1.04 fps, and 811 % CPU utilization, demonstrating a >30× efficiency gain.

Ablation studies showed that removing the Vedic cross‑term degrades transition smoothness, increasing jitter near phoneme boundaries. Varying the overlap margin Δ and stabilization factor β revealed a trade‑off between temporal stability and responsiveness; the chosen defaults (Δ = 40 ms, λ = 0.2, β = 0.85) offered the best balance.

Overall, VedicTHG provides a reproducible, training‑free, deterministic pipeline that delivers acceptable lip‑sync quality and identity preservation on commodity CPUs, making it suitable for offline or low‑bandwidth educational deployments where hardware resources are limited. Future work may extend the approach to 3D facial models, multilingual support, and user‑customizable viseme sets to further enhance realism and applicability.