Retell AI vs Vapi: What the Research Tells Us About Voice Agent Latency, TTS, and Optimization

The Retell vs Vapi Question: What We Can and Cannot Say from the Evidence The brief asks for a feature, pricing, and latency comparison of Retell AI and Vapi for building voice agents in 2025. The available research does not directly compare these two platforms. What it does provide is a set of findings about voice-to-voice architecture optimization and a multidimensional benchmark for evaluating voice AI — findings that any developer weighing Retell against Vapi should understand before committing to either platform. This article covers what the evidence supports, where it falls short, and what it means for your platform decision. ## The i-LAVA Paper: Low-Latency Voice-to-Voice Architecture On September 25, 2025, a paper titled i-LAVA: Insights on Low Latency Voice-2-Voice Architecture for Agents was published, classified under Computer Science > Sound. The paper experiments with a voice-to-voice (V-2-V) architecture that utilizes CSM1b, a model with the capability to understand tone and context of conversation by ingesting both audio and text of prior exchanges to generate contextually accurate speech. For anyone building voice agents — whether on Retell, Vapi, or a custom stack — the paper's findings about where latency comes from and how to reduce it are directly relevant. ### Where Latency Lives: TTS Dominates Real Time Factor The paper identifies the text-to-speech (TTS) component as having the highest impact on Real Time Factor (RTF) in the V-2-V system studied. RTF, the ratio the paper uses to measure processing efficiency, is the metric the authors focus on when evaluating latency. This finding matters for platform comparison because both Retell and Vapi abstract away the TTS layer from the developer — but the choice of TTS engine under the hood is what the research suggests drives the largest share of latency, at least in V-2-V systems of this type. It is important to scope this finding correctly. Fact 111 establishes that TTS has the highest impact on RTF in the specific CSM1b-based V-2-V architecture the paper studies. The paper does not claim this holds universally across all voice agent platforms or all architectures. If you are comparing Retell and Vapi on latency, the TTS engine each platform uses is likely the dominant variable in V-2-V architectures of the type studied — not necessarily across every possible configuration. Whether Retell or Vapi use architectures comparable to the one in this paper is not established by the available evidence. ### The Optimization Tradeoff: RVQ Iterations and Mimi Codebooks The paper's experimental evaluations demonstrate that for V-2-V implementations based on CSM, the most important optimizations come from reducing the number of Residual Vector Quantization (RVQ) iterations along with the codebooks used in Mimi. The architecture in the paper uses CSM1b, and the optimizations discussed apply to V-2-V implementations based on CSM. However, the research does not explicitly establish a direct link between CSM and CSM1b; it states the optimizations are for CSM-based implementations and separately notes the architecture uses CSM1b. The critical caveat: optimizing RVQ iterations by the TTS decoder comes at the cost of a decrease in the quality of the generated voice. This is a direct tradeoff between latency and voice quality, and it is the central tension the paper surfaces. For a developer choosing between Retell and Vapi, the question is whether the platform exposes enough control over TTS parameters to let you find the right point on that tradeoff curve — or whether it makes that decision for you. Neither the paper nor the available evidence tells us what level of control Retell or Vapi provide over these specific parameters. ### What the Paper Does Not Cover The i-LAVA paper is focused on a specific architecture using CSM1b. It does not compare commercial platforms. It does not measure Retell or Vapi. It does not provide pricing data. It does not evaluate telephony integrations. It does not assess model flexibility beyond the CSM1b model it uses. These are all gaps that the original brief asks about but that the available evidence cannot fill. ## The Real World Voice EQ Bench: A Framework for Evaluation Separately, a benchmark called the Real World Voice EQ Bench (RW-Voice-EQ Bench) was introduced as a multidimensional benchmark for evaluating voice AI. It evaluates across four dimensions: - Text-to-speech (TTS) - Speech-to-speech (STS) - Speech understanding (SU) - Automatic speech recognition (ASR) For TTS specifically, the benchmark treats naturalness, expressiveness, identity stability, and reliability as largely independent evaluation dimensions. This means a TTS engine can score well on one dimension and poorly on another — a platform that sounds natural may not be reliable, and one that preserves speaker identity may lack expressiveness. For STS, the benchmark finds that access to audio does not guarantee use of vocal affect, and some agents remain largely transcript-driven. In other words, a speech-to-speech system that receives audio input may still effectively ignore the emotional content of that audio and process only the transcript. This is a meaningful distinction for voice agents: if your use case depends on responding to caller emotion — frustration, urgency, sarcasm — a platform whose STS is transcript-driven will miss those signals regardless of how it is marketed. ### What the Benchmark Does Not Do The RW-Voice-EQ Bench provides a framework for evaluation. It does not, based on the available evidence, publish scores for Retell or Vapi. It does not provide a ranking of commercial platforms. It does not address pricing. It does not address telephony integrations or per-minute cost. It is a methodology and a set of evaluation dimensions, not a consumer-facing comparison. ## What This Means for Retell vs Vapi Given the evidence available, here is what we can responsibly say: On latency: The i-LAVA paper identifies TTS as the highest-impact component on RTF in the CSM1b-based V-2-V architecture it studies, and shows that reducing RVQ iterations and Mimi codebooks is the most effective optimization — at the cost of voice quality. If you are evaluating Retell and Vapi for latency-sensitive applications, ask which TTS engine each platform uses and whether you can tune parameters analogous to RVQ iterations. The research does not tell us whether either platform exposes these controls. On voice quality: The RW-Voice-EQ Bench establishes that TTS naturalness, expressiveness, identity stability, and reliability are independent dimensions. A platform that excels in one may fail in another. When comparing Retell and Vapi, evaluate across all four dimensions rather than relying on a single quality metric. On speech understanding: The benchmark's finding that some STS agents remain transcript-driven despite receiving audio input is relevant if your use case requires emotion-aware responses. Ask whether each platform's STS pipeline actually uses vocal affect or merely processes transcripts. On pricing: The available evidence does not provide pricing data for either Retell or Vapi. We cannot assert per-minute costs, tier differences, or volume discounts from the research provided. This is a gap that requires direct inquiry to each vendor. On telephony integrations: The available evidence does not address telephony integrations for either platform. This is another gap requiring direct vendor research. On model flexibility: The i-LAVA paper uses CSM1b specifically. The RW-Voice-EQ Bench evaluates across TTS, STS, SU, and ASR but does not, based on available evidence, assess which models each commercial platform supports. Whether Retell or Vapi allow you to swap TTS engines, ASR providers, or LLM backends is not established by the research. ## The Bottom Line The research available gives you a framework for asking the right questions when comparing Retell and Vapi — which TTS engine drives latency, whether quality tradeoffs are tunable, whether STS uses vocal affect, and which quality dimensions matter for your use case. It does not answer those questions for either platform. To close the gap between what the research tells us and what a platform decision requires, you would need to: 1. Ask each vendor which TTS engine they use and whether latency-critical parameters are tunable. 2. Evaluate each platform's TTS output across naturalness, expressiveness, identity stability, and reliability, using the RW-Voice-EQ Bench dimensions as a checklist. 3. Test whether each platform's STS pipeline is transcript-driven or uses vocal affect, if your use case depends on emotion awareness. 4. Obtain pricing sheets and telephony integration documentation directly from each vendor. The research sharpens the questions. The answers still require direct engagement with the platforms.