We built a real-time voice platform with a measured 700–1000ms warm round trip per conversational turn: browser voice-activity detection into Groq Whisper Turbo, a routed Llama 3.3 70B call with fallback, then ElevenLabs Flash v2.5 synthesis. Twelve personas, a 13-package monorepo, and a jurisdiction-aware compliance layer.
Conversational latency is not a performance metric, it is the product. Human turn-taking tolerates roughly a second of silence before the exchange stops feeling like a conversation and starts feeling like a query interface. Every additional component in the pipeline — voice detection, transcription, inference, synthesis, network — spends part of that budget, and there are five of them. The naive implementation is sequential and blocking: wait for the user to finish, upload the whole utterance, wait for a transcript, send it to a model, wait for the full completion, send that to a synthesiser, wait for the whole audio file, play it. That architecture cannot get under two seconds no matter how fast each component is, because it never overlaps anything. The platform is an adult-oriented conversational service, which adds a second and unrelated engineering problem: a compliance surface that is genuinely jurisdiction-dependent, where the correct behaviour differs between US states and between countries, and where getting it wrong is a legal exposure rather than a bug.
Turn detection happens in the browser rather than on the server. A voice-activity detector ticks at 60fps against the Web Audio analyser's RMS, and a turn ends on roughly 300ms of speech followed by 800ms of silence. Doing this client-side means the server is never streaming and transcribing silence, and the endpoint decision costs no network round trip. Transcription is Groq's Whisper Turbo. Inference is Llama 3.3 70B, also on Groq. Synthesis is ElevenLabs Flash v2.5 — the low-latency model in their range, chosen over the higher-quality options specifically because in a real-time turn, latency is the quality dimension that matters and a slightly better timbre arriving 600ms later is worse. `ARCHITECTURE.md` records the outcome directly: end-to-end latency, measured live and warm, of 700–1000ms per turn. The word 'warm' is doing real work in that sentence and we are keeping it — a cold first turn is slower, and any latency figure quoted without that qualifier should be distrusted.
`packages/llm-router` is a separate workspace package with its own `router.ts`, `prompt.ts` and `memory.ts`. Model selection, provider fallback and conversation memory live behind one interface rather than being scattered through the application. This is a small piece of architecture with a large payoff and we build it on nearly every AI project now. Inference providers have outages, rate limits and silent quality regressions on model updates. If your application calls a provider SDK directly from a dozen call sites, a provider incident is an outage and a model migration is a refactor. If it calls a router, both are configuration. The same monorepo pattern is applied throughout: 13 packages (`voice-engine`, `visual-engine`, `avatar-stream`, `llm-router`, `compliance`, `db`, `email`, `feature-flags`, `rate-limit`, `trpc`, `types`, `ui`, `logger`) and 7 standalone services for the heavier GPU workloads. Turborepo handles the build graph. The boundaries are drawn where the failure domains are, which is why `compliance` and `rate-limit` are packages rather than folders in the web app.
Age verification is not applied globally. It is applied where it is legally required, and the requirement is per-jurisdiction: `ARCHITECTURE.md` enumerates a set of US states with age-verification statutes plus international jurisdictions including the UK and Australia. Encoding that as a data-driven policy inside a `compliance` package — rather than as an `if` statement in a page component — is what makes it maintainable as the list changes, which it does, frequently. The public surface is deliberately separated from the gated one: marketing, pricing, demo, legal (terms, privacy, DMCA, 2257, do-not-sell), sign-in, account, billing, GDPR and status pages are all reachable without passing the gate. That separation is both a compliance requirement and an SEO one — the indexable surface and the gated surface are different sets by construction rather than by convention. `rate-limit` being a package on the same footing is the other half of that posture. In a system where each turn costs money at three vendors, rate limiting is not an anti-abuse afterthought — it is cost control, and it has to be enforced at a layer that every entry point passes through rather than added to whichever route someone remembers.
The monorepo separates code that runs on ordinary compute from code that needs a GPU. The thirteen `packages/*` workspaces are libraries consumed by the applications. The seven `services/*` workspaces — `flux-inference`, `gaussian-avatar`, `hunyuan-video`, `lora-trainer`, `vllm-uncensored`, `voice-trainer` and `wav2lip-local` — are standalone deployables, each wrapping a model that wants its own machine, its own memory profile and its own scaling behaviour. The reason to draw the boundary there rather than anywhere else is operational. A GPU service has a cold start measured in tens of seconds, a cost measured per hour rather than per request, and a failure mode where it simply is not available. Every one of those properties is incompatible with sitting inside a request-serving application, and a monorepo that pretends otherwise produces a web app that cannot deploy without a GPU. Three applications sit on top — `web`, `admin` and `creator-portal` — with Turborepo managing the build graph so a change in `llm-router` rebuilds only what depends on it. Testing is split the same way it is in most honest codebases: Vitest for the units that have deterministic answers, Playwright for the flows that only mean anything end to end, such as passing the age gate and completing a call.
Endpointing — deciding when the human has stopped talking — is the hardest problem in the pipeline and the one nobody expects. Wait too long and every turn feels sluggish; cut too early and you interrupt someone mid-sentence, which is far worse because it breaks the illusion of conversation entirely and cannot be recovered from within the turn. The 300ms-speech / 800ms-silence thresholds are not defaults, they are the output of tuning, and they are the single most consequential pair of numbers in the system. Getting under a second at all requires overlapping stages rather than sequencing them, and every overlap introduces a cancellation problem. If the user starts speaking again while synthesis is still streaming, the correct behaviour is to abort the synthesis, discard the in-flight completion, and start a new turn — cleanly, without leaking a stream or double-charging an API call. Barge-in is where real-time voice implementations usually break. The compliance surface is hard for a reason that has nothing to do with difficulty of code: the rules are a moving target set by twenty different legislatures, some of the requirements conflict, and the cost of being wrong is not a bad user experience. Building it as a policy-driven package rather than as scattered conditionals was the only structure we could see that survives the list changing every few months.
Built with TypeScript (pnpm + Turborepo), Next.js, Groq (Whisper Turbo STT), Llama 3.3 70B, ElevenLabs Flash v2.5, tRPC, Browser VAD (Web Audio analyser), Playwright + Vitest and Docker Compose. Relationship: Internal build. Stage: Prototype. Period: 2026. Our role: Voice pipeline, LLM routing, monorepo architecture, compliance surface Figures re-derived on 2026-08-23.