Hugging Face and Cerebras have joined forces to bring Google's Gemma 4 model into real-time voice AI applications, leveraging a WebSocket-based speech-to-speech pipeline. The collaboration is built around the open-source HF Realtime Voice project, which enables live, bidirectional voice conversations powered by large language models.

How It Works

The system connects users to a voice interface through WebSockets, handling the full speech-to-speech loop — from audio input to model inference to spoken output — with minimal latency.

Key components of the pipeline include:

  • Speech-to-text transcription of incoming audio
  • Gemma 4 LLM inference served via Cerebras hardware for high-throughput, low-latency responses
  • Text-to-speech synthesis to deliver natural-sounding voice output
  • Real-time WebSocket transport to maintain a persistent, low-overhead connection

Why Cerebras?

Cerebras brings its purpose-built AI inference hardware to the stack, enabling the kind of token throughput that real-time voice demands. Voice applications are uniquely latency-sensitive — even a few hundred milliseconds of delay breaks the conversational feel.

Cerebras inference makes it possible to run Gemma 4 fast enough that the pipeline feels genuinely real-time to end users.

Traditional GPU-based inference often struggles to meet these constraints at scale. Cerebras' wafer-scale architecture sidesteps memory bandwidth bottlenecks that typically slow down autoregressive generation.

Gemma 4 at the Core

Gemma 4 is Google's latest open model, offering strong multilingual performance and instruction-following capability — both critical for voice assistants that need to handle diverse, open-ended queries.

The choice of Gemma 4 signals a broader trend: open-weight models are becoming viable alternatives to proprietary APIs in production voice pipelines, especially when paired with fast inference backends.

What Developers Get

The project is available through Hugging Face, giving developers a ready-made reference architecture for building voice AI applications. The stack is designed to be modular:

  • Swap in different STT or TTS components
  • Run against Cerebras-hosted inference or self-hosted endpoints
  • Extend with custom system prompts, personas, or domain-specific logic

This kind of open, composable approach lowers the barrier for teams looking to ship voice features without building the entire pipeline from scratch.