agi.jetzt
DEEN

Model landscape

Speech & Audio

Models that understand speech, speak, hold a conversation in real time or generate music. We name the model, not the app around it.

Last checked: 10 Oct 2026

What this class is

The class covers four tasks: understanding speech (transcription), generating speech, speech-to-speech conversation in real time, and generating music or other audio.

A voice mode in an app is a product surface. Behind it stands a model, and that model is what this page lists. Where a provider offers no audio model of its own and runs its voice feature on text models, there is no entry here.

What is it optimized for?
For sound in real time: understanding speech reliably, speaking naturally, and answering fast enough for a conversation.
How does it differ from a generalist?
A generalist works on text. A speech model works on the sound itself — including what text loses: tone, pauses, emotion, two people talking at once.
What does it output?
Text from speech, speech from text, speech directly from speech, or music.
Where is it useful?
Voice assistants and phone agents, transcripts and subtitles, dubbing and voice-over, accessibility, music drafts.

One model instead of a chain

For a long time a voice assistant was three systems in a row. The newer models do it in one.

  1. The chainSpeech recognition turns sound into text, a language model answers in text, speech synthesis reads the answer out. According to the Moshi paper this costs several seconds between turns, and what is not words — emotion, non-speech sounds — is lost on the way.
  2. Speech-to-speechOne model takes sound in and gives sound out. Tone and timing can stay in the signal.
  3. Full duplexThe model listens while it speaks. That allows interruptions and short acknowledgements, as in a conversation between people.

Five notable models

A selection with reasons, not a ranking; rank claims of the providers are not repeated here.

Speech-to-speech in real time

Models that hold a spoken conversation.

Speech generation

Text becomes speech.

  • ElevenLabs

    Eleven v4

    Availablereleased 28 Sept 2026
    Purpose
    Generates speech from text.
    What is notable
    Delivery is steered by instructions inside the text, such as [laughs] or a described manner of speaking.
    Why it is here
    Speech output where tone and emotion are specified on purpose.
    Access
    In the ElevenLabs products and through the API, together with the faster variant Eleven v4 Turbo.
    Input → Output
    Text with delivery instructions → Speech

    Sources: ElevenLabs: Eleven v4 (28 September 2026), ElevenLabs docs: Models · checked 10 Oct 2026

    Model page in the registry: Eleven v4 →

Speech understanding

Speech becomes text.

Music

Generated music and audio.

  • Google

    Lyria 3.5

    Availablereleased 3 Sept 2026
    Purpose
    Generates music, up to full-length songs.
    What is notable
    According to the documentation every generated recording carries a SynthID watermark; output is 44.1 kHz stereo.
    Why it is here
    Music generation as a programming interface, with the output marked by default.
    Access
    Through the Gemini API (model ID lyria-3.5).
    Input → Output
    Text → Music, 44.1 kHz stereo

    Sources: Gemini API docs: Music generation, Gemini API changelog · checked 10 Oct 2026

    Model page in the registry: Lyria 3.5 →

No rankingThe order is the order of this list. agi.jetzt does not grade these models and does not compare them with one another.

Why is this relevant to AGI?

Speech is how people communicate most naturally. A system that perceives and answers in real time, including tone and timing, interacts differently from one that waits for typed text.

For general intelligence this matters as perception and communication: hearing what is meant, not only what is said, and reacting while the other side is still speaking.

What this does not prove about AGI

Sounding human is not thinking. A fluent voice says nothing about whether what is said is correct.

Relevance to AGI does not mean demonstrated progress toward AGI. These models are not part of the frontier models, and nothing on this page enters AGI Progress.

Research behind the general statements

Sources

All sources read on 10 Oct 2026.