I’ve been doing a ragtag version of this with sub-agents, TTS, and STT in Claude Code. Real-time would be pretty awesome, even if it’s just orchestrating other agents. I’m might have to try this on top of my Claude agents. I don’t think the model doing the talking necessarily needs to do the heavy reasoning - just needs to have context on your other agents, delegate, and remain present.
Looks like the GPT‑Realtime‑Whisper model isn’t open weight like the old whisper model. Too bad!
However, OpenAI had and still has a true lead on voice model interactions. That’s where Chinese AI companies don’t do as well: deepseek doesn’t have anything or like Kimi that can speak out in any language except English or Chinese.
I’ve been doing a ragtag version of this with sub-agents, TTS, and STT in Claude Code. Real-time would be pretty awesome, even if it’s just orchestrating other agents. I’m might have to try this on top of my Claude agents. I don’t think the model doing the talking necessarily needs to do the heavy reasoning - just needs to have context on your other agents, delegate, and remain present.
Looks like the GPT‑Realtime‑Whisper model isn’t open weight like the old whisper model. Too bad!
However, OpenAI had and still has a true lead on voice model interactions. That’s where Chinese AI companies don’t do as well: deepseek doesn’t have anything or like Kimi that can speak out in any language except English or Chinese.
> $32 / 1M audio input tokens ($0.40 for cached input tokens)
Anyone know how much audio is 1M tokens? I have no way of knowing if this is fine or prohibitively expensive.
“Perfect! “
Says the team at OpenAI whose job it is to ensure you thought that.
Fortunately there’s real competition in the voice ai field.
Presumably because it’s genuinely useful - I can easily think of applications to make with a powerful voice ui.