OpenAI released three new audio models through its Realtime API, each targeting a distinct capability in live voice applications: GPT-Realtime-2 for voice agents with reasoning, GPT-Realtime-Translate for live speech translation, and GPT-Realtime-Whisper for streaming transcription. Alongside the model releases, the Realtime API officially exits beta and is now generally available \u2014 a meaningful signal for developers who held off building production systems on it. All three models are available immediately through the OpenAI API and can be tested in the Playground.<\/p>\n
Together, they push voice applications past the basic question-and-answer loop \u2014 toward systems that can listen, reason, translate, transcribe, and act within a single conversation.<\/p>\n
The flagship release is GPT-Realtime-2, which OpenAI team describes as its first voice model with GPT-5-class reasoning. GPT-Realtime-2 can process harder requests, manage interruptions, and continue conversations naturally. OpenAI expanded the model\u2019s context window from 32K to 128K tokens, allowing longer conversations and more complex tasks without losing context.<\/p>\n
Previous voice models frequently stalled on multi-step requests or dropped earlier context during longer sessions. GPT-Realtime-2 is specifically designed to keep the conversation moving while it reasons through a request.<\/p>\n
Developers can enable short preamble phrases \u2014 like \u201clet me check that\u201d or \u201cone moment while I look into it\u201d \u2014 so users know the agent is working on the request. The model can also call multiple tools at once and narrate what it\u2019s doing while it does \u2014 so instead of dead air during a multi-step task, the user gets a running commentary. These features directly address one of the most common failure modes in deployed voice agents: awkward silence that makes the system feel broken.<\/p>\n
A particularly useful control for production builders is adjustable reasoning effort. Developers can dial reasoning intensity across five levels<\/strong>: minimal, low, medium, high, and xhigh. The default is \u201clow\u201d to keep latency down for simple requests, while tougher tasks can tap into more compute. This means teams can tune the performance-latency tradeoff at the session level depending on the use case \u2014 a quick customer lookup doesn\u2019t need the same reasoning depth as a multi-step travel booking workflow.<\/p>\n GPT-Realtime-2 also adds tone control. The model can adjust its speaking style depending on the situation \u2014 staying calm during problem-solving, shifting to empathetic when users are frustrated, and turning upbeat after a successful outcome. The model is also better at understanding industry-specific terminology, including healthcare vocabulary and proper nouns.<\/p>\n On benchmarks, the gains are measurable. GPT-Realtime-2 with high reasoning scored 96.6% on Big Bench Audio, compared to 81.4% for GPT-Realtime-1.5 \u2014 a 15.2 percentage point improvement. GPT-Realtime-2 with xhigh reasoning scored 48.5% on Audio MultiChallenge instruction following, compared to 34.7% for GPT-Realtime-1.5. <\/p>\n Big Bench Audio evaluates challenging reasoning capabilities in language models that support audio input. Audio MultiChallenge evaluates multi-turn conversational intelligence in spoken dialogue systems, including instruction following, context integration, self-consistency, and handling natural speech corrections.<\/p>\n Pricing:<\/strong> GPT-Realtime-2 is priced at $32 per 1M audio input tokens ($0.40 for cached input tokens) and $64 per 1M audio output tokens.<\/p>\n GPT-Realtime-Translate is a new live translation model that translates speech from 70+ input languages into 13 output languages while keeping pace with the speaker. Unlike GPT-Realtime-2, this model is a dedicated translation pipe \u2014 speech goes in one language and comes out in another. It is not a conversational agent; it is designed to convert one audio stream into another in real time.<\/p>\n The distinction is important for developers choosing the right tool. If your application needs a bilingual customer support flow or a live interpreter for an in-person event, GPT-Realtime-Translate is the purpose-built option. If you need the model to also reason, call functions, or hold context across turns, GPT-Realtime-2 handles that.<\/p>\n Pricing:<\/strong> GPT-Realtime-Translate is priced at $0.034 per minute.<\/p>\n GPT-Realtime-Whisper is a new streaming speech-to-text model built for low-latency speech-to-text \u2014 transcribing audio as people speak, so live products can feel faster, more responsive, and more natural.<\/p>\n The original Whisper model was designed for completed chunks of audio, making it better suited for post-session transcription. GPT-Realtime-Whisper is the streaming counterpart, purpose-built for applications that need live output. For realtime transcription, gpt-realtime-whisper gives you controllable latency \u2014 lower delay settings produce earlier partial text, while higher delay settings can improve transcript quality. <\/p>\n Use cases include live broadcast captions, meeting notes generated during the conversation, and voice agents that need to continuously understand the user rather than wait for turn-by-turn input.<\/p>\n Pricing:<\/strong> GPT-Realtime-Whisper is priced at $0.017 per minute.<\/p>\n Developers can choose between three session types<\/strong> depending on the use case: a voice-agent session when the application needs an assistant that responds to the user, a translation session when the application needs an interpreter, and a transcription session when text from audio is needed without model-generated responses.<\/p>\n On the voice output side, two new voices, Cedar and Marin, join the API roster exclusively with this release.<\/p>\n All three models \u2014 GPT-Realtime-2, GPT-Realtime-Translate, and GPT-Realtime-Whisper \u2014 are available now through the OpenAI Realtime API, which is generally available starting today.<\/p>\n Check out\u00a0the\u00a0Full Technical Details here<\/a><\/strong>.\u00a0<\/strong>Also,\u00a0feel free to follow us on\u00a0Twitter<\/mark><\/a><\/strong>\u00a0and don\u2019t forget to join our\u00a0150k+ ML SubReddit<\/a><\/strong>\u00a0and Subscribe to\u00a0our Newsletter<\/a><\/strong>. Wait! are you on telegram?\u00a0now you can join us on telegram as well.<\/a><\/strong><\/p>\n Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.?\u00a0Connect with us<\/mark><\/a><\/strong><\/p>\n The post OpenAI Releases Three Realtime Audio Models: GPT-Realtime-2, GPT-Realtime-Translate, and GPT-Realtime-Whisper in the Realtime API<\/a> appeared first on MarkTechPost<\/a>.<\/p>","protected":false},"excerpt":{"rendered":" OpenAI released three new audio models through its Realtime API, each targeting a distinct capability in live voice applications: GPT-Realtime-2 for voice agents with reasoning, GPT-Realtime-Translate for live speech translation, and GPT-Realtime-Whisper for streaming transcription. Alongside the model releases, the Realtime API officially exits beta and is now generally available \u2014 a meaningful signal for developers who held off building production systems on it. All three models are available immediately through the OpenAI API and can be tested in the Playground. Together, they push voice applications past the basic question-and-answer loop \u2014 toward systems that can listen, reason, translate, transcribe, and act within a single conversation. GPT-Realtime-2: Voice Reasoning with a 128K Context Window The flagship release is GPT-Realtime-2, which OpenAI team describes as its first voice model with GPT-5-class reasoning. GPT-Realtime-2 can process harder requests, manage interruptions, and continue conversations naturally. OpenAI expanded the model\u2019s context window from 32K to 128K tokens, allowing longer conversations and more complex tasks without losing context. Previous voice models frequently stalled on multi-step requests or dropped earlier context during longer sessions. GPT-Realtime-2 is specifically designed to keep the conversation moving while it reasons through a request. Developers can enable short preamble phrases \u2014 like \u201clet me check that\u201d or \u201cone moment while I look into it\u201d \u2014 so users know the agent is working on the request. The model can also call multiple tools at once and narrate what it\u2019s doing while it does \u2014 so instead of dead air during a multi-step task, the user gets a running commentary. These features directly address one of the most common failure modes in deployed voice agents: awkward silence that makes the system feel broken. A particularly useful control for production builders is adjustable reasoning effort. Developers can dial reasoning intensity across five levels: minimal, low, medium, high, and xhigh. The default is \u201clow\u201d to keep latency down for simple requests, while tougher tasks can tap into more compute. This means teams can tune the performance-latency tradeoff at the session level depending on the use case \u2014 a quick customer lookup doesn\u2019t need the same reasoning depth as a multi-step travel booking workflow. GPT-Realtime-2 also adds tone control. The model can adjust its speaking style depending on the situation \u2014 staying calm during problem-solving, shifting to empathetic when users are frustrated, and turning upbeat after a successful outcome. The model is also better at understanding industry-specific terminology, including healthcare vocabulary and proper nouns. On benchmarks, the gains are measurable. GPT-Realtime-2 with high reasoning scored 96.6% on Big Bench Audio, compared to 81.4% for GPT-Realtime-1.5 \u2014 a 15.2 percentage point improvement. GPT-Realtime-2 with xhigh reasoning scored 48.5% on Audio MultiChallenge instruction following, compared to 34.7% for GPT-Realtime-1.5. Big Bench Audio evaluates challenging reasoning capabilities in language models that support audio input. Audio MultiChallenge evaluates multi-turn conversational intelligence in spoken dialogue systems, including instruction following, context integration, self-consistency, and handling natural speech corrections. Pricing: GPT-Realtime-2 is priced at $32 per 1M audio input tokens ($0.40 for cached input tokens) and $64 per 1M audio output tokens. GPT-Realtime-Translate: Live Speech Translation Across 70+ Languages GPT-Realtime-Translate is a new live translation model that translates speech from 70+ input languages into 13 output languages while keeping pace with the speaker. Unlike GPT-Realtime-2, this model is a dedicated translation pipe \u2014 speech goes in one language and comes out in another. It is not a conversational agent; it is designed to convert one audio stream into another in real time. The distinction is important for developers choosing the right tool. If your application needs a bilingual customer support flow or a live interpreter for an in-person event, GPT-Realtime-Translate is the purpose-built option. If you need the model to also reason, call functions, or hold context across turns, GPT-Realtime-2 handles that. Pricing: GPT-Realtime-Translate is priced at $0.034 per minute. GPT-Realtime-Whisper: Streaming Transcription as People Speak GPT-Realtime-Whisper is a new streaming speech-to-text model built for low-latency speech-to-text \u2014 transcribing audio as people speak, so live products can feel faster, more responsive, and more natural. The original Whisper model was designed for completed chunks of audio, making it better suited for post-session transcription. GPT-Realtime-Whisper is the streaming counterpart, purpose-built for applications that need live output. For realtime transcription, gpt-realtime-whisper gives you controllable latency \u2014 lower delay settings produce earlier partial text, while higher delay settings can improve transcript quality. Use cases include live broadcast captions, meeting notes generated during the conversation, and voice agents that need to continuously understand the user rather than wait for turn-by-turn input. Pricing: GPT-Realtime-Whisper is priced at $0.017 per minute. Architecture Patterns and New Voices Developers can choose between three session types depending on the use case: a voice-agent session when the application needs an assistant that responds to the user, a translation session when the application needs an interpreter, and a transcription session when text from audio is needed without model-generated responses. On the voice output side, two new voices, Cedar and Marin, join the API roster exclusively with this release. All three models \u2014 GPT-Realtime-2, GPT-Realtime-Translate, and GPT-Realtime-Whisper \u2014 are available now through the OpenAI Realtime API, which is generally available starting today. Key Takeaways GPT-Realtime-2 brings GPT-5-class reasoning to voice with a 128K context window, five-level adjustable reasoning effort, tone control, parallel tool calls, and interruption recovery On Big Bench Audio, GPT-Realtime-2 (high) scores 96.6% vs. 81.4% for GPT-Realtime-1.5; on Audio MultiChallenge, the xhigh variant scores 48.5% vs. 34.7%. GPT-Realtime-Translate handles live speech translation across 70+ input languages into 13 output languages at $0.034\/min GPT-Realtime-Whisper streams transcription in real time with controllable latency at $0.017\/min The Realtime API exits beta and goes generally available today alongside two new voices, Cedar and Marin Check out\u00a0the\u00a0Full Technical Details here.\u00a0Also,\u00a0feel free to follow us on\u00a0Twitter\u00a0and don\u2019t forget to join our\u00a0150k+ ML SubReddit\u00a0and Subscribe to\u00a0our Newsletter. Wait! are you on telegram?\u00a0now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.?\u00a0Connect<\/p>","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"pmpro_default_level":"","site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"_pvb_checkbox_block_on_post":false,"footnotes":""},"categories":[52,5,7,1],"tags":[],"class_list":["post-88981","post","type-post","status-publish","format-standard","hentry","category-ai-club","category-committee","category-news","category-uncategorized","pmpro-has-access"],"acf":[],"yoast_head":"\nGPT-Realtime-Translate: Live Speech Translation Across 70+ Languages<\/strong><\/h3>\n
GPT-Realtime-Whisper: Streaming Transcription as People Speak<\/strong><\/h3>\n
Architecture Patterns and New Voices<\/strong><\/h3>\n
Key Takeaways<\/strong><\/h3>\n
\n
\n