Uncategorized

NVIDIA Researchers released PersonaPlex-7B-v1, a full duplex speech to speech conversational model that targets natural voice interactions with precise persona control. From ASR→LLM→TTS to a single full duplex model Conventional voice assistants usually run a cascade. Automatic Speech Recognition (ASR) converts speech to text, a language model generates a text answer, and Text to Speech (TTS) converts back to audio. Each stage adds latency, and the pipeline cannot handle overlapping speech, natural interruptions, or dense backchannels. PersonaPlex replaces this stack with a single Transformer model that performs streaming speech understanding and speech generation in one network. The model operates on continuous audio encoded with a neural codec and predicts both text tokens and audio tokens autoregressively. Incoming user audio is incrementally encoded, while PersonaPlex simultaneously generates its own speech, which enables barge in, overlaps, rapid turn taking, and contextual backchannels. PersonaPlex runs in a dual stream configuration. One stream tracks user audio, the other stream tracks agent speech and text. Both streams share the same model state, so the agent can keep listening while speaking and can adjust its response when the user interrupts. This design is directly inspired by Kyutai’s Moshi full duplex framework. Hybrid prompting, voice control and role control PersonaPlex uses two prompts to define the conversational identity. The voice prompt is a sequence of audio tokens that encodes vocal characteristics, speaking style, and prosody. The text prompt describes role, background, organization information, and scenario context. Together, these prompts constrain both the linguistic content and the acoustic behavior of the agent. On top of this, a system prompt supports fields such as name, business name, agent name, and business information, with a budget up to 200 tokens. Architecture, Helium backbone and audio path The PersonaPlex model has 7B parameters and follows the Moshi network architecture. A Mimi speech encoder that combines ConvNet and Transformer layers converts waveform audio into discrete tokens. Temporal and depth Transformers process multiple channels that represent user audio, agent text, and agent audio. A Mimi speech decoder that also combines Transformer and ConvNet layers generates the output audio tokens. Audio uses a 24 kHz sample rate for both input and output. PersonaPlex is built on Moshi weights and uses Helium as the underlying language model backbone. Helium provides semantic understanding and enables generalization outside the supervised conversational scenarios. This is visible in the ‘space emergency’ example, where a prompt about a reactor core failure on a Mars mission leads to coherent technical reasoning with appropriate emotional tone, even though this situation is not part of the training distribution. Training data blend, real conversations and synthetic roles Training has 1 stage and uses a blend of real and synthetic dialogues. Real conversations come from 7,303 calls, about 1,217 hours, in the Fisher English corpus. These conversations are back annotated with prompts using GPT-OSS-120B. The prompts are written at different granularity levels, from simple persona hints like ‘You enjoy having a good conversation’ to longer descriptions that include life history, location, and preferences. This corpus provides natural backchannels, disfluencies, pauses, and emotional patterns that are difficult to obtain from TTS alone. Synthetic data covers assistant and customer service roles. NVIDIA team reports 39,322 synthetic assistant conversations, about 410 hours, and 105,410 synthetic customer service conversations, about 1,840 hours. Qwen3-32B and GPT-OSS-120B generate the transcripts, and Chatterbox TTS converts them to speech. For assistant interactions, the text prompt is fixed as ‘You are a wise and friendly teacher. Answer questions or provide advice in a clear and engaging way.’ For customer service scenarios, prompts encode organization, role type, agent name, and structured business rules such as pricing, hours, and constraints. This design lets PersonaPlex disentangle natural conversational behavior, which comes mainly from Fisher, from task adherence and role conditioning, which come mainly from synthetic scenarios. Evaluation on FullDuplexBench and ServiceDuplexBench PersonaPlex is evaluated on FullDuplexBench, a benchmark for full duplex spoken dialogue models, and on a new extension called ServiceDuplexBench for customer service scenarios. FullDuplexBench measures conversational dynamics with Takeover Rate and latency metrics for tasks such as smooth turn taking, user interruption handling, pause handling, and backchanneling. GPT-4o serves as an LLM judge for response quality in question answering categories. PersonaPlex reaches smooth turn taking TOR 0.908 with latency 0.170 seconds and user interruption TOR 0.950 with latency 0.240 seconds. Speaker similarity between voice prompts and outputs on the user interruption subset uses WavLM TDNN embeddings and reaches 0.650. PersonaPlex outperforms many other open source and closed systems on conversational dynamics, response latency, interruption latency, and task adherence in both assistant and customer service roles. https://research.nvidia.com/labs/adlr/personaplex/ Key Takeaways PersonaPlex-7B-v1 is a 7B parameter full duplex speech to speech conversational model from NVIDIA, built on the Moshi architecture with a Helium language model backbone, code under MIT and weights under the NVIDIA Open Model License. The model uses a dual stream Transformer with Mimi speech encoder and decoder at 24 kHz, it encodes continuous audio into discrete tokens and generates text and audio tokens at the same time, which enables barge in, overlaps, fast turn taking, and natural backchannels. Persona control is handled by hybrid prompting, a voice prompt made of audio tokens sets timbre and style, a text prompt and a system prompt of up to 200 tokens defines role, business context, and constraints, with ready made voice embeddings such as NATF and NATM families. Training uses a blend of 7,303 Fisher conversations, about 1,217 hours, annotated with GPT-OSS-120B, plus synthetic assistant and customer service dialogs, about 410 hours and 1,840 hours, generated with Qwen3-32B and GPT-OSS-120B and rendered with Chatterbox TTS, which separates conversational naturalness from task adherence. On FullDuplexBench and ServiceDuplexBench, PersonaPlex reaches smooth turn taking takeover rate 0.908 and user interruption takeover rate 0.950 with sub second latency and improved task adherence. Check out the Technical details, Model weights and Repo. Also, feel free to follow us on Twitter and don’t forget to join our 100k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. The post NVIDIA

Suppose you’ve built your machine learning model, run the experiments, and stared at the results wondering what went wrong.

Alfred Wahlforss was running out of options. His startup, Listen Labs, needed to hire over 100 engineers, but competing against Mark Zuckerberg’s $100 million offers seemed impossible. So he spent $5,000 — a fifth of his marketing budget — on a billboard in San Francisco displaying what looked like gibberish: five strings of random numbers. The numbers were actually AI tokens. Decoded, they led to a coding challenge: build an algorithm to act as a digital bouncer at Berghain, the Berlin nightclub famous for rejecting nearly everyone at the door. Within days, thousands attempted the puzzle. 430 cracked it. Some got hired. The winner flew to Berlin, all expenses paid. That unconventional approach has now attracted $69 million in Series B funding, led by Ribbit Capital with participation from Evantic and existing investors Sequoia Capital, Conviction, and Pear VC. The round values Listen Labs at $500 million and brings its total capital to $100 million. In nine months since launch, the company has grown annualized revenue by 15x to eight figures and conducted over one million AI-powered interviews. “When you obsess over customers, everything else follows,” Wahlforss said in an interview with VentureBeat. “Teams that use Listen bring the customer into every decision, from marketing to product, and when the customer is delighted, everyone is.” Why traditional market research is broken, and what Listen Labs is building to fix it Listen’s AI researcher finds participants, conducts in-depth interviews, and delivers actionable insights in hours, not weeks. The platform replaces the traditional choice between quantitative surveys — which provide statistical precision but miss nuance—and qualitative interviews, which deliver depth but cannot scale. Wahlforss explained the limitation of existing approaches: “Essentially surveys give you false precision because people end up answering the same question… You can’t get the outliers. People are actually not honest on surveys.” The alternative, one-on-one human interviews, “gives you a lot of depth. You can ask follow up questions. You can kind of double check if they actually know what they’re talking about. And the problem is you can’t scale that.” The platform works in four steps: users create a study with AI assistance, Listen recruits participants from its global network of 30 million people, an AI moderator conducts in-depth interviews with follow-up questions, and results are packaged into executive-ready reports including key themes, highlight reels, and slide decks. What distinguishes Listen’s approach is its use of open-ended video conversations rather than multiple-choice forms. “In a survey, you can kind of guess what you should answer, and you have four options,” Wahlforss said. “Oh, they probably want me to buy high income. Let me click on that button versus an open ended response. It just generates much more honesty.” The dirty secret of the $140 billion market research industry: rampant fraud Listen finds and qualifies the right participants in its global network of 30 million people. But building that panel required confronting what Wahlforss called “one of the most shocking things that we’ve learned when we entered this industry”—rampant fraud. “Essentially, there’s a financial transaction involved, which means there will be bad players,” he explained. “We actually had some of the largest companies, some of them have billions in revenue, send us people who claim to be kind of enterprise buyers to our platform and our system immediately detected, like, fraud, fraud, fraud, fraud, fraud.” The company built what it calls a “quality guard” that cross-references LinkedIn profiles with video responses to verify identity, checks consistency across how participants answer questions, and flags suspicious patterns. The result, according to Wahlforss: “People talk three times more. They’re much more honest when they talk about sensitive topics like politics and mental health.” Emeritus, an online education company that uses Listen, reported that approximately 20% of survey responses previously fell into the fraudulent or low-quality category. With Listen, they reduced this to almost zero. “We did not have to replace any responses because of fraud or gibberish information,” said Gabrielli Tiburi, Assistant Manager of Customer Insights at Emeritus. How Microsoft, Sweetgreen, and Chubbies are using AI interviews to build better products The speed advantage has proven central to Listen’s pitch. Traditional customer research at Microsoft could take four to six weeks to generate insights. “By the time we get to them, either the decision has been made or we lose out on the opportunity to actually influence it,” said Romani Patel, Senior Research Manager at Microsoft. With Listen, Microsoft can now get insights in days, and in many cases, within hours. The platform has already powered several high-profile initiatives. Microsoft used Listen Labs to collect global customer stories for its 50th anniversary celebration. “We wanted users to share how Copilot is empowering them to bring their best self forward,” Patel said, “and we were able to collect those user video stories within a day.” Traditionally, that kind of work would have taken six to eight weeks. Simple Modern, an Oklahoma-based drinkware company, used Listen to test a new product concept. The process took about an hour to write questions, an hour to launch the study, and 2.5 hours to receive feedback from 120 people across the country. “We went from ‘Should we even have this product?’ to ‘How should we launch it?’” said Chris Hoyle, the company’s Chief Marketing Officer. Chubbies, the shorts brand, achieved a 24x increase in youth research participation—growing from 5 to 120 participants — by using Listen to overcome the scheduling challenges of traditional focus groups with children. “There’s school, sports, dinner, and homework,” explained Lauren Neville, Director of Insights and Innovation. “I had to find a way to hear from them that fit into their schedules.” The company also discovered product issues through AI interviews that might have gone undetected otherwise. Wahlforss described how the AI “through conversations, realized there were like issues with the the kids short line, and decided to, like, interview hundreds of kids. And I understand that there were issues in the liner of the shorts and that they were, like,

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. AI coding is now everywhere. But not everyone is convinced.   Depending who you ask, AI-powered coding is either giving software developers an unprecedented productivity boost or churning out masses of poorly designed code that saps their attention and sets software projects up for serious long term-maintenance problems.  The problem is right now, it’s not easy to know which is true.  As tech giants pour billions into large language models (LLMs), coding has been touted as the technology’s killer app. Executives enamored with the potential are pushing engineers to lean into an AI-powered future. But after speaking to more than 30 developers, technology executives, analysts, and researchers, MIT Technology Review found that the picture is not as straightforward as it might seem. Read the full story.  —Edd Gent Generative coding is one of our 10 Breakthrough Technologies this year. Learn more about why that is, and check out the rest of the list!  This story was also part of our Hype Correction package. You can read the rest of the stories here. The biotech trends to watch for in 2026 Earlier this week, MIT Technology Review published our annual list of Ten Breakthrough Technologies.  This year’s list includes tech that’s set to transform the energy industry, artificial intelligence, space travel—and of course biotech and health. Our breakthrough biotechnologies for 2026 involve editing a baby’s genes and, separately, resurrecting genes from ancient species. We also included a controversial technology that offers parents the chance to screen their embryos for characteristics like height and intelligence. Here’s the story behind our biotech choices. —Jessica Hamzelou This story is from The Checkup, our weekly newsletter all about the latest in health and biotech. Sign upto receive it in your inbox every Thursday. MIT Technology Review Narrated: What’s next for AI in 2026 Our AI writers have made some big bets for the coming year—read our story about the five hot trends to watch, or listen to it on Spotify, Apple, or wherever you get your podcasts. The must-reads I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology. 1 Minnesota shows how governing and content creation have mergedIn another era, we’d have just called this propaganda. (NPR)+ MAGA influencers are just straight up lying about what is happening there. (Vox) + Activists are trying to identify individual ICE officers while protecting their own identities. (WP $)+ A backlash against ICE is growing in Silicon Valley. (Wired $) 2 There’s probably more child abuse material online now than ever beforeOf all Big Tech’s failures, this is surely the most appalling. (The Atlantic $)+ US investigators are using AI to detect child abuse images made by AI. (MIT Technology Review)+ Grok is still being used to undress images of real people. (Quartz) 3 ChatGPT wrote a suicide lullaby for a man who later killed himselfThis shows it’s “still an unsafe product,” a lawyer representing a family in a tragically similar case said. (Ars Technica)+ An AI chatbot told a user how to kill himself—but the company doesn’t want to “censor” it. (MIT Technology Review) 4 Videos emerging from Iran show how bloody the crackdown has becomeIranians are finding ways around the internet blackout to show the rest of the world how many of them have been killed. (NBC) + Here’s how they’re getting around the blackout. (NPR)  5 China dominates the global humanoid robot market A new report by analysts found its companies account for over 80% of all deployments. (South China Morning Post) + Just how useful are the latest humanoids, though? (Nature)+ Why humanoid robots need their own safety rules. (MIT Technology Review) 6 How is Australia’s social media ban for kids going? It’s mixed—some teens welcome it, but others are finding workarounds. (CNBC) 7 Scientists are finding more objective ways to spot mental illness Biomarkers like voice cadence and heart rate proving pretty reliable for diagnosing conditions like depression. (New Scientist $) 8 The Pebble smartwatch be making a comebackThis could be the thing that tempts me back into buying wearables… (Gizmodo) 9 A new video game traps you in an online scam centerCan’t see the appeal myself, but… each to their own I guess? (NYT $) 10 Smoke detectors are poised to get a high-tech upgrade And one of the technologies boosting their capabilities is, of course, AI. (BBC) Quote of the day “I am very annoyed. I’m very disappointed. I’m seriously frustrated.”  —Pfizer CEO Albert Bourla tells attendees at a healthcare conference this week his feelings about the anti-vaccine agenda Health Secretary Robert F. Kennedy Jr. has been implementing, Bloomberg reports. One more thing ARIEL DAVIS How close are we to genuine “mind reading?” Technically speaking, neuroscientists have been able to read your mind for decades. It’s not easy, mind you. First, you must lie motionless within a fMRI scanner, perhaps for hours, while you watch films or listen to audiobooks.  If you do elect to endure claustrophobic hours in the scanner, the software will learn to generate a bespoke reconstruction of what you were seeing or listening to, just by analyzing how blood moves through your brain. More recently, researchers have deployed generative AI tools, like Stable Diffusion and GPT, to create far more realistic, if not entirely accurate, reconstructions of films and podcasts based on neural activity. So how close are we to genuine “mind reading?” Read the full story. —Grace Huckins We can still have nice things A place for comfort, fun and distraction to brighten up your day. (Got any ideas? Drop me a line or skeet ’em at me.) + Still keen to do a bit of reflecting on the year behind and the one ahead? This free guide might help!+ Turns out British comedian Rik Mayall had some pretty solid life advice.+ I want to stay in this house in São Paolo.  + If you want to stop doomscrolling, it’s worth looking at your sleep habits. ($)

Black Forest Labs releases FLUX.2 [klein], a compact image model family that targets interactive visual intelligence on consumer hardware. FLUX.2 [klein] extends the FLUX.2 line with sub second generation and editing, a unified architecture for text to image and image to image, and deployment options that range from local GPUs to cloud APIs, while keeping state of the art image quality. From FLUX.2 [dev] to interactive visual intelligence FLUX.2 [dev] is a 32 billion parameter rectified flow transformer for text conditioned image generation and editing, including composition with multiple reference images, and runs mainly on data center class accelerators. It is tuned for maximum quality and flexibility, with long sampling schedules and high VRAM requirements. FLUX.2 [klein] takes the same design direction and compresses it into smaller rectified flow transformers with 4 billion and 9 billion parameters. These models are distilled to very short sampling schedules, support the same text to image and multi reference editing tasks, and are optimized for response times below 1 second on modern GPUs. Model family and capabilities The FLUX.2 [klein] family consists of 4 main open weight variants through a single architecture. FLUX.2 [klein] 4B FLUX.2 [klein] 9B FLUX.2 [klein] 4B Base FLUX.2 [klein] 9B Base FLUX.2 [klein] 4B and 9B are step distilled and guidance distilled models. They use 4 inference steps and are positioned as the fastest options for production and interactive workloads. FLUX.2 [klein] 9B combines a 9B flow model with an 8B Qwen3 text embedder and is described as the flagship small model on the Pareto frontier for quality versus latency across text to image, single reference editing, and multi reference generation. The Base variants are undistilled versions with longer sampling schedules. The documentation lists them as foundation models that preserve the complete training signal and provide higher output diversity. They are intended for fine tuning, LoRA training, research pipelines, and custom post training workflows where control is more important than minimum latency. All FLUX.2 [klein] models support three core tasks in the same architecture. They can generate images from text, they can edit a single input image, and they can perform multi reference generation and editing where several input images and a prompt jointly define the target output. Latency, VRAM, and quantized variants The FLUX.2 [klein] model page provides approximate end to end inference times on GB200 and RTX 5090. FLUX.2 [klein] 4B is the fastest variant and is listed at about 0.3 to 1.2 seconds per image, depending on hardware. FLUX.2 [klein] 9B targets about 0.5 to 2 seconds at higher quality. The Base models require several seconds because they run with 50 step sampling schedules, but they expose more flexibility for custom pipelines. The FLUX.2 [klein] 4B model card states that 4B fits in about 13 GB of VRAM and is suitable for GPUs like the RTX 3090 and RTX 4070. The FLUX.2 [klein] 9B card reports a requirement of about 29 GB of VRAM and targets hardware such as the RTX 4090. This means a single high end consumer card can host the distilled variants with full resolution sampling. To extend the reach to more devices, Black Forest Labs also releases FP8 and NVFP4 versions for all FLUX.2 [klein] variants, developed together with NVIDIA. FP8 quantization is described as up to 1.6 times faster with up to 40 percent lower VRAM usage, and NVFP4 as up to 2.7 times faster with up to 55 percent lower VRAM usage on RTX GPUs, while keeping the core capabilities the same. Benchmarks against other image models Black Forest Labs evaluates FLUX.2 [klein] through Elo style comparisons on text to image, single reference editing, and multi reference tasks. The performance charts show FLUX.2 [klein] on the Pareto frontier of Elo score versus latency and Elo score versus VRAM.The commentary states that FLUX.2 [klein] matches or exceeds the quality of Qwen based image models at a fraction of the latency and VRAM, and that it outperforms Z Image while supporting unified text to image and multi reference editing in one architecture. https://bfl.ai/blog/flux2-klein-towards-interactive-visual-intelligence The base variants trade some speed for full customizability and fine tuning, which aligns with their role as foundation checkpoints for new research and domain specific pipelines. Key Takeaways FLUX.2 [klein] is a compact rectified flow transformer family with 4B and 9B variants that supports text to image, single image editing, and multi reference generation in one unified architecture. The distilled FLUX.2 [klein] 4B and 9B models use 4 sampling steps and are optimized for sub second inference on a single modern GPU, while the undistilled Base models use longer schedules and are intended for fine tuning and research. Quantized FP8 and NVFP4 variants, built with NVIDIA, provide up to 1.6 times speedup with about 40 percent VRAM reduction for FP8 and up to 2.7 times speedup with about 55 percent VRAM reduction for NVFP4 on RTX GPUs. Check out the Technical details, Repo and Model weights. Also, feel free to follow us on Twitter and don’t forget to join our 100k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. The post Black Forest Labs Releases FLUX.2 [klein]: Compact Flow Models for Interactive Visual Intelligence appeared first on MarkTechPost.