Committee

Since the Chinese biophysicist He Jiankui was released from prison in 2022, he has sought to make a scientific comeback and to repair his reputation after a three-year incarceration for illegally creating the world’s first gene-edited children.  While he has bounced between cities, jobs, and meetings with investors, one area of visible success on his comeback trail has been his X.com account, @Jiankui_He, which has become his main way of spreading his ideas to the world. Starting in September 2022, when he joined the platform, the account stuck to the scientist’s main themes, including promising a more careful approach to his dream of creating more gene-edited children. “I will do it, only after society has accepted it,” he posted in August 2024. He also shared mundane images of his daily life, including golf games and his family. But over time, it evolved and started to go viral. First came a series of selfies accompanied by grandiose statements (“Every pioneer or prophet must suffer”). Then, in April of this year, it became particularly outrageous and even troll-like, blasting out bizarre messages (“Good morning bitches. How many embryos have you gene edited today?”). This has left observers unsure what to take seriously. Last month, in reply to MIT Technology Review’s questions about who was responsible for the account’s transformation into a font of clever memes, He emailed us back: “It’s thanks to Cathy Tie.” You may not be familiar with Tie, but she’s no stranger to the public spotlight. A former Thiel fellow, she is a partner in the attention-grabbing Los Angeles Project, which promised to create glow-in-the-dark pets. Over the past several weeks, though, the 29-year-old Canadian entrepreneur has started to get more and more attention as the new wife to (and apparent social media mastermind behind) He Jiankui. On April 15, He announced a new venture, Cathy Medicine, that would take up his mission of editing human embryos to create people resistant to diseases like Alzheimer’s or cancer. Just a few days later, on April 18, He and Tie announced that they had married, posting pictures of themselves in traditional Chinese wedding attire. But now Tie says that just a month after she married “the most controversial scientist in the world,” her plans to relocate from Los Angeles to Beijing to be with He are in disarray; she says she’s been denied entry to China and the two “may never see each other again,” as He’s passport is being held by Chinese authorities and he can’t leave the country. Reached by phone in Manila, Tie said authorities in the Philippines had intercepted her during a layover on May 17 and told her she couldn’t board a plane to China, where she was born and where she says she has a valid 10-year visa. She claims they didn’t say why but told her she is likely “on a watch list.” (MIT Technology Review could not independently confirm Tie’s account.)  “While I’m concerned about my marriage, I am more concerned about what this means for humanity and the future of science,” Tie posted to her own X account. A match made in gene-editing heaven The romance between He and Tie has been playing out in public over the past several weeks through a series of reveals on He’s X feed, which had already started going viral late last year thanks to his style of posting awkward selfies alongside maxims about the untapped potential of heritable gene editing, which involves changing people’s DNA when they’re just embryos in an IVF dish.  “Human [sic] will no longer be controlled by Darwin’s evolution,” He wrote in March. That post, which showed him standing in an empty lab, gazing into the distance, garnered 9.7 million views. And then, a week later, he collected 13.3 million for this one: “Ethics is holding back scientific innovation and progress.”  In April, the feed started to change even more drastically.  He’s posts became increasingly provocative, with better English and a unique sensibility reflecting online culture. “Stop asking for cat girls. I’m trying to cure disease,” the account posted on April 15. Two days later, it followed up: “I literally went to prison for this shit.”  This shift coincided with the development of his romance with Tie. Tie told us she has visited China three times this year, including a three-week stint in April when she and He got married after a whirlwind romance. She bought him a silver wedding ring made up of intertwined DNA strands.  The odd behavior on He’s X feed and the sudden marriage have left followers wondering if they are watching a love story, a new kind of business venture, or performance art. It might be all three.  A wedding photo posted by Tie on the Chinese social media platform Rednote shows the couple sitting at a table in a banquet hall, with a small number of guests. MIT Technology Review has been able to identify several people who attended: Cai Xilei, He’s criminal attorney; Liu Haiyan, an investor and former business partner of He; and Darren Zhu, an artist and Thiel fellow who is making a “speculative” documentary about the biophysicist that will blur the boundaries of fiction and reality. In the phone interview, Tie declined to say if she and He are legally married. She also confirmed she celebrated a wedding less than one year ago with someone else in California, in July of 2024, but said they broke up after a few months; she also declined to describe the legal status of that marriage. In the phone call, Tie emphasized that her relationship with He is genuine: “I wouldn’t marry him if I wasn’t in love with him.” An up-and-comer Years before Tie got into a relationship with He, she was getting plenty of attention in her own right. She became a Thiel fellow in 2015, when she was just 18. That program, started by the billionaire Peter Thiel, gave her a grant of $100,000 to drop out of the University of

Google’s “sufficient context” helps refine RAG systems, reduce LLM hallucinations, and boost AI reliability for business applications.Read More

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. Meet Cathy Tie, Bride of “China’s Frankenstein” Since the Chinese biophysicist He Jiankui was released from prison in 2022, he has sought to make a scientific comeback and to repair his reputation after a three-year incarceration for illegally creating the world’s first gene-edited children. One area of visible success on his come-back trail has been his X.com account. Over the past few years, his account has evolved from sharing mundane images of his daily life to spreading outrageous, antagonistic messages. This has left observers unsure what to take seriously. Last month, in reply to MIT Technology Review’s questions about who was responsible for the account’s transformation into a font of clever memes, He emailed us back: “It’s thanks to Cathy Tie.” Tie is no stranger to the public spotlight. A former Thiel fellow, she is a partner in a project which promised to create glow-in-the-dark pets. Over the past several weeks, though, the Canadian entrepreneur has started to get more and more attention as the new wife to He Jiankui. Read the full story. —Caiwei Chen & Antonio Regalado Anthropic’s new hybrid AI model can work on tasks autonomously for hours at a time Anthropic has announced two new AI models that it claims represent a major step toward making AI agents truly useful. AI agents trained on Claude Opus 4, the company’s most powerful model to date, raise the bar for what such systems are capable of by tackling difficult tasks over extended periods of time and responding more usefully to user instructions, the company says. They’ve achieved some impressive results: Opus 4 created a guide for the video game Pokémon Red while playing it for more than 24 hours straight. The company’s previously most powerful model was capable of playing for just 45 minutes. Read the full story. —Rhiannon Williams The FDA plans to limit access to covid vaccines. Here’s why that’s not all bad. This week, two new leaders at the US Food and Drug Administration announced plans to limit access to covid vaccines, arguing that there is not much evidence to support the value of annual shots in healthy people. New vaccines will be made available only to the people who are most vulnerable—namely, those over 65 and others with conditions that make them more susceptible to severe disease. The plans have been met with fear and anger in some quarters. But they weren’t all that shocking to me. In the UK, where I live, covid boosters have been offered only to vulnerable groups for a while now. And the immunologists I spoke to agree: The plans make sense. Read the full story. —Jessica Hamzelou This article first appeared in The Checkup, MIT Technology Review’s weekly biotech newsletter. To receive it in your inbox every Thursday, and read articles like this first, sign up here. The must-reads I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology. 1 Thousands of Americans are facing extreme weatherBut help from the federal government may never arrive. (Slate $)+ States struck by tornadoes and floods are begging the Trump administration for aid. (Scientific American $) 2 Spain’s grid operator has accused power plants of not doing their jobIt claims they failed to control the system’s voltage shortly before the blackout. (FT $)+ Did solar power cause Spain’s blackout? (MIT Technology Review) 3 Google is facing a DoJ probe over its AI chatbot dealIt will probe whether Google’s deal with Character.AI gives it an unfair advantage. (Bloomberg $)+ It may not lead to enforcement action, though. (Reuters) 4 DOGE isn’t bad news for everyoneThese smaller US government IT contractors say it’s good for business—for now. (WSJ $)+ It appears that DOGE used a Meta AI model to review staff emails, not Grok. (Wired $)+ Can AI help DOGE slash government budgets? It’s complex. (MIT Technology Review)5 Google’s new shopping tool adds breasts to minorsTry it On distorts uploaded photos to clothing models’ proportions, even when they’re children. (The Atlantic $)+ It feels like this could have easily been avoided. (Axios)+ An AI companion site is hosting sexually charged conversations with underage celebrity bots. (MIT Technology Review) 6 Apple is reportedly planning a smart glasses product launchBy the end of next year. (Bloomberg $)+ It’s playing catchup with Meta and Google, among others. (Engadget)+ What’s next for smart glasses. (MIT Technology Review) 7 What it’s like to live in Elon Musk’s corner of TexasComplete with an ugly bust and furious locals. (The Guardian)+ West Lake Hills residents are pushing back against his giant fences. (Architectural Digest $) 8 Our solar system may contain a hidden ninth planetA possible dwarf planet has been spotted orbiting beyond Neptune. (New Scientist $) 9 Wikipedia does swag nowHow else will you let everyone know you love the open web? (Fast Company $) 10 One of the last good apps is shutting downMozilla is closing Pocket, its article-saving app, and the internet is worse for it. (404 Media)+ Parent company Mozilla said the way people use the web has changed. (The Verge) Quote of the day “This is like the Mount Everest of corruption.” —Senator Jeff Merkley protests outside Donald Trump’s exclusive dinner for the highest-paying customers of his personal cryptocurrency, the New York Times reports. One more thing The iPad was meant to revolutionize accessibility. What happened? On April 3, 2010, Steve Jobs debuted the iPad. What for most people was basically a more convenient form factor was something far more consequential for non-speakers: a life-­changing revolution in access to a portable, powerful communication device for just a few hundred dollars. But a piece of hardware, however impressively designed and engineered, is only as valuable as what a person can do with it. After the iPad’s release, the flood of new, easy-to-use augmentative and alternative communication apps that users were in desperate need of never came. Today, there are only

If you’ve been into machine learning for a while, you’ve probably noticed that the same books get recommended over and over again.

Enchant is launching a new zero-equity accelerator for gaming and AI startups, with applications now open for its three-month program.Read More

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. The data center boom in the desert In the high desert east of Reno, Nevada, construction crews are flattening the golden foothills of the Virginia Range, laying the foundations of a data center city. Google, Tract, Switch, EdgeCore, Novva, Vantage, and PowerHouse are all operating, building, or expanding huge facilities nearby. Meanwhile, Microsoft has acquired more than 225 acres of undeveloped property, and Apple is expanding its existing data center just across the Truckee River from the industrial park. The corporate race to amass computing resources to train and run artificial intelligence models and store information in the cloud has sparked a data center boom in the desert—and it’s just far enough away from Nevada’s communities to elude wide notice and, some fear, adequate scrutiny. Read the full story. —James Temple This story is part of Power Hungry: AI and our energy future—our new series shining a light on the energy demands and carbon costs of the artificial intelligence revolution. Check out the rest of the package here. A new atomic clock in space could help us measure elevations on Earth In 2003, engineers from Germany and Switzerland began building a bridge across the Rhine River simultaneously from both sides. Months into construction, they found that the two sides did not meet. The German side hovered 54 centimeters above the Swiss one. The misalignment happened because they measured elevation from sea level differently. To prevent such costly construction errors, in 2015 scientists in the International Association of Geodesy voted to adopt the International Height Reference Frame, or IHRF, a worldwide standard for elevation. Now, a decade after its adoption, scientists are looking to update the standard—by using the most precise clock ever to fly in space. Read the full story. —Sophia Chen Three takeaways about AI’s energy use and climate impacts —Casey Crownhart This week, we published Power Hungry, a package all about AI and energy. At the center of this package is the most comprehensive look yet at AI’s growing power demand, if I do say so myself. This data-heavy story is the result of over six months of reporting by me and my colleague James O’Donnell (and the work of many others on our team). Over that time, with the help of leading researchers, we quantified the energy and emissions impacts of individual queries to AI models and tallied what it all adds up to, both right now and for the years ahead. There’s a lot of data to dig through, and I hope you’ll take the time to explore the whole story. But in the meantime, here are three of my biggest takeaways from working on this project. Read the full story. This article is from The Spark, MIT Technology Review’s weekly climate newsletter. To receive it in your inbox every Wednesday, sign up here. MIT Technology Review Narrated: Congress used to evaluate emerging technologies. Let’s do it again. Artificial intelligence comes with a shimmer and a sheen of magical thinking. And if we’re not careful, politicians, employers, and other decision-makers may accept at face value the idea that machines can and should replace human judgment and discretion. One way to combat that might be resurrecting the Office of Technology Assessment, a Congressional think tank that detected lies and tested tech until it was shuttered in 1995. This is our latest story to be turned into a MIT Technology Review Narrated podcast, which we’re publishing each week on Spotify and Apple Podcasts. Just navigate to MIT Technology Review Narrated on either platform, and follow us to get all our new content as it’s released. The must-reads I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology. 1 OpenAI is buying Jony Ive’s AI startupThe former Apple design guru will work with Sam Altman to design an entirely new range of devices. (NYT $)+ The deal is worth a whopping $6.5 billion. (Bloomberg $)+ Altman gave OpenAI staff a preview of its AI ‘companion’ devices. (WSJ $)+ AI products to date have failed to set the world alight. (The Atlantic $) 2 Microsoft has blocked employee emails containing ‘Gaza’ or ‘Palestine’Although the term ‘Israel’ does not trigger such a block. (The Verge)+ Protest group No Azure for Apartheid has accused the company of censorship. (Fortune $) 3 DOGE needs to do its work in secretThat’s what the Trump administration is claiming to the Supreme Court, at least. (Ars Technica)+ It’s trying to avoid being forced to hand over internal documents. (NYT $)+ DOGE’s tech takeover threatens the safety and stability of our critical data. (MIT Technology Review) 4 US banks are racing to embrace cryptocurrencyAhead of new stablecoin legislation. (The Information $)+ Attendees at Trump’s crypto dinner paid over $1 million for the privilege. (NBC News)+ Bitcoin has surged to an all-time peak yet again. (Reuters) 5 China is making huge technological leapsThanks to the billions it’s poured into narrowing the gap between it and the US. (WSJ $)+ Nvidia’s CEO has branded America’s chip curbs on China ‘a failure.’ (FT $)+ There can be no winners in a US-China AI arms race. (MIT Technology Review) 6 Disordered eating content is rife on TikTokBut a pocket of creators are dedicated to debunking the worst of it. (Wired $) 7 The US military is interested in the world’s largest aircraftThe gigantic WindRunner plane will have an 80-metre wingspan. (New Scientist $)+ Phase two of military AI has arrived. (MIT Technology Review) 8 How AI is shaking up animationNew tools are slashing the costs of creating episodes by up to 90%. (NYT $)+ Generative AI is reshaping South Korea’s webcomics industry. (MIT Technology Review) 9 Tesla’s Cybertruck is a flopSorry, Elon. (Fast Company $)+ The vehicles’ resale value is plummeting. (The Daily Beast) 10 Google’s new AI video generator loves this terrible jokeWhich appears to originate from a Reddit post. (404 Media)+ What happened

Someone also appears to have published a full scrape of the Time article online on the news aggregator app Newsbreak.Read More

Researchers are reimagining how models operate as demand skyrockets for faster, smarter, and more private AI on phones, tablets, and laptops. The next generation of AI isn’t just lighter and faster; it’s local. By embedding intelligence directly into devices, developers are unlocking near-instant responsiveness, slashing memory demands, and putting privacy back into users’ hands. With mobile hardware rapidly advancing, the race is on to build compact, lightning-fast models that are intelligent enough to redefine everyday digital experiences. A major concern is delivering high-quality, multimodal intelligence within the constrained environments of mobile devices. Unlike cloud-based systems that have access to extensive computational power, on-device models must perform under strict RAM and processing limits. Multimodal AI, capable of interpreting text, images, audio, and video, typically requires large models, which most mobile devices cannot handle efficiently. Also, cloud dependency introduces latency and privacy concerns, making it essential to design models that can run locally without sacrificing performance. Earlier models like Gemma 3 and Gemma 3 QAT attempted to bridge this gap by reducing size while maintaining performance. Designed for use on cloud or desktop GPUs, they significantly improved model efficiency. However, these models still required robust hardware and could not fully overcome mobile platforms’ memory and responsiveness constraints. Despite supporting advanced functions, they often involved compromises limiting their real-time smartphone usability. Researchers from Google and Google DeepMind introduced Gemma 3n. The architecture behind Gemma 3n has been optimized for mobile-first deployment, targeting performance across Android and Chrome platforms. It also forms the underlying basis for the next version of Gemini Nano. The innovation represents a significant leap forward by supporting multimodal AI functionalities with a much lower memory footprint while maintaining real-time response capabilities. This marks the first open model built on this shared infrastructure and is made available to developers in preview, allowing immediate experimentation. The core innovation in Gemma 3n is the application of Per-Layer Embeddings (PLE), a method that drastically reduces RAM usage. While the raw model sizes include 5 billion and 8 billion parameters, they behave with memory footprints equivalent to 2 billion and 4 billion parameter models. The dynamic memory consumption is just 2GB for the 5B model and 3GB for the 8B version. Also, it uses a nested model configuration where a 4B active memory footprint model includes a 2B submodel trained through a technique known as MatFormer. This allows developers to dynamically switch performance modes without loading separate models. Further advancements include KVC sharing and activation quantization, which reduce latency and increase response speed. For example, response time on mobile improved by 1.5x compared to Gemma 3 4B while maintaining better output quality. Image Source The performance metrics achieved by Gemma 3n reinforce its suitability for mobile deployment. It excels in automatic speech recognition and translation, allowing seamless speech conversion to translated text. On multilingual benchmarks like WMT24++ (ChrF), it scores 50.1%, highlighting its strength in Japanese, German, Korean, Spanish, and French. Its mix’n’match capability allows the creation of submodels optimized for various quality and latency combinations, offering developers further customization. The architecture supports interleaved inputs from different modalities, text, audio, images, and video, allowing more natural and context-rich interactions. It also performs offline, ensuring privacy and reliability even without network connectivity. Use cases include live visual and auditory feedback, context-aware content generation, and advanced voice-based applications. Image Source Several Key Takeaways from the Research on Gemma 3n include: Built using collaboration between Google, DeepMind, Qualcomm, MediaTek, and Samsung System LSI. Designed for mobile-first deployment. Raw model size of 5B and 8B parameters, with operational footprints of 2GB and 3GB, respectively, using Per-Layer Embeddings (PLE). 1.5x faster response on mobile vs Gemma 3 4B. Multilingual benchmark score of 50.1% on WMT24++ (ChrF). Accepts and understands audio, text, image, and video, enabling complex multimodal processing and interleaved inputs. Supports dynamic trade-offs using MatFormer training with nested submodels and mix’n’match capabilities. Operates without an internet connection, ensuring privacy and reliability. Preview is available via Google AI Studio and Google AI Edge, with text and image processing capabilities. In conclusion, this innovation provides a clear pathway for making high-performance AI portable and private. By tackling RAM constraints through innovative architecture and enhancing multilingual and multimodal capabilities, researchers offer a viable solution for bringing sophisticated AI directly into everyday devices. The flexible submodel switching, offline readiness, and fast response time mark a comprehensive approach to mobile-first AI. The research addresses the balance of computational efficiency, user privacy, and dynamic responsiveness. The result is a system capable of delivering real-time AI experiences without sacrificing capability or versatility, fundamentally expanding what users can expect from on-device intelligence. Check out the Technical details and Try it here. All credit for this research goes to the researchers of this project. Also, feel free to follow us on Twitter and don’t forget to join our 95k+ ML SubReddit and Subscribe to our Newsletter. The post Google DeepMind Releases Gemma 3n: A Compact, High-Efficiency Multimodal AI Model for Real-Time On-Device Use appeared first on MarkTechPost.

Multimodal mathematical reasoning enables machines to solve problems involving textual information and visual components like diagrams and figures. This requires combining language understanding and visual interpretation to make sense of complex mathematical contexts. Such capabilities are vital in education, automated tutoring, and document analysis, where problems are often presented with a blend of text and images. A major obstacle in this area is the lack of high-quality, precise alignment between math images and their textual or symbolic representations. Most datasets used to train large multimodal models are derived from image captions in natural settings, which often miss the detailed elements essential for mathematical accuracy. This creates problems for models that rely on these data sources, making them unreliable when dealing with geometry, figures, or technical diagrams. A model’s performance in mathematical reasoning depends heavily on its ability to correctly interpret and link these visual details with mathematical expressions or instructions. In the past, some approaches tried to address this by either enhancing the visual encoders or using manually crafted datasets. However, these methods tend to produce low image diversity, relying on hand-coded or template-based generation, which limits their applicability. Some efforts, like Math-LLaVA and MAVIS, developed synthetic datasets and used templates or predefined categories. Still, they could not dynamically create a wide variety of math-related visuals. This shortfall restricts the learning scope of models and leaves them struggling with more complex or less structured mathematical problems. Researchers from the Multimedia Laboratory at The Chinese University of Hong Kong and CPII under InnoHK introduced a novel approach called MathCoder-VL. This method combines a vision-to-code model named FigCodifier and a synthetic data engine. They constructed the ImgCode-8.6M dataset using a model-in-the-loop strategy, which allowed them to build the largest image-code dataset to date iteratively. Further, they developed MM-MathInstruct-3M, a multimodal instruction dataset enriched with newly synthesized images. The MathCoder-VL model is trained in two stages: mid-training on ImgCode-8.6M to improve visual-text alignment and fine-tuning on MM-MathInstruct-3M to strengthen reasoning abilities. The FigCodifier model works by translating mathematical figures into code that can recreate those figures exactly. This code-image pairing ensures strict alignment and accuracy, unlike caption-based datasets. The process begins with 119K image-code pairs from DaTikZ and expands through iterative training using images collected from textbooks, K12 datasets, and arXiv papers. The final dataset includes 8.6 million code-image pairs and covers various mathematical topics. FigCodifier also supports Python-based rendering, which adds variety to image generation. The system filters low-quality data by checking code validity and removing redundant or unhelpful visuals, resulting in 4.3M high-quality TikZ and 4.3M Python-based pairs. Performance evaluations show that MathCoder-VL outperforms multiple open-source models. The 8B version achieved 73.6% accuracy on the MathVista Geometry Problem Solving subset, surpassing GPT-4o and Claude 3.5 Sonnet by 8.9% and 9.2%, respectively. It also scored 26.1% on MATH-Vision and 46.5% on MathVerse. In Chinese-language benchmarks, it achieved 51.2% on GAOKAO-MM. On the We-Math benchmark, it solved two-step problems at 58.6%, outperforming GPT-4o’s 58.1%. Its performance on three-step problems reached 52.1%, again exceeding GPT-4o’s 43.6%. Compared to its base model InternVL2-8B, it showed gains of 6.1% on MATH-Vision and 11.6% on MathVista. This work clearly defines the problem of insufficient visual-textual alignment in multimodal math reasoning and provides a scalable and innovative solution. The introduction of FigCodifier and synthetic datasets allows models to learn from accurate, diverse visuals paired with exact code, significantly boosting their reasoning abilities. MathCoder-VL represents a practical advancement in this field, demonstrating how thoughtful model design and high-quality data can overcome longstanding limitations in mathematical AI. Check out the Paper and GitHub Page. All credit for this research goes to the researchers of this project. Also, feel free to follow us on Twitter and don’t forget to join our 95k+ ML SubReddit and Subscribe to our Newsletter. The post This AI Paper Introduces MathCoder-VL and FigCodifier: Advancing Multimodal Mathematical Reasoning with Vision-to-Code Alignment appeared first on MarkTechPost.

Addressing Architectural Trade-offs in Language Models As language models scale, balancing expressivity, efficiency, and adaptability becomes increasingly challenging. Transformer architectures dominate due to their strong performance across a wide range of tasks, but they are computationally expensive—particularly for long-context scenarios—due to the quadratic complexity of self-attention. On the other hand, Structured State Space Models (SSMs) offer improved efficiency and linear scaling, yet often lack the nuanced sequence modeling required for complex language understanding. A combined architecture that leverages the strengths of both approaches is needed to support diverse applications across environments. Introducing Falcon-H1: A Hybrid Architecture The Falcon-H1 series, released by the Technology Innovation Institute (TII), introduces a hybrid family of language models that combine Transformer attention mechanisms with Mamba2-based SSM components. This architecture is designed to improve computational efficiency while maintaining competitive performance across tasks requiring deep contextual understanding. Falcon-H1 covers a wide parameter range—from 0.5B to 34B—catering to use cases from resource-constrained deployments to large-scale distributed inference. The design aims to address common bottlenecks in LLM deployment: memory efficiency, scalability, multilingual support, and the ability to handle extended input sequences. Source: https://falcon-lm.github.io/blog/falcon-h1/ Architectural Details and Design Objectives Falcon-H1 adopts a parallel structure where attention heads and Mamba2 SSMs operate side by side. This design allows each mechanism to independently contribute to sequence modeling: attention heads specialize in capturing token-level dependencies, while SSM components support efficient long-range information retention. The series supports a context length of up to 256K tokens, which is particularly useful for applications in document summarization, retrieval-augmented generation, and multi-turn dialogue systems. Model training incorporates a customized microparameterization (μP) recipe and optimized data pipelines, allowing for stable and efficient training across model sizes. The models are trained with a focus on multilingual capabilities. The architecture is natively equipped to handle 18 languages, with coverage including English, Chinese, Arabic, Hindi, French, and others. The framework is extensible to over 100 languages, supporting localization and region-specific model adaptation. Empirical Results and Comparative Evaluation Despite relatively modest parameter counts, Falcon-H1 models demonstrate strong empirical performance: Falcon-H1-0.5B achieves results comparable to 7B-parameter models released in 2024. Falcon-H1-1.5B-Deep performs on par with leading 7B to 10B Transformer models. Falcon-H1-34B matches or exceeds the performance of models such as Qwen3-32B, Llama4-Scout-17B/109B, and Gemma3-27B across several benchmarks. Evaluations emphasize both general-purpose language understanding and multilingual benchmarks. Notably, the models achieve strong performance across both high-resource and low-resource languages without requiring excessive fine-tuning or additional adaptation layers. Source: https://falcon-lm.github.io/blog/falcon-h1/ Deployment and inference are supported through integration with open-source tools such as Hugging Face Transformers. FlashAttention-2 compatibility further reduces memory usage during inference, offering an attractive efficiency-performance balance for enterprise use. Conclusion Falcon-H1 represents a methodical effort to refine language model architecture by integrating complementary mechanisms—attention and SSMs—within a unified framework. By doing so, it addresses key limitations in both long-context processing and scaling efficiency. The model family provides a range of options for practitioners, from lightweight variants suitable for edge deployment to high-capacity configurations for server-side applications. Through its multilingual coverage, long-context capabilities, and architectural flexibility, Falcon-H1 offers a technically sound foundation for research and production use cases that demand performance without compromising on efficiency or accessibility. Check out the Official Release, Models on Hugging Face and GitHub Page. All credit for this research goes to the researchers of this project. Also, feel free to follow us on Twitter and don’t forget to join our 95k+ ML SubReddit and Subscribe to our Newsletter. The post Technology Innovation Institute TII Releases Falcon-H1: Hybrid Transformer-SSM Language Models for Scalable, Multilingual, and Long-Context Understanding appeared first on MarkTechPost.