Committee

Consider what passes for success in technology policy. An AI company promises that its chatbot will not encourage suicide. A social media platform makes it more difficult for children to buy fentanyl. These are important protections, and they should have existed all along. But companies can clear these low bars while continuing to make products that leave young people dependent on simulated friendships or trapped in feeds built to keep them scrolling. We have historically allowed the debate over technology to be defined by the worst thing a product might do, with remarkably little discussion of the insidious damage it does every day. As we transition into an era defined by powerful AI products, we should demand that the tech industry do more than merely address catastrophic harms. We should take measures to ensure that its products don’t erode how we think, learn, and interact with one another.  A teenager who develops an unhealthy attachment to an AI companion may never appear in a lawsuit or a newspaper, except in the most egregious cases. Neither will the student who begins relying on a chatbot to write every paper, or the young person who measures their life against a social media feed engineered to provoke envy and keep them watching. A product can meet every narrow safety requirement and still leave our entire culture worse off in ways that don’t always make headlines. A narrow definition of success defined around safety is politically convenient for the technology industry, and companies actively promote such definitions. Taking steps to prevent spectacular disasters is good for business: It can mean fewer lawsuits and less pressure for serious regulation. Confronting the everyday consequences of dependence would require something considerably more threatening to their bottom line. It might mean asking companies to add more friction to their products or actively limit engagement or features that lead to negative outcomes, such as addictive attention-hacking algorithms. Technology giants have instead worked to shape legislation around the harms they are already under direct pressure to address. This year, for example, Google backed bills to regulate AI chatbots in several states. The legislation mandated basic protections against sexual content and promotion of self-harm that tech companies should have established regardless, but with broad carve-outs that could exempt some of the most popular services. Passing these bills allowed legislators to claim a safety win while efforts to promote stronger legislation—laws designed to prevent children from becoming emotionally dependent on AI—failed. This playbook worked in states like Nebraska, producing legislation heavily influenced by industry at the expense of bills that would have done more to hold companies accountable. We have been through a version of this with social media. Globally, people who use the internet now spend an average of more than two hours a day on social media platforms—time that has been quietly taken from ordinary life and handed over to companies whose profits depend on keeping us there. No one, outside perhaps a few tech CEOs, wanted these algorithms to occupy such a large portion of our waking hours, and their cultural engineering can actually contribute to violence on a devastating scale. Meta’s platforms have been cited for contributing to the Rohingya genocide in Myanmar, anti-Muslim riots in Sri Lanka, and alleged calls for violence during Ethiopia’s Tigray war. The same incentives operate in the feeds that shape young Americans’ understanding of politics, their peers, and the wider world. AI is extending this logic into parts of life that social media could influence only from a distance. It offers to write our thoughts for us and stand in for the people we might otherwise turn to. A third of teens now choose to discuss serious matters with AI companions instead of other people. MIT researchers found that young people using ChatGPT to write essays showed weaker brain connectivity during the task, and their evidence indicated that “cognitive debt” persisted even after the task was complete. There is plenty we still need to learn about the long-term effects. We already have reason to ask whether technology marketed as making us more capable is eroding the habits and relationships that help us develop capabilities in the first place. Follow this way of thinking to its logical endpoint and you get a world of perfectly sanitized entertainment, with chatbots ready to spare us the difficulty of thinking or of dealing with another person’s feelings (or even our own). But anyone who has struggled to write something worth reading, or repair a damaged friendship, knows how much gets lost in that bargain. We build judgment through effort and friction, and we learn to live with other people by confronting the fact that they have needs and opinions of their own.  We should demand safety as a minimum, yes—and then demand considerably more. Technology companies and school districts should have to show beyond a reasonable doubt that AI improves learning and critical thinking before making it a fixture in classrooms. Companies should be prohibited from designing companions that pretend to be conscious or profess love to users, and they should face independent scrutiny of how their systems affect relationships and people’s ability to think for themselves. The companies that have inserted their products into nearly every part of modern life have earned obligations far greater than avoiding the next terrible headline. Sam Hiner is the cofounder and executive director of the Young People’s Alliance, where he coordinated a coalition of advocates to pass laws regulating AI companions in eight states in 2026—laws such as California’s SB1119. Sam graduated from the University of North Carolina atChapel Hill in 2025 and was named to the 2026 Forbes 30 Under 30 Social Impact list. Emma Leiken is executive director of the Responsible Technology Youth Power Fund at Hopelab, which has raised more than $9 million for youth-led tech accountability work, and teaches technology policy at the Columbia School of International and Public Affairs. She was previously TikTok’s global head of youth safety and well-being policy and worked

arXiv:2610.32144v1 Announce Type: new Abstract: When a language model is asked whether its own answer is correct, three languages are involved: that of the question, that requested for the answer, and that of the verifier instruction asking for the judgment. Multilingual confidence evaluations seldom separate the three. We vary them independently in a six-language factorial design with ten open-weight models (3B-14B) on LSQA, MMMLU and TriviaQA, holding each answer fixed across verifier languages and measuring confidence with a first-token P(True) probe. The three languages move different parts of the evaluation: which of them governs accuracy depends on the benchmark, matching the response language to the question or verifier language sharpens discrimination but not calibration, and the verifier language alone shifts calibration error by up to 0.04 without changing accuracy. The scorer matters as well: an unvalidated judge would have shrunk the MMMLU English-Hindi accuracy gap from 0.198 to 0.038. Multilingual confidence results, model rankings included, are therefore comparable only when all three languages are reported and the scorer is validated.

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here. After weeks of frenzied speculation, Anthropic has filed its IPO prospectus. The 335-page document (100 pages or so are focused on risks) lays out the company’s unique view of the technology it is building and the ways it believes it will change the world.  But Anthropic has been honing its storytelling skills for years. In fact, its public messaging is a master class in narrative capture. Earlier this month, Jack Clark, the company’s cofounder and head of policy, gave a speech at the IMF meeting in Bangkok in which he announced to the assembled economists, bankers, and lawmakers that the singularity is already here. Introduced by the science fiction writer and futurist Vernor Vinge around 40 years ago (and long debated on the nerdier fringes of the internet), the singularity is supposed to be the point at which technology gets so good that it outstrips the abilities of its makers. Humans are left in the dust, the idea goes, no longer able to compete with—or even understand—the fruits of their former creation. And this fictional world that many of us have imagined since we were kids? Apparently we’re already living in it: “I believe that this year we have gained convincing evidence that it has now begun,” Clark told his audience. The company tells a vivid (and in many ways scary) story about a future it portrays as inevitable but one that has not yet happened, often co-opted from well-worn science fiction memes. It presents its technology as mysterious and dangerous, and insists that it is the only company trustworthy enough to tame it. (What investor wouldn’t want in on that?) Take another example. Last year, Chris Olah, another Anthropic cofounder, was invited to address the Vatican, where he told Pope Leo XIV and other religious scholars and officials what his company was learning about its technology. “I will be honest: We keep finding things that are mysterious, even unsettling,” he said. “We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease.” Never mind that many researchers would scoff at such an overt suggestion that Anthropic’s software has humanlike inner experience. When so little seems certain, what Anthropic (and its fellow frontier AI companies) say is snatched up and clung to like pieces of a sinking ship. I’m not saying they’re necessarily wrong about any of this. (That’s for another day.) But in a world where nobody knows what’s coming, companies like Anthropic get to fill the vacuum.  It’s worth taking a moment to ask: What is Anthropic? It might sound like a starter question on a family quiz show, but it’s far harder to answer than you might think. The question has been on my mind since reading an exchange on X last week kicked off by Nathan Calvin, who is chief strategy officer at Encode AI, an organization based in Washington, DC, that advocates for regulation. Calvin was unhappy about the legislation that Anthropic proposed in a draft bill it circulated in secret to US lawmakers (which was then leaked).   Anthropic wants to be seen as the good guy, Calvin said. But why should we trust it? “What is Anthropic?” he asked. Here’s one answer: Anthropic is a company whose IPO values it at more than $1.5 trillion, netting its cofounders, including Clark and Olah, vast wealth.  But Anthropic has always insisted it is more than that. As Calvin noted: Anthropic might instead be considered an AI safety organization focused on developing powerful technology that is reliable and (almost) risk-free. Or it might be considered a group of people hell-bent on creating a technology with superhuman capabilities before anyone else—maybe because they believe they’re the most responsible people to do so, or maybe because of their egos. In fact, Anthropic is all of these things. As CEO Dario Amodei put it in a reply to Calvin’s post: “Our strategy has always been what I’ve called the ‘race to the top’—by showing how it’s possible to build the technology responsibly, we can raise the bar for the entire industry. The forces that drive the race to the top are hugely more effective if the company practicing them is at the frontier; that’s one lesson I’ve learned repeatedly. And looking at that strategy as a whole, I’m proud of our track record and stand by it.” Amodei makes it sound as if he’s got it all tied up, but playing multiple roles is hard. There are huge tensions between the priorities of an AI safety organization and a profit-driven megacorp, for example. (Amodei admits in his post that he has made mistakes.) Ultimately, the stories Anthropic tells itself should help us understand the stories it tells us. They should help us figure out how much to trust it to self-regulate or whether we should be forcing it to give external safety testers access to its models on our terms rather than theirs. They should help us give it the benefit of the doubt (or not) when what looks like a self-serving piece of legislation comes to light. But I think the stories reveal a multiheaded hydra. Anthropic is many things to many people, and I’m not sure that makes it more trustworthy. In many ways, a good old-fashioned technology company—without the science fiction role-play, without the woo-woo—would be far easier to read.

arXiv:2610.30485v1 Announce Type: cross Abstract: Personal intelligence systems are taking on increasingly complex everyday tasks, yet repeated questions, unnecessary confirmations, and unwanted outreach can turn capable assistance into interaction burden. Reducing this burden requires agents to remember how users want to be assisted. However, existing memory benchmarks focus on factual recall and personalized answers and rarely test how agents learn and act on interaction preferences. We introduce MemPABench, an interactive environment for on-policy evaluation of memory in personal agents. MemPABench poses long-horizon tasks requiring agents to decide when to clarify, confirm, or act autonomously. Four grounded simulated users hold 14 preferences across work and personal contexts. Their feedback spans 99 sessions over four months, with five event-driven changes per user testing memory updates. We evaluate preference selection, delivered behavior, and sensitivity to context and preference change. Evaluating four memory systems across four LLM backbones, we find that current memory brings limited gains in preference selection. Retrieval-based systems (i.e. Mem0, MemOS, simple RAG) do not outperform No Memory, while File-based memory improves accuracy but applies preferences beyond relevant contexts. Our analysis shows that current memory struggles to retrieve interaction preferences and adapt to preference changes. Even with the correct preference available, overriding the agent’s default interaction habits remains challenging. The environment and dataset are available at https://mempabench.com.

arXiv:2608.28886v2 Announce Type: replace Abstract: What does a generation loop gain from learning on its own verified successes? In cycles of generate, verify, select and LoRA-consolidate on online bin packing, training on value-filtered candidates shifts what the model writes on held-out variants toward value (-1.7 points of excess, p=0.008; -3.1 against a random-consolidation control, p=0.004) while the best observed candidate converges to the classic heuristic’s level and no further. A confirmation battery replicates the whole procedure three times, with fresh seeds and a never-consulted held-out set read exactly once: the mean was nearly identical in all three lineages (-2.0, -1.8, -1.9), and after aggregating within held-out variant all seven evaluable variants favored consolidation (p=0.008). The best observed candidate moved to the classic heuristic’s level, exactly (0.021028, the lowest variant’s classic level, in all three lineages and both arms); attract never went beyond its variant’s classic level, while the random control did in 7 of 24 lineage-variant cells. A matched SFT-only control shows the supervised anchor, not repulsion from bad candidates, does the concentrating (96% of candidates land exactly at the classic heuristic’s level). On far families the tails cut both ways: consolidation lowers the per-candidate rate of better-than-classic candidates (10% to 3.9%) while its larger production yields more such candidates absolutely (5 against 1, on few events). As motivation we report the inference-time ledger that led here: a model-written schematic recap buys judged document integration and nothing buys development; a verifier written into the stream is imitated, 16.4 fabricated verdict lines per notebook. In this arrangement (an 8-bit model in MLX, five cycles), mean quality among valid candidates can be bought and replicated; under attraction the observed best goes to the classic and, so far, never beyond it.

Earlier this week, I attended a screening of You Can See Everything, the buzzy new documentary about the disgraced former biotech CEO Elizabeth Holmes.  I went in not really knowing what to expect. I’d avoided spoilers (and won’t be giving anything away here, either!). I was wondering what new information this film could reveal, given how familiar many of us are with the Theranos story thanks to years of media coverage, the best-selling book Bad Blood, and the Hulu series The Dropout, starring Amanda Seyfried. Like many others who have seen the film, I came away feeling unsettled. But not just about Holmes’s fraud and the somewhat disturbing interactions between the subjects. The film reignited a concern I’ve had about covering science for most of my career: Despite our best efforts, we never can actually see everything. In case you’re rusty on the details, Elizabeth Holmes dropped out of Stanford University in 2003 to run Theranos, a diagnostics company she founded, when she was just 19. She went on to claim that her company had invented a way to run over 100 diagnostic tests on just a few drops of blood. Holmes raised hundreds of millions of dollars for Theranos, which was valued at $9 billion in 2014. Forbes named her the youngest self-made female billionaire—she was worth $4.5 billion when she was just 30 years old. Things started to unravel in 2015, when an investigation by the Wall Street Journal raised concerns by former Theranos employees that the tests didn’t work. Some also said that the company’s device had not been tested properly, and that blood samples had been analyzed on machines from other companies instead. MIT Technology Review named it one of the biggest technology failures of 2015. Forbes had revised Holmes’s worth to $0 by 2016. That year, a damning government report said that the company’s lab was struggling with basic procedures and gave patients results even when they knew they could be wrong. In 2018, Theranos, Holmes, and former company president Ramesh Balwani were charged with “massive fraud.” In 2022, she was sentenced to a little over 11 years in prison for defrauding investors. She began serving her sentence in May 2023. It was around a month before this that filmmakers Nathan Fielder, Lance Oppenheim, and their colleagues began documenting Holmes and her partner, Billy Evans, at home with their two young children. I’m not giving anything away when I say it’s an extremely intimate portrayal—at least of how the two adults interact with Fielder, who is also a central character in the film. As you might know if you’ve seen the trailer, there are plenty of super close-ups. I felt as if I spent most of the film staring straight into the faces of those people. And almost the whole way through, they all seemed to be trying to figure each other out. Holmes is known for her dishonesty, but she invited cameras into her home in an apparent bid to reframe the narrative and position herself as an honest person with an unusual mind. Will anyone believe her? I couldn’t help feeling some empathy for Holmes, a mother we see preparing to leave her two kids for a lengthy prison sentence. And even though I have long known the facts, Holmes is clearly an effective manipulator. You can see her plant the seeds of doubt in Fielder’s mind. At times, I felt she was convincing me, sitting in my cinema seat. Maybe she really didn’t mean to lie? But then I remembered how other journalists felt after being taken in by her claims. “I blew it,” Fortune’s Roger Parloff reflected in 2015, a year after profiling Holmes and her company. “I’m pretty sure she lied to me,” STAT’s Matthew Herper wrote recently, having interviewed Holmes in 2015. Even the former employee turned whistleblower Tyler Shultz, who knew the product didn’t work, said that he could “have a five-minute conversation with her and walk away feeling like my work was changing the world,” he said in 2022. I didn’t cover Theranos in the 2010s. If I’d interviewed Holmes, would I have fallen for it all? One of the early red flags in the Theranos case was the lack of published, peer-reviewed data. Much of the company’s work was classified as a “trade secret.” Scientists tend to publish their work in journals—a slow process that involves having the work evaluated by often anonymous experts in the field, who provide feedback, highlight questionable conclusions, and often ask for clarification. Peer review is meant to add rigor to the publication process. You should be able to trust published scientific papers. Except that sometimes you can’t. Scientists aren’t perfect. Some will exaggerate their results or inflate their importance. Some manipulate data to serve a hypothesis. Sometimes these infractions are caught during peer review, but not always. There are plenty of examples of scientific findings that could never be replicated by other labs and ended up collapsing under the weight of real evidence. Many papers ultimately end up with corrections. Some are withdrawn—10,000 research papers were retracted in 2023. At any rate, there’s been a shift to sharing research in other ways. Today, many scientists opt to publish their work at preprint servers like arXiv, medRxiv, and bioRxiv before they submit them to established journals. Those outlets bypass peer review. So do company press releases, social media posts, and many conference presentations. And I get a lot of my stories from conversations and interviews. I’ll admit I’ve looked at a scientific finding and thought: A scientist has just said they’ve done that. How can I be sure they really have? I like to think that I hold myself to high journalistic standards here, as does everyone at MIT Technology Review. I would never publish a story about some new scientific development without doing my own form of peer review, and asking scientists—ideally people I know and trust—what they think of a new claim.  But the film is a stark