AI

A deep neural network can be understood as a geometric system, where each layer reshapes the input space to form increasingly complex decision boundaries. For this to work effectively, layers must preserve meaningful spatial information — particularly how far a data point lies from these boundaries — since this distance enables deeper layers to build rich, non-linear representations. Sigmoid disrupts this process by compressing all inputs into a narrow range between 0 and 1. As values move away from decision boundaries, they become indistinguishable, causing a loss of geometric context across layers. This leads to weaker representations and limits the effectiveness of depth. ReLU, on the other hand, preserves magnitude for positive inputs, allowing distance information to flow through the network. This enables deeper models to remain expressive without requiring excessive width or compute. In this article, we focus on this forward-pass behavior — analyzing how Sigmoid and ReLU differ in signal propagation and representation geometry using a two-moons experiment, and what that means for inference efficiency and scalability. Setting up the dependencies Copy CodeCopiedUse a different Browser import numpy as np import matplotlib.pyplot as plt import matplotlib.gridspec as gridspec from matplotlib.colors import ListedColormap from sklearn.datasets import make_moons from sklearn.preprocessing import StandardScaler from sklearn.model_selection import train_test_split Copy CodeCopiedUse a different Browser plt.rcParams.update({ “font.family”: “monospace”, “axes.spines.top”: False, “axes.spines.right”: False, “figure.facecolor”: “white”, “axes.facecolor”: “#f7f7f7”, “axes.grid”: True, “grid.color”: “#e0e0e0”, “grid.linewidth”: 0.6, }) T = { “bg”: “white”, “panel”: “#f7f7f7”, “sig”: “#e05c5c”, “relu”: “#3a7bd5”, “c0”: “#f4a261”, “c1”: “#2a9d8f”, “text”: “#1a1a1a”, “muted”: “#666666”, } Creating the dataset To study the effect of activation functions in a controlled setting, we first generate a synthetic dataset using scikit-learn’s make_moons. This creates a non-linear, two-class problem where simple linear boundaries fail, making it ideal for testing how well neural networks learn complex decision surfaces. We add a small amount of noise to make the task more realistic, then standardize the features using StandardScaler so both dimensions are on the same scale — ensuring stable training. The dataset is then split into training and test sets to evaluate generalization. Finally, we visualize the data distribution. This plot serves as the baseline geometry that both Sigmoid and ReLU networks will attempt to model, allowing us to later compare how each activation function transforms this space across layers. Copy CodeCopiedUse a different Browser X, y = make_moons(n_samples=400, noise=0.18, random_state=42) X = StandardScaler().fit_transform(X) X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.25, random_state=42 ) fig, ax = plt.subplots(figsize=(7, 5)) fig.patch.set_facecolor(T[“bg”]) ax.set_facecolor(T[“panel”]) ax.scatter(X[y == 0, 0], X[y == 0, 1], c=T[“c0″], s=40, edgecolors=”white”, linewidths=0.5, label=”Class 0″, alpha=0.9) ax.scatter(X[y == 1, 0], X[y == 1, 1], c=T[“c1″], s=40, edgecolors=”white”, linewidths=0.5, label=”Class 1″, alpha=0.9) ax.set_title(“make_moons — our dataset”, color=T[“text”], fontsize=13) ax.set_xlabel(“x₁”, color=T[“muted”]); ax.set_ylabel(“x₂”, color=T[“muted”]) ax.tick_params(colors=T[“muted”]); ax.legend(fontsize=10) plt.tight_layout() plt.savefig(“moons_dataset.png”, dpi=140, bbox_inches=”tight”) plt.show() Creating the Network Next, we implement a small, controlled neural network to isolate the effect of activation functions. The goal here is not to build a highly optimized model, but to create a clean experimental setup where Sigmoid and ReLU can be compared under identical conditions. We define both activation functions (Sigmoid and ReLU) along with their derivatives, and use binary cross-entropy as the loss since this is a binary classification task. The TwoLayerNet class represents a simple 3-layer feedforward network (2 hidden layers + output), where the only configurable component is the activation function. A key detail is the initialization strategy: we use He initialization for ReLU and Xavier initialization for Sigmoid, ensuring that each network starts in a fair and stable regime based on its activation dynamics. The forward pass computes activations layer by layer, while the backward pass performs standard gradient descent updates. Importantly, we also include diagnostic methods like get_hidden and get_z_trace, which allow us to inspect how signals evolve across layers — this is crucial for analyzing how much geometric information is preserved or lost. By keeping architecture, data, and training setup constant, this implementation ensures that any difference in performance or internal representations can be directly attributed to the activation function itself — setting the stage for a clear comparison of their impact on signal propagation and expressiveness. Copy CodeCopiedUse a different Browser def sigmoid(z): return 1 / (1 + np.exp(-np.clip(z, -500, 500))) def sigmoid_d(a): return a * (1 – a) def relu(z): return np.maximum(0, z) def relu_d(z): return (z > 0).astype(float) def bce(y, yhat): return -np.mean(y * np.log(yhat + 1e-9) + (1 – y) * np.log(1 – yhat + 1e-9)) class TwoLayerNet: def __init__(self, activation=”relu”, seed=0): np.random.seed(seed) self.act_name = activation self.act = relu if activation == “relu” else sigmoid self.dact = relu_d if activation == “relu” else sigmoid_d # He init for ReLU, Xavier for Sigmoid scale = lambda fan_in: np.sqrt(2 / fan_in) if activation == “relu” else np.sqrt(1 / fan_in) self.W1 = np.random.randn(2, 8) * scale(2) self.b1 = np.zeros((1, 8)) self.W2 = np.random.randn(8, 8) * scale(8) self.b2 = np.zeros((1, 8)) self.W3 = np.random.randn(8, 1) * scale(8) self.b3 = np.zeros((1, 1)) self.loss_history = [] def forward(self, X, store=False): z1 = X @ self.W1 + self.b1; a1 = self.act(z1) z2 = a1 @ self.W2 + self.b2; a2 = self.act(z2) z3 = a2 @ self.W3 + self.b3; out = sigmoid(z3) if store: self._cache = (X, z1, a1, z2, a2, z3, out) return out def backward(self, lr=0.05): X, z1, a1, z2, a2, z3, out = self._cache n = X.shape[0] dout = (out – self.y_cache) / n dW3 = a2.T @ dout; db3 = dout.sum(axis=0, keepdims=True) da2 = dout @ self.W3.T dz2 = da2 * (self.dact(z2) if self.act_name == “relu” else self.dact(a2)) dW2 = a1.T @ dz2; db2 = dz2.sum(axis=0, keepdims=True) da1 = dz2 @ self.W2.T dz1 = da1 * (self.dact(z1) if self.act_name == “relu” else self.dact(a1)) dW1 = X.T @ dz1; db1 = dz1.sum(axis=0, keepdims=True) for p, g in [(self.W3,dW3),(self.b3,db3),(self.W2,dW2), (self.b2,db2),(self.W1,dW1),(self.b1,db1)]: p -= lr * g def train_step(self, X, y, lr=0.05): self.y_cache = y.reshape(-1, 1) out = self.forward(X, store=True) loss = bce(self.y_cache, out) self.backward(lr) return loss def get_hidden(self, X, layer=1): “””Return post-activation values for layer 1 or 2.””” z1 = X @ self.W1 + self.b1; a1 = self.act(z1) if

A rare warm spell in January melted enough snow to uncover Cornell University’s newest athletic field, built for field hockey. Months before, it was a meadow teeming with birds and bugs; now it’s more than an acre of synthetic turf roughly the color of the felt on a pool table, almost digital in its saturation. The day I walked up the hill from a nearby creek to take a look, the metal fence around the field was locked, but someone had left a hallway-size piece of the new simulated grass outside the perimeter. It was bristly and tough, but springy and squeaky under my booted feet. I could imagine running around on it, but it would definitely take some getting used to. My companion on this walk seemed even less favorably disposed to the thought. Yayoi Koizumi, a local environmental advocate, has been fighting synthetic-turf projects at Cornell since 2023. A petite woman dressed that day in a faded plum coat over a teal vest, with a scarf the colors of salmon, slate, and sunflowers, Koizumi compulsively picked up plastic trash as we walked: a red Solo cup, a polyethylene Dunkin’ container, a five-foot vinyl panel. She couldn’t bear to leave this stuff behind to fragment into microplastic bits—as she believes the new field will. “They’ve covered the living ground in plastic,” she said. “It’s really maddening.”  The new pitch is one part of a $70 million plan to build more recreational space at the university. As of this spring, Cornell plans to install something like a quarter million square feet of synthetic grass—what people have colloquially called “astroturf” since the middle of the last century. University PR says it will be an important part of a “health-promoting campus” that is “supportive of holistic individual, social, and ecological well-being.” Koizumi runs an anti-plastic environmental group called Zero Waste Ithaca, which says that’s mostly nonsense. This fight is more than just the usual town-versus-gown tension. Synthetic turf used to be the stuff of professional sports arenas and maybe a suburban yard or two; today communities across the United States are debating whether to lay it down on playgrounds, parks, and dog runs. Proponents say it’s cheaper and hardier than grass, requiring less water, fertilizer, and maintenance—and that it offers a uniform surface for more hours and more days of the year than grass fields, a competitive advantage for athletes and schools hoping for a more robust athletic program. But while new generations of synthetic turf look and feel better than that mid-century stuff, it’s still just plastic. Some evidence suggests it sheds bits that endanger users and the environment, and that it contains PFAS “forever chemicals”—per- and polyfluoroalkyl substances, which are linked to a host of health issues. The padding within the plastic grass is usually made from shredded tires, which might also pose health risks. And plastic fields need to be replaced about once a decade, creating lots of waste. Yet people are buying a lot of the stuff. In 2001, Americans installed just over 7 million square meters of synthetic turf, just shy of 11,000 metric tons. By 2024, that number was 79 million square meters—enough to carpet all of Manhattan and then some, almost 120,000 metric tons. Synthetic turf covers 20,000 athletic fields and tens of thousands of parks, playgrounds, and backyards. And the US is just 20% of the global market.  Where real estate is limited and demand for athletic facilities is high, artificial turf is tempting. “It all comes down to land and demand.” Frank Rossi, professor of turf science, Cornell Those increases worry folks who study microplastics and environmental pollution. Any actual risk is hard to parse; the plastic-making industry insists that synthetic fields are safe if properly installed, but lots of researchers think that isn’t so. “They’re very expensive, they contain toxic chemicals, and they put kids at unnecessary risk,” says Philip Landrigan, a Boston College epidemiologist who has studied environmental toxins like lead and microplastics. But at Cornell, where real estate is limited and demand for athletic facilities is high, synthetic turf was a tempting option. As Frank Rossi, a professor of turf science at Cornell, told me: “It all comes down to land and demand.” In 1965, Houston’s new, domed base­ball stadium was an icon of space-age design. But the Astrodome had a problem: the sun. Deep in the heart of Texas, it shined brightly through the Astrodome’s skylights—so much so that players kept missing fly balls. So the club painted over the skylights. Denied sunlight, the grass in the outfield withered and died. A replacement was already in the works. In the late 1950s a Ford Foundation–funded educational laboratory determined that a soft, grasslike surface material would give city kids more places to play outside and had prevailed upon the Monsanto corporation to invent one. The result was clipped blades of nylon stuck to a rubber base, which the company called ChemGrass. Down it went into Houston’s outfield, where it got a new, buzzier name: AstroTurf. Workers lay artificial turf at the Astrodome in Houston on July 13, 1966. Developed by Monsanto, the material was originally known as ChemGrass but was later renamed AstroTurf after the stadium.AP PHOTO/ED KOLENOVSKY, FILE That first generation of simulated lawn was brittle and hard, but quality has improved. Today, there are a few competing products, but they’re all made by extruding a petroleum-based polymer—that’s plastic—through tiny holes and then stitching or fusing the resulting fibers to a carpetlike bottom. That gets attached to some kind of padding, also plastic. In the 1970s the industry started layering that over infill, usually sand; by the 1990s, “third generation” synthetic turf had switched to softer fibers made of polyethylene. Beneath that, they added infill that combined sand and a soft, cheap shredded rubber made from discarded automobile tires, which pile up by the hundreds of millions every year. This “crumb rubber” provides padding and fills spaces between the blades and the backing. In the early 1980s, nearly

Most

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. Is fake grass a bad idea? The AstroTurf wars are far from over.  In 2001, Americans installed just over 7 million square meters of synthetic turf. By 2024, that number was 79 million square meters—enough to carpet all of Manhattan and then some. The increase worries folks who study microplastics and environmental pollution.   While the plastic-making industry insists that synthetic fields are safe if properly installed, lots of researchers think that isn’t so. Find out why AstroTurf has ignited heated debates. —Douglas Main  This story is from the next issue of our print magazine, packed with stories all about nature. Subscribe now to read the full thing when it lands on Wednesday, April 22.  Mustafa Suleyman: AI development won’t hit a development wall anytime soon—here’s why  —Mustafa Suleyman, Microsoft AI CEO and Google DeepMind co-founder  The skeptics keep predicting that AI compute will soon hit a wall—and keep getting proven wrong. To understand why that is, you need to look at the forces driving the AI explosion.   Three advances are enabling exponential progress: faster basic calculators, high-bandwidth memory, and technologies that turn disparate GPUs into enormous supercomputers. Where does all this get us? Read the full op-ed on the future of AI development to learn more.   Desalination technology, by the numbers  —Casey Crownhart  When I started digging into desalination technology for a new story, I couldn’t help but obsess over the numbers.  I knew on some level that desalination—pulling salt out of seawater to produce fresh water—was an increasingly important technology, especially in water-stressed regions including the Middle East. But just how much some countries rely on desalination, and how big a business it is, still surprised me. Here are the extraordinary numbers behind the crucial water source.  This story is from The Spark, our weekly newsletter on the tech that could combat the climate crisis. Sign up to receive it in your inbox every Wednesday.  The must-reads  I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology.  1 Meta has launched the first AI model from its Superintelligence LabsMuse Spark is the company’s first model in a year. (Reuters $) + The closed model brings reasoning capabilities to the Meta AI app. (Engadget) + It’s built by Meta’s Superintelligence Labs, the unit led by Alexandr Wang. (TechCrunch)  2 Anthropic has lost a bid to pause the Pentagon’s blacklisting An appeals court in Washington, DC denied the request. (CNBC) + A California judge had temporarily blocked the blacklisting in March. (NPR) + The mixed rulings leave Anthropic in a legal limbo. (Wired $) + And open doors for smaller AI rivals. (Reuters $)  3 New evidence suggests Adam Back invented Bitcoin The British cryptographer may be the real Satoshi Nakamoto. (NYT $) + Back denies the claims. (BBC) + There’s a dark side to crypto’s permissionless dream. (MIT Technology Review)  4 Gen Z is cooling on AI The share feeling angry about it has risen from 22% to 31% in a year. (Axios) + Anti-AI protests are also growing. (MIT Technology Review)  5 War in the Gulf could tilt the cloud race toward China Huawei is pitching “multi-cloud” resilience to Gulf clients. (Rest of World)  6 Meta has killed a leaderboard of its AI token users It showed the top 250 users. (The Information $) + Meta blamed data leaks for the shutdown. (Fortune) + It encouraged “tokenmaxxing,” a growing phenomenon in Big Tech. (NYT $)  7 Did Artemis II really tell us anything new about space? Or was it primarily a PR exercise? (Ars Technica)  8 Israeli attacks have brutally exposed Lebanon’s digital infrastructure It’s managing a modern crisis without modern technology. (Wired $)  9 AI models could offer mathematicians a common language They hope it will simplify the process of verifying proofs. (Economist)   10 A “self-doxing’ rave is helping trans people stay safe online It’s among a series of digital self-defenses. (404 Media)  Quote of the day  “I feel like anything that I’m interested in has the potential of maybe getting replaced, even in the next few years.”  —Sydney Gill, a freshman at Rice University, tells the New York Times why she’s soured on AI.  One More Thing  A view inside ATLAS,one of two general-purpose detectors at the Large Hadron Collider.MAXIMILIEN BRICE/CERN Inside the hunt for new physics at the world’s largest particle collider  In 2012, data from CERN’s Large Hadron Collider (LHC) unearthed a particle called the Higgs boson. The discovery answered a nagging question: where do fundamental particles, such as the ones that make up all the protons and neutrons in our bodies, get their mass? But now particle physicists have reached an impasse in their quest to discover, produce, and study new particles at colliders. Find out what they’re trying to do about it. —Dan Garisto  We can still have nice things  A place for comfort, fun and distraction to brighten up your day. (Got any ideas? Drop me a line.)  + Enjoy this tale of the “joke” sound that accidentally defined 90s rave culture. + Take a nostalgic trip through the websites of the early 00s. + One for animal lovers: sperm whales have teamed up to support a newborn. + Here’s a long overdue answer to a vital question: can the world’s largest mousetrap catch a limousine? 

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. Desalination plants in the Middle East are increasingly vulnerable  As the conflict in Iran has escalated, a crucial resource is under fire: the desalinization technology that supplies water in the region.  President Donald Trump has threatened to destroy “possibly all desalinization plants” in Iran if the Strait of Hormuz is not reopened. The impact on farming, industry, and—crucially—drinking in the Middle East could be severe. Find out why.  —Casey Crownhart  This story is part of MIT Technology Review Explains, our series untangling the complex, messy world of technology to help you understand what’s coming next. You can read more from the series here.  AI is changing how small online sellers decide what to make  For small entrepreneurs, deciding what to sell and where to make it has traditionally been a slow, labor-intensive process. Now that work is increasingly being done by AI.    Tools like Alibaba’s Accio compress weeks of product research and supplier hunting into a single chat. Business owners and e-commerce experts say they’re making sourcing more accessible—and slashing the time from product idea to launch.   Read the full story on how AI is leveling the path to global manufacturing.  —Caiwei Chen  The gig workers who are training humanoid robots at home  When Zeus, a medical student in Nigeria, returns to his apartment from a long day at the hospital, he straps his iPhone to his forehead and records himself doing chores.   Zeus is a data recorder for Micro1, which sells the data he collects to robotics firms. As these companies race to build humanoids, videos from workers like Zeus have become the hottest new way to train them.    Micro1 has hired thousands of them in more than 50 countries, including India, Nigeria, and Argentina. The jobs pay well locally, but raise thorny questions around privacy and informed consent. The work can be challenging—and weird. Read the full story.   —Michelle Kim  This is our latest story to be turned into an MIT Technology Review Narrated podcast, which we’re publishing each week on Spotify and Apple Podcasts. Just navigate to MIT Technology Review Narrated on either platform, and follow us to get all our new content as it’s released.  The must-reads  I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology.  1 Anthropic’s new model found security problems in every OS and browser Claude Mythos has been heralded as a cybersecurity “reckoning.” (The Verge)  + Anthrophic is limiting the rollout over hacking fears. (CNBC) + It’s also launching a project that lets Mythos flag vulnerabilities. (Gizmodo) + Apple, Google, and Microsoft have joined the initiative. (ZDNET)  2 Iranian hackers are targeting American critical infrastructure Their focus is on energy and water infrastructure. (Wired) + They’re targeting industrial control devices. (TechCrunch)   3 Google’s AI Overviews deliver millions of incorrect answers per hour Despite a 90% accuracy rate. (NYT $) + AI means the end of internet search as we’ve known it. (MIT Technology Review)  4 Elon Musk is trying to oust OpenAI CEO Sam Altman in a lawsuit As remedies for Altman allegedly defrauding him. (CNBC) + Musk wants any damages given to OpenAI’s nonprofit arm. (WSJ $)  5 ICE has admitted it’s using powerful spyware The tools that can intercept encrypted messages. (NPR) + Immigration agencies are also weaponizing AI videos. (MIT Technology Review)  6 Greece has joined the countries banning kids from social media Under-15s will be blocked from 2027. (Reuters) + Australia introduced the world’s first social media ban for children. (Guardian) + Indonesia recently rolled out the first one in Southeast Asia. (DW)  + Experts say they’re a lazy fix. (CNBC)  7 Intel will help Elon Musk build his Terafab in Texas They aim to manufacture chips for AI projects. (Engadget) + Musk says it will be the largest-ever semiconductor factory. (Engadget) + Future AI chips could be built on glass. (MIT Technology Review)   8 TikTok is building a second billion-euro data center in Finland It’s moving data storage for European users. (Reuters) + Finland has become a magnet for data centers. (Bloomberg $) + But nobody wants one in their backyard. (MIT Technology Review)  9 Plans for Canada’s first “virtual gated community” have sparked a row The AI-powered surveillance system has divided neighbors. (Guardian) + Is the Pentagon allowed to surveil Americans with AI? (MIT Technology Review)  10 The high-tech engineering of the “space toilet” has been revealed Artemis II is the first mission to carry one around the world. (Vox)  Quote of the day  “This case has always been about Elon generating more power and more money for what he wants. His lawsuit remains nothing more than a harassment campaign that’s driven by ego, jealousy and a desire to slow down a competitor.”  —OpenAI criticizes Musk’s legal action in an X post.  One More Thing  USWDS Inside the US government’s brilliantly boring websites  You may not notice it, but your experience on every US government website is carefully crafted.  Each site aligns an official web design and a custom typeface. They aim to make government websites not only good-looking but accessible and functional for all.  MIT Technology Review dug into the system’s history and features. Find out what we discovered.  —Jon Keegan  We can still have nice things  A place for comfort, fun and distraction to brighten up your day. (Got any ideas? Drop me a line.)  + Rejoice in the splendor of the “Earthset” image captured by Artemis II. + Meet the fearless cat chasing off bears. + This document vividly explains what makes the octopus so unique. + Revealed: the rhythmic secret that makes emo music so angsty.