YouZum

Uncategorized

AI, Committee, Nachrichten, Uncategorized

The Download: Musk and Altman’s legal showdown, and AI’s profit problem

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. Elon Musk and Sam Altman are going to court over OpenAI’s future Elon Musk and OpenAI CEO Sam Altman head to trial this week in a case with sweeping consequences. Ahead of OpenAI’s IPO, the court could rule on whether the company can exist as a for-profit enterprise. It could even oust its leadership. Musk, an OpenAI co-founder, claims he was deceived into bankrolling the firm under false pretenses. He’s seeking $134 billion in damages, the removal of Altman and president Greg Brockman, and the company’s restoration to a non-profit. Find out how the trial could upend the global AI race. —Michelle Kim The missing step between hype and profit In a celebrated South Park episode, a community of gnomes sneak out at night to steal underpants. Why? The gnomes present their pitch deck. “Phase 1: Collect underpants. Phase 2: ? Phase 3: Profit.” It’s a business plan that captures the current state of AI.  Companies have built the tech (Step 1) and promised transformation (Step 3). But how they get there is still a big question mark. Read about the potential paths forward. —Will Douglas Heaven This story originally appeared in The Algorithm, our weekly newsletter giving you the inside track on all things AI. Sign up to receive it in your inbox every Monday. Welcome to the era of weaponized deepfakes For years, experts have warned that deepfakes could be deployed in malicious ways. These dangers are now here. Cheap, accessible models now produce weaponized deepfakes—from sexually explicit images to political propaganda—that look startlingly real. They’re already inciting violence, changing minds, and sowing mistrust, with women and marginalized groups disproportionately affected. Experts fear that they’re cratering trust and critical thinking. Here’s why they’re alarmed. —Eileen Guo Weaponized deepfakes are on our list of the 10 Things That Matter in AI Right Now, MIT Technology Review’s guide to what’s really worth your attention in the busy, buzzy world of AI.  The must-reads I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology. 1 OpenAI has ended its exclusive partnership with MicrosoftThe new deal allows OpenAI to court rivals such as Amazon. (Reuters $)+ Microsoft will still license OpenAI’s tech, but no longer exclusively. (NYT $)+ OpenAI is missing key growth targets ahead of its IPO. (WSJ $) 2 Google has signed a classified AI deal with the PentagonIt permits AI use for “any lawful government purpose.” (The Information $)+ Over 600 Google workers had called for a block on the deal. (QZ)+ AI firms are set to train military versions of their models on classified data. (MIT Technology Review) 3 The EU has told Google to open Android to AI rivalsIt wants to end Gemini’s built-in advantage. (Ars Technica)+ Google calls the move an “unwarranted intervention.” (WSJ $)+ A final decision is expected by the end of July. (Reuters $) 4 OpenAI is reportedly developing an AI-first smartphoneIt would replace apps with agents. (TechCrunch)+ Qualcomm and MediaTek may be developing its processors. (Gizmodo) 5 A brain implant for depression is moving into human testingThe FDA has approved a human study of the device. (Wired $)+ BCIs have thus far struggled to reach the market. (MIT Technology Review) 6 A populist backlash against AI is gaining momentum in rural AmericaFrom Indiana to Idaho, voters are pushing back against the technology. (NYT $)+ Anti-AI protests are expanding worldwide. (MIT Technology Review) 7 DeepSeek has priced its new model 97% below OpenAI’s GPT-5.5It aims to attract more enterprises, developers, and agent-based users. (SCMP)+ Here are three reasons why DeepSeek V4 matters. (MIT Technology Review) 8 AI now generates a third of new websitesA study found it’s making the web more cheery and less verbose. (404 Media) 9 Top talent is leaving Big Tech to launch their own AI startupsMeta, Google, and OpenAI are facing a brain drain. (CNBC) 10 Taylor Swift is trademarking her voice and imageThe Grammy winner has been the target of numerous deepfakes. (NBC News)+ A growing number of celebrities are fighting AI with trademarks. (BBC) Quote of the day “The reality is people don’t like him.” —Judge Yvonne Gonzalez Rogers reacts to prospective jurors confessing their negative views of Elon Musk ahead of his legal battle with Sam Altman, The Verge reports. One More Thing How covid conspiracy theories led to an alarming resurgence in AIDS denialism When Joe Rogan falsely declared that “party drugs” were an “important factor in AIDS,” several million people were listening. He also asserted that AZT, the earliest drug used to treat AIDS, killed people “quicker” than the disease itself—another claim that has been disproven. Such comments illustrate an unmistakable resurgence in AIDS denialism: a false collection of theories arguing either that HIV does not cause AIDS or that there is no such thing as HIV at all. By the dawn of the millennium, these claims had largely fallen out of favour. That changed when the coronavirus arrived. Follow the digital path from Covid skepticism to the return of a deadly conspiracy theory. —Anna Merlan We can still have nice things A place for comfort, fun and distraction to brighten up your day. (Got any ideas? Drop me a line.) + Explore the planets from your laptop with this live sky map.+ This marathon DJ set from Daphni is an incredible journey through electronic music.+ NASA’s stunning Artemis II wallpapers bring a high-res piece of deep space to your phone.+ This fascinating GPS explainer breaks down how your phone figures out exactly where you are.

The Download: Musk and Altman’s legal showdown, and AI’s profit problem Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

The Download: DeepSeek’s latest AI breakthrough, and the race to build world models

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. Three reasons why DeepSeek’s new model matters On Friday, Chinese AI firm DeepSeek released a preview of V4, its long-awaited new flagship model. Notably, the model can process much longer prompts than its last generation, thanks to a new design that handles large amounts of text more efficiently. While the model remains open source, its performance matches leading closed-source rivals from Anthropic, OpenAI, and Google. It is also DeepSeek’s first release optimized Huawei’s Ascend chips—a key test of China’s dependence on Nvidia. Here are three ways V4 could shake up AI. —Caiwei Chen The rise of world models AI systems have already gained impressive mastery over the digital world, but the physical world remains humanity’s domain. As it turns out, building an AI that composes novels or code apps is far easier than developing one to fold laundry or navigate city streets. To bridge this gap, many researchers believe you need something called a world model. Proponents like Stanford professor Fei-Fei Li and AMI Labs founder Yann LeCun argue these models can overcome the well-known limitations of LLMs—and realize AI’s promise for robotics. Find out why they’ve brought world models to the forefront of the field. —Grace Huckins World models are on our list of the 10 Things That Matter in AI Right Now, our essential guide to what’s really worth your attention in the field. Subscribers can watch an exclusive roundtable unveiling the technologies and trends on the list, with analysis from MIT Technology Review’s AI reporter Grace Huckins and executive editors Amy Nordrum and Niall Firth. The must-reads I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology. 1 China has blocked Meta’s $2 billion acquisition of AI startup ManusRegulators cited national security grounds. (WSJ $)+ Beijing called the deal a “conspiratorial” attempt to hollow out its tech base. (FT $)+ The country is tightening its grip on AI firms that try to leave. (TechCrunch)+ The decision escalates China’s AI rivalry with the US. (Bloomberg $)+ But there will be no winners in their competition. (MIT Technology Review) 2 Google is investing up to $40 billion in AnthropicIn a deal valuing the AI firm at $350 billion. (CNBC)+ The funding will support the firm’s growing computing needs. (TechCrunch)+ Anthropic and OpenAI are fighting for compute capacity. (Axios) 3 President Trump just fired the entire National Science BoardThe NSF has played a crucial role in developing technology. (The Verge)+ The move heightens fears over political interference in US science. (Nature) 4 Conspiracy theories about the Washington shooting are proliferating onlineOver 300,000 posts appeared on X using the keyword “staged.” (NYT $)+ The theories are also swirling on Bluesky and Instagram. (Wired) 5 The AI compute crunch is starting to hit the broader economy.It’s affecting jobs, gadgets, and electricity prices. (404 Media)+ The AI compute explosion is the tech story of our time. (MIT Technology Review) 6 Elon Musk says a new banking tool brings X close to a “super app”He’s pledged to launch the tool this month. (Bloomberg) 7 AI optimism is surging across Asia while US sentiment coolsThe divide could shape where adoption happens fastest. (Rest of World) 8 Apple is tying its new CEO’s ascent to its first foldable iPhoneIt wants to build the buzz around John Ternus. (Gizmodo)  9 Twelve firms are developing the Golden Dome’s space-based interceptorsThey’ve won contracts worth up to $3.2 billion. (Ars Technica) 10 NASA has shared promising results from Artemis IIThe spacecraft and rocket fared well. (Engadget) Quote of the day “Getting out the truth and establishing facts and reliable information takes time. But our audiences really don’t have that kind of patience.” —Amanda Crawford, associate professor at the University of Connecticut, tells the NYT why conspiracy theories are gaining traction online. One More Thing MIRIAM MARTINCIC Welcome to Kenya’s Great Carbon Valley: a bold new gamble to fight climate change Kenya’s Great Rift Valley is home to five geothermal power stations, which harness clouds of steam to generate about a quarter of the country’s electricity. But some of the energy escapes into the atmosphere, while even more remains underground for lack of demand. That’s what brought Octavia Carbon here. Last year, the startup began harnessing some of that excess energy to remove CO2 from the air. The company says the method is efficient, affordable, and—crucially—scalable. But the project also faces fierce opposition.  Read the full story on the future of Kenya’s “Great Carbon Valley.” —Diana Kruzman We can still have nice things A place for comfort, fun and distraction to brighten up your day. (Got any ideas? Drop me a line.) + Fred Again’s Tiny Desk Concert is a masterclass in intimate performance.+ Here’s a delightful look at how we’re all linked through geography and shared heritage.+ Take a short, peaceful break to watch Tokyo’s cherry blossoms from a bird’s eye view.+ There’s something oddly satisfying about watching an industrial shredder turn everyday items into confetti.

The Download: DeepSeek’s latest AI breakthrough, and the race to build world models Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

Meta AI Releases Sapiens2: A High-Resolution Human-Centric Vision Model for Pose, Segmentation, Normals, Pointmap, and Albedo

If you’ve ever watched a motion capture system struggle with a person’s fingers, or seen a segmentation model fail to distinguish teeth from gums, you already understand why human-centric computer vision is hard. Humans are not just objects, they come with articulated structure, fine surface details, and enormous variation in pose, clothing, lighting, and ethnicity. Getting a model to understand all of that, at once, across arbitrary real-world images, is genuinely difficult. Meta AI research team introduced Sapiens2, the second generation of its foundation model family for human-centric vision. Trained on a newly curated dataset of 1 billion human images, spanning model sizes from 0.4B to 5B parameters, and designed to operate at native 1K resolution with hierarchical variants supporting 4K, Sapiens2 is a substantial leap over its predecessor across every benchmark the team evaluated. https://arxiv.org/pdf/2604.21681 What Sapiens2 is Trying to Solve The original Sapiens model relied primarily on Masked Autoencoder (MAE) pretraining. MAE works by masking a large portion of input image patches, 75% in this case, and training the model to reconstruct the missing pixels. This forces the model to learn spatial details and textures, which is useful for dense prediction tasks like segmentation or depth estimation. The problem is that MAE, as a form of masked image modeling (MIM), learns largely through compression. It doesn’t naturally learn high-level semantics. It can tell you what something looks like, but not necessarily what it means in the context of a human body. That’s where contrastive learning (CL) methods like DINO and SimCLR shine: they organize representations semantically by training the model to treat different views of the same image as similar and views of different images as distinct. But CL has its own tradeoff. Its aggressive augmentation strategies like color jitter, blurring, can strip away appearance cues like skin tone or lighting conditions that are critical for tasks like albedo estimation (recovering the true color of a surface independent of lighting). This is what the research team calls representation drift. Sapiens2 addresses this problem directly by combining both objectives: a masked image reconstruction loss (LMAE) to preserve low-level fidelity, and a global contrastive loss (LCL) on the [CLS] token using a student-teacher framework based on DINOv3, where the teacher’s parameters are an exponential moving average (EMA) of the student. Crucially, color augmentations are not applied to global views used for the MAE objective, preserving the appearance cues needed for photorealistic tasks. The joint objective is L = LMAE + λLCL. https://arxiv.org/pdf/2604.21681 The Data: Humans-1B Getting 1 billion training images right required a multi-stage filtering pipeline. Starting from a web-scale pool of approximately 4 billion images, Meta team applied bounding box detection, head-pose estimation, aesthetic and realism scoring, CLIP-based feature filtering, and text-overlay detection. The result is a curated corpus where every image contains at least one prominent person with a minimum short-side resolution of 384 pixels. To ensure diversity, the research team used perceptual hashing and deep-feature nearest-neighbor pruning for deduplication, then clustered visual embeddings and applied selective sampling to balance the dataset across poses, viewpoints, occlusion levels, clothing types, and lighting conditions. No task labels or human-specific priors were injected during pretraining — just images. The Architecture: Scaling to 5B and 4K Sapiens2 introduces four model sizes: 0.4B, 0.8B, 1B, and 5B parameters, each at native 1K resolution. The 5B model is the highest-FLOPs vision transformer reported to date at 15.722 TFLOPs. For 4K resolution, the research team adopted a hierarchical windowed attention design. The first K layers apply windowed self-attention locally to capture fine texture and boundaries within spatial windows. A [CLS]-guided pooling step then downsamples the 2D token grid by a spatial stride √ω, and the subsequent L layers apply global self-attention over this reduced sequence. This layout is compatible with MAE-style pretraining because masked tokens can be dropped after the local stage, preventing information from leaking across masked regions — a problem that convolutional backbones typically need masked convolutions to avoid. The masking strategy itself is also carefully designed: Sapiens2 uses mixed blockwise/patchwise masking (blockwise probability 0.4) at a 75% mask ratio with patch size 16. At 1024×768 resolution (64×48 = 3072 patches), this masks approximately 2304 patches per image which is enough to create coarse occlusions that regularize MAE while preserving sufficient context for the contrastive objective. For stability at scale, the architecture incorporates several improvements: RMSNorm replacing LayerNorm, Grouped-Query Attention (GQA) in mid-depth blocks for higher throughput, QK-Norm for robust high-resolution training, and SwiGLU feed-forward layers. The decoder uses pixel-shuffle upsampling for sub-pixel reasoning. Decoder output resolution was also increased from 0.5K to 1K for base backbones, and to 2K for 4K backbones. Post-Training: Five Human Tasks, 10× More Supervision A critical improvement over the original Sapiens is the scale and quality of task-specific supervision. Relative to the first generation, Sapiens2 scales task-specific labels by 10×, typically reaching around 1 million labels per task. After pretraining, the backbone is fine-tuned for five downstream tasks using lightweight task-specific heads while leaving the backbone unchanged: Pose Estimation: A 308-keypoint full-body skeleton with dense face (243 keypoints) and hand (40 keypoints) coverage. The research team newly annotated 100K in-the-wild images to complement studio capture data, improving generalization significantly. Body-Part Segmentation: 29 semantic classes (extended from 28 by adding eyeglasses), trained with per-pixel weighted cross-entropy combined with Dice loss for sharper boundaries. Pointmap Estimation: Rather than predicting relative depth, Sapiens2 regresses a per-pixel 3D pointmap P̂(u) ∈ ℝ³ in the camera frame — a harder task that requires reasoning about camera intrinsics. Normal Estimation: Per-pixel surface unit normals, decoded using multiple PixelShuffle layers for artifact-free upsampling. Albedo Estimation: Per-pixel diffuse albedo Â(u) ∈ [0,1]³, trained purely on synthetic high-fidelity data and designed to recover true skin tone and clothing color under varying illumination. Results The numbers are difficult to argue with. On the 11K-image in-the-wild pose test set, Sapiens2-5B achieves 82.3 mAP compared to 78.3 mAP for Sapiens-2B — a +4 mAP improvement. On body-part segmentation, even the smallest model, Sapiens2-0.4B, scores 79.5 mIoU (+21.3 over

Meta AI Releases Sapiens2: A High-Resolution Human-Centric Vision Model for Pose, Segmentation, Normals, Pointmap, and Albedo Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

Rebuilding the data stack for AI

Artificial intelligence may be dominating boardroom agendas, but many enterprises are discovering that the biggest obstacle to meaningful adoption is the state of their data. While consumer-facing AI tools have dazzled users with speed and ease, enterprise leaders are discovering that deploying AI at scale requires something far less glamorous but far more consequential: data infrastructure that is unified, governed, and fit for purpose. That gap between AI ambition and enterprise readiness is becoming one of the defining challenges of this next phase of digital transformation. As Bavesh Patel, senior vice president of Databricks, puts it, “the quality of that AI and how effective that AI is, is really dependent on information in your organization.” Yet in many companies, that information remains fragmented across legacy systems, siloed applications, and disconnected formats, making it nearly impossible for AI systems to generate trustworthy, context-rich outputs. “Really, the big competitive differentiator for most organizations is their own data and then their third-party data that they can add to it,” says Patel. For enterprise AI to deliver value, data must be consolidated into open formats, governed with precision, and made accessible across functions. Without that foundation, businesses risk “terrible AI,” as Patel bluntly describes it. That means moving beyond siloed SaaS platforms and disconnected dashboards toward a unified, open data architecture capable of combining structured and unstructured data, preserving real-time context, and enforcing rigorous access controls. When the groundwork is laid correctly, organizations can move toward measurable outcomes, unlocking efficiencies, automating complex workflows, and even launching entirely new lines of business. That value focus is critical, says Rajan Padmanabhan, unit technology officer at Infosys, especially as enterprises seek precision in the outputs driving business decisions. Rather than treating AI initiatives as isolated innovation projects, leading companies are tying AI deployment directly to business metrics, using governance frameworks to determine what delivers results and what should be abandoned quickly. “We see this big opportunity just with AI literacy with business users, where they’re very eager to understand how they should be thinking about AI,” adds Patel. “What does AI mean when you peel the covers? What are the pieces and the building blocks that you need to put in place, both from a technology and a training and an enablement standpoint?” The possibilities ahead are substantial. As AI agents evolve from copilots into autonomous operators capable of managing workflows and transactions, the organizations that win will be those that build the right foundation now. “What we are seeing as a new way of thinking is moving from a system of execution or a system of engagement to a system of action,” notes  Padmanabhan. “That is the new way we see the road ahead.” The future of AI in the enterprise will be determined by whether businesses can turn fragmented information into a strategic asset capable of powering both smarter decisions and entirely new ways of operating. This episode of Business Lab is produced in partnership with Infosys Topaz. Full Transcript: Megan Tatum: From MIT Technology Review, I’m Megan Tatum, and this is Business Lab, the show that helps business leaders make sense of new technologies coming out of the lab and into the marketplace. This episode is produced in partnership with Infosys Topaz. Now, recent advancements in AI may have unlocked some compelling new industrial applications, but a reliance on inadequate data models means that many enterprises are hitting a brick wall. AI and agentic AI in particular place a whole new set of demands on data. The technology requires greater access, context, and guardrails to operate effectively. Existing data models often fall short. They’re too fragmented or siloed. Data itself often lacks quality. To bridge the gap, they require an AI-ready upgrade. Two words for you: data reconfigured. My guest today, are Bavesh Patel, senior vice president for Go-to-Market at Databricks, and Rajan Padmanabhan, unit technology officer for data analytics and AI at Infosys. Welcome, Bavesh and Rajan. Rajan Padmanabhan: Thank you. Thanks for having us. Bavesh Patel: Thanks for having us. Megan: Fantastic. Thank you both so much for joining us today. Bavesh, if I could come to you first, when we talk about AI-ready data, what exactly do we mean? What new demands does AI place on data, and how does this impact the way it needs to be structured and used? Bavesh: Yeah. Great question. Appreciate you hosting us today. I think that obviously the whole world is enamored with AI because of all of the power that we can all see as users. AI is now democratized across hundreds of millions of users. And when we think about enterprises and businesses using AI, the quality of that AI and how effective that AI is really dependent on information in your organization, and that’s data. And what we found is that most enterprises, their data is kind of locked away in these different applications and different systems. And it’s very difficult to get a good view of, what is all my data? How trustworthy is it? How recent and fresh is it? And all of that is being injected into the AI. Unless you have a proper understanding of your data, the ability to ensure that it’s data that’s accurate and that can be used so that the AI can take advantage of it, you’re actually going to end up having terrible AI. We see a lot of customers spend time on cleansing their data, organizing their data, making sure it’s access controlled correctly, and that tends to be the fuel of good AI. Megan: Yeah. It’s such a foundational thing, isn’t it? But it can be missed, I think, quite easily. Rajan, what difference can having AI-ready data really make for enterprises as they unlock that full potential of AI and its applications? Rajan: First and foremost, thanks for having us. It’s a pleasure. I think in continuation of what Bavesh talked about, see, data and AI is pretty synonymous. And similarly, the consumer AI and enterprise AI and

Rebuilding the data stack for AI Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

A Coding Implementation on kvcached for Elastic KV Cache Memory, Bursty LLM Serving, and Multi-Model GPU Sharing

In this tutorial, we explore kvcached, a dynamic KV-cache implementation on top of vLLM, to understand how dynamic KV-cache allocation transforms GPU memory usage for large language models. We begin by setting up the environment and deploying lightweight Qwen2.5 models through an OpenAI-compatible API, ensuring a realistic inference workflow. We then design controlled experiments where we simulate bursty workloads to observe how memory behaves under both elastic and static allocation strategies. Through systematic measurement and visualization, we directly compare VRAM utilization and latency, and extend the setup to a multi-model scenario where we observe how memory flexibly shifts across active workloads in real time. Copy CodeCopiedUse a different Browser import os, sys, time, json, subprocess, threading, signal, shutil from pathlib import Path def sh(cmd, check=True): return subprocess.run(cmd, check=check, shell=isinstance(cmd, str)) try: import torch except ImportError: sh([sys.executable, “-m”, “pip”, “install”, “-q”, “torch”]) import torch assert torch.cuda.is_available(), “No GPU detected. In Colab: Runtime > Change runtime type > GPU.” props = torch.cuda.get_device_properties(0) print(f”[GPU] {torch.cuda.get_device_name(0)} ” f”({props.total_memory / 1e9:.1f} GB, ” f”compute capability {props.major}.{props.minor})”) def pip_install(*pkgs, extra=()): subprocess.run([sys.executable, “-m”, “pip”, “install”, “-q”, *pkgs, *extra], check=True) print(“[install] vLLM …”) pip_install(“vllm==0.10.2”) print(“[install] kvcached (compiles a small CUDA extension) …”) pip_install(“kvcached”, extra=[“–no-build-isolation”]) print(“[install] misc (matplotlib, requests, pynvml) …”) pip_install(“matplotlib”, “requests”, “pynvml”, “numpy”) MODEL_A = “Qwen/Qwen2.5-0.5B-Instruct” MODEL_B = “Qwen/Qwen2.5-1.5B-Instruct” PORT_A, PORT_B = 8001, 8002 MAX_MODEL_LEN = 2048 We start by setting up the environment and verifying that a GPU is available for our experiments. We install all required dependencies including vLLM and kvcached along with supporting libraries. We then define our model configurations and ports to prepare for launching the inference servers. Copy CodeCopiedUse a different Browser def launch_vllm(model, port, kvcached=True, gpu_mem_util=0.55, log_path=None): “””Start a vLLM OpenAI-compatible server as a subprocess. With kvcached=True the autopatch hooks replace vLLM’s KV-cache allocator with the elastic one.””” env = os.environ.copy() env[“VLLM_USE_V1”] = “1” if kvcached: env[“ENABLE_KVCACHED”] = “true” env[“KVCACHED_AUTOPATCH”] = “1” env[“KVCACHED_IPC_NAME”] = f”kvc_{port}” cmd = [ sys.executable, “-m”, “vllm.entrypoints.openai.api_server”, “–model”, model, “–port”, str(port), “–max-model-len”, str(MAX_MODEL_LEN), “–disable-log-requests”, “–no-enable-prefix-caching”, “–enforce-eager”, ] if not kvcached: cmd += [“–gpu-memory-utilization”, str(gpu_mem_util)] log = open(log_path or os.devnull, “w”) proc = subprocess.Popen(cmd, env=env, stdout=log, stderr=subprocess.STDOUT, preexec_fn=os.setsid) return proc, log def wait_ready(port, timeout=420): import requests url = f”http://localhost:{port}/v1/models” t0 = time.time() while time.time() – t0 < timeout: try: if requests.get(url, timeout=2).status_code == 200: return True except Exception: pass time.sleep(3) raise TimeoutError(f”vLLM on port {port} didn’t come up within {timeout}s”) def shutdown(proc, log): if proc and proc.poll() is None: try: os.killpg(os.getpgid(proc.pid), signal.SIGTERM) proc.wait(timeout=45) except Exception: os.killpg(os.getpgid(proc.pid), signal.SIGKILL) if log and not log.closed: log.close() time.sleep(3) We implement helper functions to launch and manage the vLLM server with and without kvcached enabled. We configure environment variables to activate dynamic KV-cache behavior and ensure proper server initialization. We also define utilities to wait for server readiness and safely shut down processes after execution. Copy CodeCopiedUse a different Browser import pynvml pynvml.nvmlInit() NV_HANDLE = pynvml.nvmlDeviceGetHandleByIndex(0) def vram_used_mb(): info = pynvml.nvmlDeviceGetMemoryInfo(NV_HANDLE) return info.used / (1024 ** 2) class MemorySampler(threading.Thread): def __init__(self, interval=0.2): super().__init__(daemon=True) self.interval = interval self.samples = [] self._stop = threading.Event() def run(self): t0 = time.time() while not self._stop.is_set(): self.samples.append((time.time() – t0, vram_used_mb())) time.sleep(self.interval) def stop(self): self._stop.set(); self.join() import requests from concurrent.futures import ThreadPoolExecutor PROMPTS = [ “Explain quantum entanglement to a curious 10-year-old.”, “Write a Python function that detects cycles in a linked list.”, “Summarize the plot of Hamlet in one paragraph.”, “List 5 surprising household uses for baking soda with explanations.”, “Compose a vivid haiku about rainy Monday mornings.”, “Describe the Fermi paradox and three plausible resolutions.”, “Translate ‘knowledge is power’ into French, German, and Japanese.”, “Explain the difference between TCP and UDP with real examples.”, ] def bursty_workload(port, model, n_bursts=3, burst_size=6, pause=6.0, max_tokens=180): “””Fire n_bursts waves of burst_size concurrent requests with an idle gap between waves. The idle gap is where kvcached releases physical VRAM — a static-allocation engine simply cannot.””” url = f”http://localhost:{port}/v1/chat/completions” def one(i): body = { “model”: model, “messages”: [{“role”: “user”, “content”: PROMPTS[i % len(PROMPTS)]}], “max_tokens”: max_tokens, “temperature”: 0.7, } t0 = time.time() r = requests.post(url, json=body, timeout=180) r.raise_for_status() return time.time() – t0 latencies = [] with ThreadPoolExecutor(max_workers=burst_size) as ex: for b in range(n_bursts): print(f” burst {b+1}/{n_bursts} ({burst_size} concurrent)”) latencies += list(ex.map(one, range(burst_size))) if b < n_bursts – 1: time.sleep(pause) return latencies We initialize GPU memory tracking using pynvml to monitor VRAM usage in real time. We create a background sampling thread that continuously records memory consumption during experiments. We then define a bursty workload generator that sends concurrent requests to simulate realistic LLM usage patterns. Copy CodeCopiedUse a different Browser print(“n=== Experiment 1: vLLM + kvcached ===”) proc, log = launch_vllm(MODEL_A, PORT_A, kvcached=True, log_path=”/tmp/vllm_kvc.log”) try: wait_ready(PORT_A) idle_kvc = vram_used_mb() print(f” Idle VRAM after load (weights only): {idle_kvc:.0f} MB”) sampler = MemorySampler(); sampler.start() lat_kvc = bursty_workload(PORT_A, MODEL_A) time.sleep(6) sampler.stop() mem_kvc = sampler.samples finally: shutdown(proc, log) print(“n=== Experiment 2: vLLM baseline (static KV allocation) ===”) proc, log = launch_vllm(MODEL_A, PORT_A, kvcached=False, log_path=”/tmp/vllm_base.log”) try: wait_ready(PORT_A) idle_base = vram_used_mb() print(f” Idle VRAM (weights + pre-reserved KV pool): {idle_base:.0f} MB”) sampler = MemorySampler(); sampler.start() lat_base = bursty_workload(PORT_A, MODEL_A) time.sleep(6) sampler.stop() mem_base = sampler.samples finally: shutdown(proc, log) We run the first experiment with kvcached enabled and capture both memory usage and latency metrics. We then execute the same workload under a baseline static allocation setup for comparison. We collect and store all results to enable a clear side-by-side evaluation of both approaches. Copy CodeCopiedUse a different Browser import numpy as np import matplotlib.pyplot as plt fig, axes = plt.subplots(1, 2, figsize=(14, 4.5)) tk, mk = zip(*mem_kvc); tb, mb = zip(*mem_base) axes[0].plot(tk, mk, label=”with kvcached”, linewidth=2, color=”#1f77b4″) axes[0].plot(tb, mb, label=”baseline (static)”, linewidth=2, linestyle=”–“, color=”#d62728″) axes[0].axhline(idle_kvc, color=”#1f77b4″, alpha=.3, linestyle=”:”) axes[0].axhline(idle_base, color=”#d62728″, alpha=.3, linestyle=”:”) axes[0].set_xlabel(“time (s)”); axes[0].set_ylabel(“GPU memory used (MB)”) axes[0].set_title(“VRAM under a bursty workloadn(dotted = idle-baseline VRAM)”) axes[0].grid(alpha=.3); axes[0].legend() axes[1].boxplot([lat_kvc, lat_base], labels=[“kvcached”, “baseline”]) axes[1].set_ylabel(“request latency (s)”) axes[1].set_title(f”Latency across {len(lat_kvc)} requests”) axes[1].grid(alpha=.3) plt.tight_layout() plt.savefig(“/content/kvcached_single_model.png”, dpi=120, bbox_inches=”tight”) plt.show() print(“n— Single-model summary ——————————————–“) print(f” Idle VRAM kvcached: {idle_kvc:>6.0f} MB ” f”baseline: {idle_base:>6.0f} MB ” f”(savings: {idle_base – idle_kvc:>5.0f} MB)”) print(f” Peak VRAM kvcached: {max(mk):>6.0f} MB ” f”baseline: {max(mb):>6.0f} MB”) print(f” Median lat. kvcached: {np.median(lat_kvc):>6.2f} s

A Coding Implementation on kvcached for Elastic KV Cache Memory, Bursty LLM Serving, and Multi-Model GPU Sharing Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

A Coding Tutorial on Datashader on Rendering Massive Datasets with High-Performance Python Visual Analytics

In this tutorial, we explore Datashader, a powerful, high-performance visualization library for rendering massive datasets that quickly overwhelm traditional plotting tools. We work through its full rendering pipeline in Google Colab, starting from dense point clouds and reduction-based aggregations to categorical rendering, line visualizations, raster data, quadmesh grids, compositing, and dashboard-style analytical views. As we move through each section, we focus on how Datashader transforms raw large-scale data into meaningful visual structure with speed, flexibility, and visual clarity, while keeping Matplotlib as the final presentation layer. Copy CodeCopiedUse a different Browser import subprocess, sys subprocess.check_call([sys.executable, “-m”, “pip”, “install”, “-q”, “datashader”, “colorcet”, “numba”, “scipy”]) import numpy as np import pandas as pd import datashader as ds import datashader.transfer_functions as tf from datashader import reductions as rd import colorcet as cc import matplotlib.pyplot as plt import matplotlib.colors as mcolors from matplotlib.gridspec import GridSpec from scipy.stats import multivariate_normal import time, warnings warnings.filterwarnings(“ignore”) print(“Datashader version:”, ds.__version__) def show(img, title=””, ax=None, figsize=(6, 5)): standalone = ax is None if standalone: fig, ax = plt.subplots(figsize=figsize) rgba = img.to_pil() ax.imshow(rgba, origin=”upper”, aspect=”auto”) ax.set_title(title, fontsize=11, fontweight=”bold”) ax.axis(“off”) if standalone: plt.tight_layout() plt.show() print(“n=== SECTION 1: Core Pipeline ===”) rng = np.random.default_rng(42) N = 2_000_000 x = np.concatenate([rng.normal(-1, 0.5, N//3), rng.normal( 1, 0.5, N//3), rng.normal( 0, 1.5, N//3)]) y = np.concatenate([rng.normal(-1, 0.5, N//3), rng.normal( 1, 0.5, N//3), rng.normal( 0, 0.5, N//3)]) df_base = pd.DataFrame({“x”: x, “y”: y}) canvas = ds.Canvas(plot_width=600, plot_height=500, x_range=(-4, 4), y_range=(-4, 4)) agg = canvas.points(df_base, “x”, “y”, agg=rd.count()) fig, axes = plt.subplots(1, 3, figsize=(15, 4)) combos = [ (“Linear / blues”, tf.shade(agg, cmap=cc.blues, how=”linear”)), (“Log / fire”, tf.shade(agg, cmap=cc.fire, how=”log” )), (“Eq-hist / bmy”, tf.shade(agg, cmap=cc.bmy, how=”eq_hist”)), ] for ax, (title, img) in zip(axes, combos): show(img, title, ax=ax) plt.suptitle(“Section 1 – 2 M points: Linear vs Log vs Eq-Hist normalisation”, fontsize=13, fontweight=”bold”) plt.tight_layout() plt.show() print(“n=== SECTION 2: Reduction Types ===”) n_actual = len(df_base) df_base[“value”] = rng.exponential(scale=2, size=n_actual) df_base[“label”] = pd.Categorical( rng.choice([“A”, “B”, “C”], size=n_actual), categories=[“A”, “B”, “C”] ) canvas2 = ds.Canvas(plot_width=400, plot_height=350, x_range=(-4, 4), y_range=(-4, 4)) reductions_cfg = [ (“count()”, rd.count(), cc.kbc), (“sum(value)”, rd.sum(“value”), cc.CET_L3), (“mean(value)”, rd.mean(“value”), cc.CET_D4), (“std(value)”, rd.std(“value”), cc.CET_L16), (“min(value)”, rd.min(“value”), cc.CET_L17), (“max(value)”, rd.max(“value”), cc.bgyw), (“var(value)”, rd.var(“value”), cc.CET_L18), (“count_cat(label)”, rd.count_cat(“label”), None), ] fig, axes = plt.subplots(2, 4, figsize=(18, 9)) axes = axes.flat for ax, (name, agg_fn, cmap) in zip(axes, reductions_cfg): agg_r = canvas2.points(df_base, “x”, “y”, agg=agg_fn) if cmap is None: img = tf.shade(agg_r, color_key={“A”:”#e41a1c”,”B”:”#377eb8″,”C”:”#4daf4a”}) else: img = tf.shade(agg_r, cmap=cmap, how=”eq_hist”) show(img, name, ax=ax) plt.suptitle(“Section 2 – All Reduction Types on 2 M points”, fontsize=14, fontweight=”bold”) plt.tight_layout() plt.show() print(“n=== SECTION 3: Categorical Visualisation ===”) N_cat = 500_000 categories = [“Cluster A”, “Cluster B”, “Cluster C”, “Cluster D”] centers = [(-2, -2), (-2, 2), (2, -2), (2, 2)] colors = {“Cluster A”:”#e41a1c”,”Cluster B”:”#377eb8″, “Cluster C”:”#4daf4a”,”Cluster D”:”#ff7f00″} frames = [] for cat, (cx, cy) in zip(categories, centers): n = N_cat // len(categories) frames.append(pd.DataFrame({ “x”: rng.normal(cx, 0.8, n), “y”: rng.normal(cy, 0.8, n), “cat”: pd.Categorical([cat]*n, categories=categories), })) df_cat = pd.concat(frames, ignore_index=True) canvas3 = ds.Canvas(plot_width=500, plot_height=500, x_range=(-5, 5), y_range=(-5, 5)) agg_cat = canvas3.points(df_cat, “x”, “y”, agg=rd.count_cat(“cat”)) fig, axes = plt.subplots(1, 3, figsize=(16, 5)) img_raw = tf.shade(agg_cat, color_key=colors) show(img_raw, “Raw (no spread)”, ax=axes[0]) img_sp1 = tf.spread(tf.shade(agg_cat, color_key=colors), px=1) show(img_sp1, “Spread px=1″, ax=axes[1]) img_bg = tf.set_background(tf.shade(agg_cat, color_key=colors), color=”black”) show(img_bg, “Black background”, ax=axes[2]) for cat, col in colors.items(): axes[2].plot([], [], “o”, color=col, label=cat, markersize=8) axes[2].legend(loc=”lower right”, fontsize=8, framealpha=0.6) plt.suptitle(“Section 3 – Categorical Rendering (500 k points)”, fontsize=13, fontweight=”bold”) plt.tight_layout() plt.show() We install the required libraries and import everything needed to build a complete Datashader workflow in Google Colab. We define a helper function to display Datashader images with Matplotlib, which keeps the rendering pipeline simple and visually consistent. We then begin with the core Datashader pipeline, explore multiple reduction types, and show how categorical data can be rendered clearly using color keys, spreading, and background adjustments. Copy CodeCopiedUse a different Browser print(“n=== SECTION 4: Line Rendering ===”) n_series, n_steps = 5_000, 500 t = np.linspace(0, 1, n_steps) xs = np.tile(t, n_series) walks = np.cumsum(rng.normal(0, 0.05, (n_series, n_steps)), axis=1) ys = walks.ravel() series_id = np.repeat(np.arange(n_series), n_steps) df_lines = pd.DataFrame({“x”: xs, “y”: ys, “id”: series_id}) canvas4 = ds.Canvas(plot_width=700, plot_height=450, x_range=(0, 1), y_range=(-6, 6)) agg_lines = canvas4.line(df_lines, “x”, “y”, agg=rd.count(), line_width=1) fig, axes = plt.subplots(1, 2, figsize=(14, 5)) show(tf.shade(agg_lines, cmap=cc.fire, how=”eq_hist”), “5 000 random walks – eq_hist / fire”, ax=axes[0]) show(tf.shade(agg_lines, cmap=cc.blues, how=”log”), “5 000 random walks – log / blues”, ax=axes[1]) plt.suptitle(“Section 4 – Line / Time-Series Rendering”, fontsize=13, fontweight=”bold”) plt.tight_layout() plt.show() print(“n=== SECTION 5: Raster / Grid Data ===”) import xarray as xr res = 1000 lon = np.linspace(-180, 180, res) lat = np.linspace(-90, 90, res) LON, LAT = np.meshgrid(lon, lat) z = ( multivariate_normal.pdf(np.stack([LON, LAT], -1), mean=[30, 30], cov=[[800,0],[0,500]]) + multivariate_normal.pdf(np.stack([LON, LAT], -1), mean=[-60, -20], cov=[[600,0],[0,400]]) + 0.02 * rng.standard_normal((res, res))) da = xr.DataArray(z, dims=[“y”, “x”], coords={“x”: lon, “y”: lat}) canvas5 = ds.Canvas(plot_width=700, plot_height=400, x_range=(-180, 180), y_range=(-90, 90)) agg_raster = canvas5.raster(da) fig, axes = plt.subplots(1, 2, figsize=(14, 4)) show(tf.shade(agg_raster, cmap=cc.CET_L18, how=”eq_hist”), “Synthetic elevation – eq_hist”, ax=axes[0]) show(tf.shade(agg_raster, cmap=cc.rainbow, how=”linear”), “Synthetic elevation – linear”, ax=axes[1]) plt.suptitle(“Section 5 – Raster / Grid (xarray DataArray)”, fontsize=13, fontweight=”bold”) plt.tight_layout() plt.show() print(“n=== SECTION 6: QuadMesh / 2-D Grid Glyph ===”) lon6 = np.concatenate([np.linspace(-180, -60, 80), np.linspace(-60, 60, 30), np.linspace( 60, 180, 80)]) lat6 = np.concatenate([np.linspace(-90, -30, 40), np.linspace(-30, 30, 20), np.linspace( 30, 90, 40)]) LON6, LAT6 = np.meshgrid(lon6, lat6) def vortex(lon0, lat0, amp=1.0): return amp * np.exp(-((LON6-lon0)**2/1200 + (LAT6-lat0)**2/600)) field6 = vortex(-40, 30, 1.2) + vortex(120, -20, 0.9) + 0.05 * rng.standard_normal(LON6.shape) da6 = xr.DataArray(field6.astype(np.float32), dims=[“y”, “x”], coords={“x”: lon6, “y”: lat6}, name=”intensity”) canvas6 = ds.Canvas(plot_width=700, plot_height=380, x_range=(-180, 180), y_range=(-90, 90)) agg6 = canvas6.quadmesh(da6) canvas6z = ds.Canvas(plot_width=500, plot_height=400, x_range=(-80, 0), y_range=(0, 60)) agg6z = canvas6z.quadmesh(da6) field6_smooth = vortex(-40, 30, 1.0) + vortex(120, -20, 0.8) da6_diff = xr.DataArray((field6 – field6_smooth).astype(np.float32), dims=[“y”,”x”], coords={“x”: lon6, “y”: lat6}, name=”anomaly”) agg6d = canvas6.quadmesh(da6_diff) fig, axes = plt.subplots(1, 3, figsize=(18, 5)) show(tf.shade(agg6, cmap=cc.fire, how=”eq_hist”), “Global field – eq_hist”, ax=axes[0]) show(tf.shade(agg6z, cmap=cc.CET_L3, how=”linear”), “N. Atlantic zoom – linear”, ax=axes[1]) show(tf.shade(agg6d, cmap=cc.CET_D4, how=”eq_hist”), “Residual (anomaly) – eq_hist”,ax=axes[2]) plt.suptitle(“Section 6 – canvas.quadmesh(): non-uniform 2-D grids”, fontsize=13, fontweight=”bold”) plt.tight_layout() plt.show() We move beyond point clouds and use Datashader to

A Coding Tutorial on Datashader on Rendering Massive Datasets with High-Performance Python Visual Analytics Beitrag lesen »

We use cookies to improve your experience and performance on our website. You can learn more at Datenschutzrichtlinie and manage your privacy settings by clicking Settings.

Privacy Preferences

You can choose your cookie settings by turning on/off each type of cookie as you wish, except for essential cookies.

Allow All
Manage Consent Preferences
  • Always Active

Save
de_DE