YouZum

Committee

AI, Committee, News, Uncategorized

Cartesia Ships Sonic-3.6: A Streaming TTS Model That Now Leads Both Artificial Analysis Speech Arenas

Cartesia has released Sonic-3.6, the newest version of its real-time text-to-speech model. It arrives roughly three months after Sonic-3.5. The new change is naturalness, and this one is independently checkable. Sonic 3.6 now holds #1 on both Artificial Analysis speech leaderboards — 1,283 Elo on the Provider Voice board and 1,123 on the Controlled Voice board. The second result matters more. That board clones every model onto the same eight reference voices, which isolates the synthesis engine from the voice catalog. Sonic-3.6 leads it, with Sonic-3.5 second and ElevenLabs Eleven v3 third. The model runs on state space models rather than transformers, and Cartesia states sub-90ms time-to-first-audio. It is available in beta. Is it deployable? YES, it is available in beta and as a hosted API. Not as self-hosted weights. Sonic is a closed, commercial model. There are no open weights and no Hugging Face repo. You rent it. Company level: Solo developers and startups (Free/Pro $5 tiers), scaleups running contact centers (Startup $49 / Scale $299), and regulated enterprises needing DPAs, BAAs, and SSO. Industries: Financial services, healthcare, retail and e-commerce, logistics, recruiting, SaaS support, consumer companion apps, media localization Applications: Inbound support agents, outbound qualification calls, IVR replacement, appointment reminders, sales-training simulators, audio localization, in-product voice UI The Architecture Sonic runs on state space models rather than transformers. Cartesia’s launch page frames the usual tradeoffs — speed versus naturalness, accuracy versus cost — as architectural, not inevitable. The practical output is time-to-first-audio. Cartesia states sub-90ms TTS latency, and 100ms transcript latency for its Ink-2 speech-to-text model. Both are vendor-stated model latency, not measured end-to-end round trips. Interactive explainer Features that matter in production Sonic exposes controls built for agent transcripts rather than narration: Inline expression tags. Non-verbal expressions like [laughter] go directly in the transcript. Instant voice cloning from about 10 seconds of audio. Custom pronunciation dictionaries, including IPA overrides such as <<s|ə|ˈ|p|i|n|ə>> for subpoena. Speed, volume, and emotion parameters exposed through the API and integrations like the LiveKit Agents plugin. Native alphanumerics. Order numbers, phone numbers, and confirmation codes read correctly without preprocessing. Cartesia’s launch demos show English with natural pauses and filler words, plus Hinglish code-switching between Hindi and English. Pricing reality Artificial Analysis normalizes Sonic 3.6 at $49.00 per 1M characters. That is half of ElevenLabs Eleven v3 at $100.00, and well above Speechify Simba 3.2 at $10.00 for a 1,240 Elo. Cartesia sells credits, not characters. Scale at $299 per month includes roughly 10,667 TTS minutes and 15 concurrent requests. Line voice agents bill separately at $0.06 per minute. Key Takeaways Sonic-3.6 is #1 on both Artificial Analysis speech arenas — 1,283 Elo Provider Voice, 1,123 Controlled Voice. Winning the Controlled board means the engine improved, not just the voice catalog. It is beta on Cartesia’s API only; docs still list Sonic 3.5 as stable, and partners carry 3.5. Deployable as a hosted API, not self-hosted weights; commercial use starts at the $5 Pro tier. Latency claims (sub-90ms TTFA) are vendor-stated model latency, so benchmark your own round trip. Check out the Project Page-Cartesia Sonic, Cartesia launch page, Cartesia pricing, Cartesia docs, Artificial Analysis Speech Arena and @cartesia on X. All figures verified August 18, 2026.. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Cartesia Ships Sonic-3.6: A Streaming TTS Model That Now Leads Both Artificial Analysis Speech Arenas appeared first on MarkTechPost.

Cartesia Ships Sonic-3.6: A Streaming TTS Model That Now Leads Both Artificial Analysis Speech Arenas Read Post »

AI, Committee, News, Uncategorized

The Download: how people really use AI, and Flock’s design choices

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. We still don’t know how people are really using AI AI companies like Anthropic and OpenAI regularly publish reports on how people are using their products. But they only release the data they want us to see, AI researchers say, and there’s no independent source to corroborate it. A new research project called the AI Observatory aims to fill in the gap. Its analysis shows many more sensitive behaviors than are captured in reports from major AI companies, which focus more on work than on personal use. The researchers also found significant differences between models. People were more likely to turn to Anthropic for coding, Gemini for social and roleplay uses, and ChatGPT for homework assistance. Here’s what the AI Observatory reveals about how people use AI. —Eileen Guo What Flock’s defenders are missing Flock Safety, the police-tech giant known for its network of some 120,000 automatic license plate readers around the US, recently announced changes to its platform. The updates are meant to prevent officers from using it for illegal or illegitimate purposes, including stalking. Amid all this, there have recently been several arguments defending Flock: If these cameras help solve crime, is it as big a deal? On a good day they might help catch a kidnapper, and if not, they’re simply snapping pictures of my car that nobody will bother to look at.  But this all skips over a more important question: What kind of crime-fighting system has Flock chosen to build? Its network works the way it does because of decisions about what information to collect, who can search it, how long to keep it, and how widely to share it. Those decisions set the terms of the bargain between security and civil liberties. Find out what a narrower Flock system could look like. —James O’Donnell This story is from The Algorithm, our weekly AI newsletter. Sign up to receive it in your inbox every Monday. The must-reads I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology. 1 A child privacy trial starting today could change Meta foreverMore than half of the states in the US have joined the lawsuit. (CNBC)+ They say Meta deliberately designed addictive social networks. (Guardian)+ And want changes including ending “like” counts and infinite scroll. (BBC)+ Four of them are demanding $1.4 trillion in damages. (WP $) 2 Nvidia has committed up to $105 billion to OpenAI’s Ohio data centerIt’s slated to cost up to $500 billion and come online in 2028. (CNBC)+ OpenAI will lease the eight-gigawatt site for 20 years. (NYT $)+ How virtual power plants could provide energy for data centers. (MIT Technology Review) 3 Tesla is set to launch Cybercab robotaxi rides in Austin this monthThe rollout could begin with employee rides on public roads. (Information $)+ The EVs will then enter Tesla’s robotaxi service ‌a few days later. (Reuters $)+ The Tesla Semi could also be a big deal for EVs. (MIT Technology Review) 4 Unitree has unveiled a humanoid it claims is faster than any humanThe new “Superman” robot can cover 12.66 meters in a second. (Gizmodo)+ It arrives ahead of the Chinese company’s IPO on Wednesday. (Reuters $)+ US robot startups are struggling with new China restrictions. (Rest of World) 5 A tracked rare book shipment led to Amazon’s AI training operationAmazon scans and destroys books at the facility to train AI. (404 Media $)+ Rare books provide unique, high-quality training data. (Ars Technica)+ But feeding them to AI risks creating a cultural void. (New Scientist $) 6 China wants its data to shape what the world’s AI knowsBeijing is distributing datasets reflecting “mainstream Chinese values.” (NYT $)+ What’s next for Chinese open-source AI? (MIT Technology Review) 7 An AI studio wants to become the HBO of adult contentRogue’s AI video tool produces uncensored adult content. (Wired $)+ AI is creating new risks for porn actors. (MIT Technology Review) 8 Women are being left behind in the AI jobs boomThey accounted for just 26% of new AI hires last year. (Axios) 9 AI has revealed new clues to how breast cancer progressesThe findings could help predict how tumours will develop. (Independent) 10 An unearthed video suggests Apple is adding cameras to AirPodsIt shows AirPods identifying a book using visual intelligence. (Gizmodo) Quote of the day “It is a black hole; it absorbs energy and personality and then re-presents it as spectacle.”  —Writer Carmen Hermosillo made a prescient observation about life online in a 1994 essay, quoted by the Guardian in a story on warnings we ignored about the digital age. One More Thing TERRY RATZLAFF Is this the electric grid of the future? When a slow-moving blizzard hit Nebraska, nearly 10% of Lincoln Electric System’s 150,000 customers lost power. CEO Emeka Anyanwu watched the outage map as crews battled the storm. Yet a spring blizzard like this is the least of his problems. What will happen soon—not only at Lincoln Electric but for all electric utilities—is a challenge of a different order. In the industry, they call it the “trilemma”: the seemingly intractable problem of balancing reliability, affordability, and sustainability. Electricity demand is surging, driven in part by AI, while the industry attempts to transition from power generated with fossil fuels to power generated from renewable sources like solar and wind. Lincoln Electric offers a lens through which to examine those challenges. Explore the challenges of building the grid of the future. —Andrew Blum We can still have nice things A place for comfort, fun, and distraction to brighten up your day. (Got any ideas? Drop me a line.) + Chinese engineers have built the world’s first true all-material printer.+ Discover how you truly feel about artificial intelligence by completing the AI Compass quiz.+ Get a sense of the space’s extraordinary size with this interactive scale of the universe tool.+ Enjoy a fresh look at “Breaking Bad” in this modified scene showing

The Download: how people really use AI, and Flock’s design choices Read Post »

AI, Committee, News, Uncategorized

Meet SAM (Sovereign Agent Mesh): A Zero-Config, Zero-Trust P2P Network for AI Agents

google/sam is not Segment Anything. SAM here means Sovereign Agent Mesh, an Apache-2.0 networking project for autonomous AI agents. The problem it targets is concrete. Agents now run across cloud servers, on-prem datacenters, laptops, Raspberry Pis and Android devices. Letting them share tools usually means exposing internal scripts, LLM endpoints or private APIs to the public internet. SAM’s alternative is a zero-config, zero-trust P2P overlay — closer to a private VPN, but scoped to agent-to-agent tool sharing over the Model Context Protocol. Nodes discover each other automatically, survive NAT, and authorize every call cryptographically. Note: The repo carries an explicit disclaimer: this is not an officially supported Google product. Is it deployable? Partially, the engineering is production-shaped, but the public mesh is still labelled a beta testnet. What ships now: Go binaries, an install script, ghcr.io Docker images, a charts/sam-mesh Helm chart, a production Kubernetes guide, and Android/iOS support. The public testnet is bananas.sam-mesh.dev. For real workloads, self-host your control plane. The docs call this “DIY Mode” and it is the path to full data and policy control. Company level: Best fit is mid-market and enterprise engineering orgs running agents across more than one network boundary. Startups inside a single VPC gain less; the value shows up once agents span cloud, datacenter and laptops. Industries: Financial services, healthcare, public sector and defense, and industrial or robotics edge fleets. Broadly, any regulated org that cannot publish internal tools to the internet. Applications: Cross-cloud MCP tool sharing, hybrid on-prem to cloud agent calls, brokered inference endpoints, sandboxed agents with credential injection, and pooled warm workers. Architecture: three binaries sam-control-plane — identity registration, token issuing, policy distribution. sam-router — libp2p bootstrap points and GossipSub routing overlays. sam-node — the P2P client providing mesh transport, self-healing connectivity, and a local MCP HTTP interface. A node joins with sam-node join, then runs with sam-node run. libp2p uses 5001/udp and 5002/tcp; the local MCP API defaults to 8080. Identity: OIDC in, Biscuit out This is the interesting part. The control plane verifies an OIDC JWT. It then translates the claims into Datalog facts and seals them into a Biscuit token. sub becomes user(…), each group becomes group(…), and the peer ID binds in as client_peer_id(…). The consequence: nodes authorize offline. A node evaluates the presented token against its own local rules without calling home. Enforcement is strict default-deny. Access needs an explicit capability fact such as granted_service_exact(…). There are no built-in exceptions — even the discovery catalog system://sam.catalog must be granted. Services use a strict type://name convention with wildcard support (mcp://*, mcp://build-runner.*). Every request runs a two-stage pipeline. Stage 1 gates the connection against ban and revocation caches. Stage 2 runs exactly two Biscuit authorizer passes. The first covers the node’s own identity token to emit target_fact assertions. The second covers the caller’s token. A baseline check blocks replay by requiring the connection peer ID to match the token. Operators can attenuate locally, denying a write tool after 9 PM or blocking contractors. Local allows still cannot bypass control-plane check if constraints. Interactive explainer What an agent actually calls The node exposes standard MCP tools: discover_remote_services, find_remote_tools, and call_remote_tool. Guides cover Gemini, Claude Code, Claude Desktop, Google Antigravity and OpenClaw. sam-node skill install writes a SKILL.md so an agent can bring the node online itself. The enrollment login stays with a human by design. Egress control: sam-box and nano-init The Secure Outbound Gateway targets a real agent-security gap. nano-init runs as PID 1 in the sandbox and sets the proxy environment variables. For tools that ignore them, it LD_PRELOADs an interceptor hooking the C connect() syscall on ports 80 and 443. Traffic reaches sam-box over a Unix domain socket. The gateway verifies the Biscuit, injects the real credential from secrets.yaml, and upgrades the request to HTTPS. The agent sandbox never holds the key. A worked pattern: warm agent pool The code-reviewer pool example fans batch work across identical running workers using ordinary MCP services. A manager learns peers via DHT discovery and tracks busy state with leases. Correctness comes from synchronous lease assignment, fencing tokens, grace eviction, and a POOL_BUSY backstop. Workers verify a short-lived HMAC token offline; anything else returns NO_LEASE. Key Takeaways SAM is Sovereign Agent Mesh, an Apache-2.0 P2P overlay — not Segment Anything. Three binaries: control plane for identity and policy, routers for libp2p transport, nodes for MCP. OIDC claims are translated into Biscuit Datalog facts, so nodes authorize offline. Default-deny is absolute; even the discovery catalog needs an explicit grant. Production use means self-hosting a control plane — the public mesh is a beta testnet. Check out the GitHub Repo. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Meet SAM (Sovereign Agent Mesh): A Zero-Config, Zero-Trust P2P Network for AI Agents appeared first on MarkTechPost.

Meet SAM (Sovereign Agent Mesh): A Zero-Config, Zero-Trust P2P Network for AI Agents Read Post »

AI, Committee, News, Uncategorized

NVIDIA Releases TensorRT Model Connect in Public Preview: Hugging Face Checkpoint to Native C++ Inference in Two Commands

NVIDIA has released TensorRT Model Connect (TRTMC) in public preview, an open-source project that takes a supported Hugging Face or local checkpoint to end-to-end TensorRT inference in two commands. There is no intermediate ONNX export step. The build produces a versioned .bundle artifact that runs through native C++ task APIs, so inference can execute in a C++ service, embedded application, or robotics stack without PyTorch in the runtime path. The project is Apache-2.0 licensed and ships as a collection of family-owned reference implementations rather than a single generic converter. NVIDIA also states that the entire project — model implementations, performance tuning, tests, integrations, and docs — was built using OpenAI Codex agents under human direction and review. Is it deployable? Yes, for evaluation and native integration work, with real conditions. The code is open and installable. Release wheels currently target Linux aarch64 only, with Python 3.10 or 3.12, glibc 2.39 or newer, and TensorRT 11.1.0.106. x86_64 wheels are not published; x86_64 users must take the Docker source-build path. Company level: Best fit today is teams that already own their inference stack: NVIDIA-shop startups, robotics and device companies, and platform or inference teams inside mid-size and large enterprises. Small teams shipping a Python service get less from it. Regulated enterprises should wait for a tagged release before standardizing on it. Industries: Robotics and autonomous machines, industrial inspection and manufacturing, automotive in-vehicle compute, medical devices, defense and aerospace edge systems, and media processing — anywhere inference has to live inside a C++ binary rather than a Python server. Applications: On-device text generation, speech recognition and synthesis, OCR and document parsing, embeddings and reranking for a retrieval service written in C++, diffusion image and video generation, segmentation, and time-series forecasting. The two commands The quick start builds and runs Qwen3-0.6B: trtmc build Qwen/Qwen3-0.6B –precision bf16 –max-cache-length 16384 –output qwen3-0.6b.bundle trtmc run ./qwen3-0.6b.bundle –prompt “What is the capital of France? Answer in one word.” –chat-template –no-thinking The same .bundle loads from C++ with trtmc::load(“./qwen3-0.6b.bundle”). The bundle is the actual design decision TRTMC splits build and runtime at a versioned artifact. Python owns checkpoint resolution and TensorRT engine construction. Native profiles then execute inference in C++ without PyTorch. A small number of hybrid profiles invoke a helper Python executable, and their manifests declare that dependency explicitly. Applications call task APIs — generate(), transcribe(), generate_image(), embed(), solve() — instead of maintaining conversion stages and per-model application glue. trtmc inspect exposes bundle kind, model family, precision, runtime identity, and engines, which makes the artifact auditable rather than opaque. NVIDIA frames the conventional route as PyTorch → ONNX or TorchScript → TensorRT → model-specific C++ integration, and names the failure modes it removes: export gaps, repeated per-model integration, and validation spread across several conversion artifacts. Key Takeaways Two commands take a supported Hugging Face checkpoint to native C++ TensorRT inference, with no ONNX step. A versioned .bundle is the handoff between the Python build and a PyTorch-free C++ runtime. The July 29, 2026 GB300 snapshot covers 105 profiles across 76 families; 102 beat their declared reference by more than 5%. Wheels are Linux aarch64 only today; x86_64 requires the Docker source build. Check out the GitHub Repo. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post NVIDIA Releases TensorRT Model Connect in Public Preview: Hugging Face Checkpoint to Native C++ Inference in Two Commands appeared first on MarkTechPost.

NVIDIA Releases TensorRT Model Connect in Public Preview: Hugging Face Checkpoint to Native C++ Inference in Two Commands Read Post »

AI, Committee, News, Uncategorized

The Download: dead robot friends and the “censorship-industrial complex”

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. What happens when a kid’s robot best friend dies? When Xander first met Moxie, she taught him how to calm down when he was anxious or mad. Six years later, she mostly watches him play Minecraft and talks to him about his stuffed animals. Moxie is a robot—a 15-inch-tall device that looks a bit like a blue, legless astronaut. It belongs to a subset of robots designed to assist neurodivergent children by providing connection and helping kids practice social skills usually learned from therapists.  Xander still uses Moxie when he feels like he needs someone to talk to, but the device has had a troubled life. The robot’s maker went out of business, its servers were shut down, and parents rushed to convert their Moxies before the servers went offline.  Read our story about how it exposes the promise and pitfalls of AI companion toys for children. —Sara Harrison This article is from the next issue of our print magazine, which is all about kids. Subscribe now to read it when it lands. Inside the “censorship-industrial complex” idea shaping US policy The idea of a “censorship-industrial complex” has moved from the fringes of right-wing online discourse into US policy. The basic theory is that, under the guise of combating disinformation, government agencies, academics, civil society groups, and Big Tech platforms have worked together to suppress conservative and populist speech online. It has now made its way into the Trump administration. Over the past nine months, MIT Technology Review investigated its rise. In a recent Roundtables session, senior reporter Eileen Guo and executive editor Amy Nordrum revealed what they discovered, where the theory is going, and what it could mean for the future of democracy and the internet. Subscribers and MIT alumni can now watch the full event here. The must-reads I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology. 1 The US plans to force partners to pick sides in the AI race A draft letter warns allies against joining China’s rival AI initiative. (Reuters $)+ Beijing is using open-weight AI to expand its governance. (FT $)+ Chinese AI has divided the White House. (MIT Technology Review) 2 Memory chipmaker CXMT is now China’s most valuable companyIts rise reflects Beijing’s push for strategic hardware. (Bloomberg $)+ The US has urged Apple not to buy Chinese memory chips. (WSJ $) 3 Astronomers have found evidence for a “black hole star”It’s 100,000 times bigger than the Sun. (Futurism)+ And may explain mysterious red spots in the early universe. (Wired $) 4 Meta has patented facial recognition for AI glasses to identify peopleAnd make highlight reels of dinner parties, apparently. (404 Media)+ Meta’s “pervert glasses” issue is killing an impressive product. (Independent) 5 Puerto Rico is rationing water. It could’ve harvested rainwater insteadUp to 30 billion gallons could be collected annually. (Wired $)+ Puerto Rico is enduring major power struggles. (MIT Technology Review) 6 The backlash against Flock could become a broader tech rebellionPeople are sick of tech billionaires trying to control their lives. (Salon)+ Flock is tightening its rules following the backlash. (MIT Technology Review) 7 Amazon is trying to crush class-action suits before they get startedA terms update requires all disputes to be resolved via arbitration. (Verge) 8 Aging may be a programmed process, not just wear and tearScientists found distinct stages of aging across mouse cells. (Quanta)+ Scientists created female clones of male mice. (MIT Technology Review) 9 The latest online shopping trend involves buying nothing“Dopamine sites” recreate shopping rituals without the buying. (Rest of World) 10 Ice cream is becoming a surprisingly high-tech businessAI, robotics, and unusual flavors are changing how it’s made. (BBC) Quote of the day “The thing that will work is actually curing cancer.” —Anthropic CEO Dario Amodei proposes a way to win over AI skeptics in a rare post on X. One More Thing The quest to find out how our bodies react to extreme temperatures Climate change is forcing us to reckon with the knotty science of how our bodies interact with the environment.  As extreme temperatures become more common, scientists are trying to understand what happens when heat and cold push our bodies toward their limits. But the science of keeping warm or cool is surprisingly full of blind spots.  Researchers are finding that long-held assumptions about how our bodies respond to extreme temperatures may be more complicated than we thought. Find out what they’re learning about the limits of the human body. —Max G. Levy We can still have nice things A place for comfort, fun, and distraction to brighten up your day. (Got any ideas? Drop me a line.) + Trace the enthralling history of sumo wrestling in this succinct animated documentary.+ A meow-sical pet owner (sorry) has sampled his cat’s purrs into a booming bassline.+ The true-scale Universe Atlas lets you zoom from inside a proton out to the cosmic web.+ This selection of places that only exist at certain times of day is a goldmine for punctual travelers. 

The Download: dead robot friends and the “censorship-industrial complex” Read Post »

AI, Committee, News, Uncategorized

Developing an End-to-End Document Intelligence Pipeline with docTR for OCR, Layout Analysis, KIE, Benchmarking, and Searchable PDFs

In this tutorial, we develop an end-to-end OCR workflow with docTR and explore how modern document understanding pipelines combine text detection, recognition, geometry, layout analysis, structured extraction, and export. We generate realistic synthetic invoice documents, load images and PDFs through DocumentFile, construct GPU-aware OCR predictors, and benchmark different detection–recognition architecture combinations for speed and accuracy. We then inspect the internal Document hierarchy, visualize confidence-aware bounding boxes, use standalone detection and recognition models, implement two-pass recognition for low-confidence words, tune detection thresholds, and introduce custom pipeline hooks for box filtering and padding. We also handle rotated and skewed documents, experiment with layout detection and KIE, reconstruct reading order and tabular information, extract structured invoice fields, and export results as text, JSON, hOCR, synthesized document images, and searchable PDFs. Finally, we examine practical performance, fine-tuning, batching, and deployment considerations to understand how to move from a basic OCR example to a production-oriented document intelligence pipeline. Copy CodeCopiedUse a different Browser import os, sys, io, json, time, math, re, subprocess, warnings from collections import Counter, defaultdict warnings.filterwarnings(“ignore”) os.environ.setdefault(“USE_TORCH”, “1”) def _pip(*pkgs): subprocess.run([sys.executable, “-m”, “pip”, “install”, “-q”, *pkgs], check=False) try: import doctr except ImportError: print(“>> Installing python-doctr (this takes ~1-2 min on Colab)…”) _pip(“python-doctr[viz]”) try: import reportlab except ImportError: _pip(“reportlab”) import numpy as np import torch import matplotlib import matplotlib.pyplot as plt from matplotlib import font_manager from matplotlib.patches import Rectangle, Polygon as MplPolygon from PIL import Image, ImageDraw, ImageFont import doctr from doctr.io import DocumentFile from doctr.models import ( ocr_predictor, kie_predictor, detection_predictor, recognition_predictor, ) DEVICE = “cuda” if torch.cuda.is_available() else “cpu” print(“=” * 78) print(f”docTR : {doctr.__version__}”) print(f”torch : {torch.__version__}”) print(f”device : {DEVICE}” + (f” ({torch.cuda.get_device_name(0)})” if DEVICE == “cuda” else “”)) print(f”python : {sys.version.split()[0]}”) print(“=” * 78) print(“NOTE: if the import above failed, restart the runtime ” “(Runtime > Restart session) and re-run this cell.n”) CFG = dict( RUN_BENCHMARK = True, RUN_SECOND_PASS = True, RUN_ROTATION = True, RUN_LAYOUT = True, RUN_KIE = True, RUN_SYNTHESIS = True, RUN_PDF_EXPORT = True, ) WORK = “/content/doctr_demo” if os.path.isdir(“/content”) else “./doctr_demo” os.makedirs(WORK, exist_ok=True) print(f”working dir: {WORK}n”) _FONT = font_manager.findfont(font_manager.FontProperties(family=”DejaVu Sans”)) _FONT_B = font_manager.findfont( font_manager.FontProperties(family=”DejaVu Sans”, weight=”bold”)) A4 = (1240, 1754) INVOICE_LINES = [ ( 80, 70, “NORTHWIND TRADING CO.”, 38, True ), ( 80, 122, “42 Harbour Road, Bristol BS1 5TY”, 22, False), ( 80, 152, “VAT GB 884 5521 09”, 22, False), (820, 70, “INVOICE”, 44, True ), (820, 132, “Invoice No: INV-2024-00817”, 22, False), (820, 162, “Date: 14/03/2024”, 22, False), (820, 192, “Due Date: 13/04/2024”, 22, False), ( 80, 260, “BILL TO”, 24, True ), ( 80, 296, “Aurora Robotics Ltd”, 24, False), ( 80, 328, “Unit 7 Fenway Business Park”, 22, False), ( 80, 358, “Cambridge CB4 0WS”, 22, False), ( 80, 388, “Contact: procurement@aurorarobotics.co.uk”,22, False), ( 80, 470, “DESCRIPTION”, 24, True ), (640, 470, “QTY”, 24, True ), (780, 470, “UNIT PRICE”, 24, True ), (1010,470, “AMOUNT”, 24, True ), ( 80, 520, “Servo controller board Rev C”, 22, False), (640, 520, “12”, 22, False), (780, 520, “84.50”, 22, False), (1010,520, “1014.00”, 22, False), ( 80, 560, “Harmonic drive gearbox 50:1”, 22, False), (640, 560, “4”, 22, False), (780, 560, “312.75”, 22, False), (1010,560, “1251.00”, 22, False), ( 80, 600, “Shielded encoder cable 2m”, 22, False), (640, 600, “20”, 22, False), (780, 600, “11.40”, 22, False), (1010,600, “228.00”, 22, False), ( 80, 640, “Calibration service on-site”, 22, False), (640, 640, “1”, 22, False), (780, 640, “450.00”, 22, False), (1010,640, “450.00”, 22, False), (780, 720, “Subtotal”, 22, False), (1010,720, “2943.00”, 22, False), (780, 756, “VAT 20%”, 22, False), (1010,756, “588.60”, 22, False), (780, 796, “TOTAL DUE”, 26, True ), (1010,796, “3531.60”, 26, True ), ( 80, 900, “PAYMENT TERMS”, 24, True ), ( 80, 936, “Net 30 days. Late payments accrue interest at 2% per month.”, 20, False), ( 80, 968, “Bank: Lloyds Sort Code: 30-96-26 Account: 41775302”, 20, False), ( 80,1010, “Reference: INV-2024-00817”, 20, False), ] PAGE2_LINES = [ ( 80, 70, “APPENDIX A – DELIVERY SCHEDULE”, 34, True ), ( 80, 140, “All shipments leave the Bristol warehouse before 16:00 GMT.”, 22, False), ( 80, 176, “Tracking numbers are emailed on the day of dispatch.”, 22, False), ( 80, 240, “MILESTONE”, 24, True ), (700, 240, “TARGET DATE”, 24, True ), ( 80, 288, “Purchase order acknowledged”, 22, False), (700, 288, “18/03/2024”, 22, False), ( 80, 328, “Controller boards shipped”, 22, False), (700, 328, “25/03/2024”, 22, False), ( 80, 368, “Gearboxes shipped”, 22, False), (700, 368, “02/04/2024”, 22, False), ( 80, 408, “On-site calibration window”, 22, False), (700, 408, “08/04/2024”, 22, False), ( 80, 480, “Questions? Call +44 117 496 0022 or email ops@northwind.example”, 20, False), ] def render_page(lines, size=A4, bg=250): “””Draw a clean document page from a list of (x, y, text, size, bold).””” img = Image.new(“RGB”, size, (bg, bg, bg)) d = ImageDraw.Draw(img) for x, y, text, sz, bold in lines: font = ImageFont.truetype(_FONT_B if bold else _FONT, sz) d.text((x, y), text, fill=(18, 18, 22), font=font) d.line([(80, 455), (1160, 455)], fill=(60, 60, 60), width=2) d.line([(80, 505), (1160, 505)], fill=(160, 160, 160), width=1) d.line([(760, 700), (1160, 700)], fill=(60, 60, 60), width=2) return img def scanify(img, angle=0.0, noise=6.0, jpeg_quality=72, blur_shadow=True): “””Degrade a clean render so it behaves like a phone photo / flatbed scan.””” if angle: img = img.rotate(angle, expand=True, resample=Image.BICUBIC, fillcolor=(250, 250, 250)) arr = np.asarray(img).astype(np.float32) if blur_shadow: h, w = arr.shape[:2] gx = np.linspace(-1, 1, w)[None, :] gy = np.linspace(-1, 1, h)[:, None] shade = 1.0 – 0.10 * (gx ** 2 + 0.6 * gy ** 2) arr *= shade[…, None] if noise: arr += np.random.normal(0, noise, arr.shape) arr = np.clip(arr, 0, 255).astype(np.uint8) out = Image.fromarray(arr) if jpeg_quality: buf = io.BytesIO() out.save(buf, format=”JPEG”, quality=jpeg_quality) buf.seek(0) out = Image.open(buf).convert(“RGB”) return out clean1 = render_page(INVOICE_LINES) clean2 = render_page(PAGE2_LINES) page1_path = os.path.join(WORK, “invoice_p1.png”) page2_path = os.path.join(WORK, “invoice_p2.png”) rotated_path = os.path.join(WORK, “invoice_rotated.png”) pdf_path = os.path.join(WORK, “invoice.pdf”) scanify(clean1, angle=0.4).save(page1_path) scanify(clean2, angle=-0.3).save(page2_path) scanify(clean1, angle=13.0, noise=8.0).save(rotated_path) clean1.save(pdf_path, save_all=True, append_images=[clean2], resolution=150) GT_WORDS_P1 = [w for _, _, t, _, _ in INVOICE_LINES for w in t.split()] print(f”generated: {page1_path}, {page2_path}, {rotated_path}, {pdf_path}”) print(f”ground-truth words on page 1:

Developing an End-to-End Document Intelligence Pipeline with docTR for OCR, Layout Analysis, KIE, Benchmarking, and Searchable PDFs Read Post »

AI, Committee, News, Uncategorized

MiniMax Releases MiniMax-Music3: An Open-Weights Music Model Generating Complete Five-Minute Songs From Lyrics and a Structured Caption

MiniMax released MiniMax-Music3, an open-weights text-to-music model. The model takes two separate inputs: lyrics carrying section tags, and a detailed music description. It returns a complete song of up to five minutes in a single generation, as 32 kHz, 16-bit stereo WAV. The architecture pairs a Hybrid-LM, an 8B Global LLM with a 0.6B Local LLM, with a continuous synthesis stack built on flow matching and a Flow-VAE. Weights, inference code and three documented serving paths shipped the same day. Is it deployable? Yes, MiniMax published usable weights, inference code and three documented serving paths on day one, so this is deployable now rather than a research preview. Company level: Solo creators, indie studios and mid-market teams can ship on it directly. The MiniMax-Music3 Community License permits commercial use, but it requires you to display ‘MiniMax-Music3’ prominently in the product UI, and any organization whose aggregate yearly revenue from those products exceeds US$ 20 million must obtain separate prior written authorization from MiniMax. Anyone hosting third-party generation must also implement and maintain safeguards against infringing outputs. Industries: Game development, advertising and brand agencies, short-form video and creator tools, e-learning, podcasting, fitness and wellness apps, retail in-store audio, and music-tech SaaS. Applications: Background scoring for UGC video, adaptive game and level music, localized ad beds and sonic branding, scratch and demo tracks for songwriters, mood-conditioned playlist generation, and offline batch generation where per-song API cost is the constraint. The Architecture MiniMax-Music3 combines a hierarchical autoregressive stack with a continuous synthesis path. The training tokenizer uses eight layers of residual vector quantization (RVQ). The first, semantic codebook has 16,384 entries and carries core musical semantics and structure. The remaining seven acoustic codebooks have 1,024 entries each and encode residual detail. Training optimizes the semantic layer first, then all eight jointly. The Hybrid-LM splits the modeling problem. An 8B Global LLM predicts the first RVQ codebook frame by frame and holds long-range structure; a 0.6B Local LLM predicts the remaining codebooks within each frame. The model card and license state the Global LLM is initialized from Qwen3-8B; the MiniMax Research post says Qwen3.5-8B, so treat the exact base checkpoint as unsettled. The synthesis stage is the more interesting design choice. Rather than decoding from discrete RVQ tokens, MiniMax fuses the final hidden states of both LLMs and conditions a 2.4B flow-matching module on them, which maps into a latent space decoded by a 123M Flow-VAE inherited from MiniMax Speech. At inference the discrete tokenizer decoder is not loaded at all. Two-input control Lyrics carry the words and section tags on their own lines: [Intro], [Verse], [Pre-Chorus], [Chorus], [Post-Chorus], [Bridge], [Instrumental], [Solo], [Outro]. A separate Structured Caption carries Global Metadata, Vocal Details and Arrangement. MiniMax also ships a music-caption-rewriter agent skill that expands a short description into that three-part format offline. Interactive explainer Running it Three documented paths. SGLang-Omni is the reference server; the GitHub page specifies two CUDA GPUs, with GPU 0 running Qwen3 and RVQ autoregressive generation and GPU 1 running flow matching and DAV decoding. The diffusers modular pipeline fits under 24 GB VRAM at full precision, about 22 GB with automatic CPU offload, and down to 8 GB with leaf-level group offloading. ComfyUI has a native Text to Music template using repacked FP16/INT8 weights from Comfy-Org. Key Takeaways Open-weights model generating complete five-minute songs at 32 kHz, 16-bit stereo, released August 13, 2026. Hybrid-LM design: 8B Global LLM plus 0.6B Local LLM, feeding 2.4B flow matching and a 123M Flow-VAE. Synthesis runs on fused continuous hidden states, skipping the discrete tokenizer decoder entirely. Runs on two GPUs via SGLang-Omni, under 24 GB via diffusers, or 8 GB with group offloading. Commercial use allowed with visible attribution; above USD 20M revenue needs written authorization. Check out the Model on HF and GitHub Repo. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post MiniMax Releases MiniMax-Music3: An Open-Weights Music Model Generating Complete Five-Minute Songs From Lyrics and a Structured Caption appeared first on MarkTechPost.

MiniMax Releases MiniMax-Music3: An Open-Weights Music Model Generating Complete Five-Minute Songs From Lyrics and a Structured Caption Read Post »

We use cookies to improve your experience and performance on our website. You can learn more at Privacy Policy and manage your privacy settings by clicking Settings.

Privacy Preferences

You can choose your cookie settings by turning on/off each type of cookie as you wish, except for essential cookies.

Allow All
Manage Consent Preferences
  • Always Active

Save
en_US