Committee

MIT Technology Review Explains: Let our writers untangle the complex, messy world of technology to help you understand what’s coming next. You can read more from the series here. It’s been a big year for video generation. In the last nine months OpenAI made Sora public, Google DeepMind launched Veo 3, the video startup Runway launched Gen-4. All can produce video clips that are (almost) impossible to distinguish from actual filmed footage or CGI animation. This year also saw Netflix debut an AI visual effect in its show The Eternaut, the first time video generation has been used to make mass-market TV. Sure, the clips you see in demo reels are cherry-picked to showcase a company’s models at the top of their game. But with the technology in the hands of more users than ever before—Sora and Veo 3 are available in the ChatGPT and Gemini apps for paying subscribers—even the most casual filmmaker can now knock out something remarkable.  The downside is that creators are competing with AI slop, and social media feeds are filling up with faked news footage. Video generation also uses up a huge amount of energy, many times more than text or image generation.  With AI-generated videos everywhere, let’s take a moment to talk about the tech that makes them work. How do you generate a video? Let’s assume you’re a casual user. There are now a range of high-end tools that allow pro video makers to insert video generation models into their workflows. But most people will use this technology in an app or via a website. You know the drill: “Hey, Gemini, make me a video of a unicorn eating spaghetti. Now make its horn take off like a rocket.” What you get back will be hit or miss, and you’ll typically need to ask the model to take another pass or 10 before you get more or less what you wanted.  So what’s going on under the hood? Why is it hit or miss—and why does it take so much energy? The latest wave of video generation models are what’s known as latent diffusion transformers. Yes, that’s quite a mouthful. Let’s unpack each part in turn, starting with diffusion.  What’s a diffusion model? Imagine taking an image and adding a random spattering of pixels to it. Take that pixel-spattered image and spatter it again and then again. Do that enough times and you will have turned the initial image into a random mess of pixels, like static on an old TV set.  A diffusion model is a neural network trained to reverse that process, turning random static into images. During training, it gets shown millions of images in various stages of pixelation. It learns how those images change each time new pixels are thrown at them and, thus, how to undo those changes.  The upshot is that when you ask a diffusion model to generate an image, it will start off with a random mess of pixels and step by step turn that mess into an image that is more or less similar to images in its training set.  But you don’t want any image—you want the image you specified, typically with a text prompt. And so the diffusion model is paired with a second model—such as a large language model (LLM) trained to match images with text descriptions—that guides each step of the cleanup process, pushing the diffusion model toward images that the large language model considers a good match to the prompt.  An aside: This LLM isn’t pulling the links between text and images out of thin air. Most text-to-image and text-to-video models today are trained on large data sets that contain billions of pairings of text and images or text and video scraped from the internet (a practice many creators are very unhappy about). This means that what you get from such models is a distillation of the world as it’s represented online, distorted by prejudice (and pornography). It’s easiest to imagine diffusion models working with images. But the technique can be used with many kinds of data, including audio and video. To generate movie clips, a diffusion model must clean up sequences of images—the consecutive frames of a video—instead of just one image.  What’s a latent diffusion model?  All this takes a huge amount of compute (read: energy). That’s why most diffusion models used for video generation use a technique called latent diffusion. Instead of processing raw data—the millions of pixels in each video frame—the model works in what’s known as a latent space, in which the video frames (and text prompt) are compressed into a mathematical code that captures just the essential features of the data and throws out the rest.  A similar thing happens whenever you stream a video over the internet: A video is sent from a server to your screen in a compressed format to make it get to you faster, and when it arrives, your computer or TV will convert it back into a watchable video.  And so the final step is to decompress what the latent diffusion process has come up with. Once the compressed frames of random static have been turned into the compressed frames of a video that the LLM guide considers a good match for the user’s prompt, the compressed video gets converted into something you can watch.   With latent diffusion, the diffusion process works more or less the way it would for an image. The difference is that the pixelated video frames are now mathematical encodings of those frames rather than the frames themselves. This makes latent diffusion far more efficient than a typical diffusion model. (Even so, video generation still uses more energy than image or text generation. There’s just an eye-popping amount of computation involved.)  What’s a latent diffusion transformer? Still with me? There’s one more piece to the puzzle—and that’s how to make sure the diffusion process produces a sequence of frames that are consistent, maintaining objects and lighting and so on from one frame

In this tutorial, we build an Advanced OCR AI Agent in Google Colab using EasyOCR, OpenCV, and Pillow, running fully offline with GPU acceleration. The agent includes a preprocessing pipeline with contrast enhancement (CLAHE), denoising, sharpening, and adaptive thresholding to improve recognition accuracy. Beyond basic OCR, we filter results by confidence, generate text statistics, and perform pattern detection (emails, URLs, dates, phone numbers) along with simple language hints. The design also supports batch processing, visualization with bounding boxes, and structured exports for flexible usage. Check out the FULL CODES here. Copy CodeCopiedUse a different Browser !pip install easyocr opencv-python pillow matplotlib import easyocr import cv2 import numpy as np from PIL import Image, ImageEnhance, ImageFilter import matplotlib.pyplot as plt import os import json from typing import List, Dict, Tuple, Optional import re from google.colab import files import io We start by installing the required libraries, EasyOCR, OpenCV, Pillow, and Matplotlib, to set up our environment. We then import all necessary modules so we can handle image preprocessing, OCR, visualization, and file operations seamlessly. Check out the FULL CODES here. Copy CodeCopiedUse a different Browser class AdvancedOCRAgent: “”” Advanced OCR AI Agent with preprocessing, multi-language support, and intelligent text extraction capabilities. “”” def __init__(self, languages: List[str] = [‘en’], gpu: bool = True): “””Initialize OCR agent with specified languages.””” print(” Initializing Advanced OCR Agent…”) self.languages = languages self.reader = easyocr.Reader(languages, gpu=gpu) self.confidence_threshold = 0.5 print(f” OCR Agent ready! Languages: {languages}”) def upload_image(self) -> Optional[str]: “””Upload image file through Colab interface.””” print(” Upload your image file:”) uploaded = files.upload() if uploaded: filename = list(uploaded.keys())[0] print(f” Uploaded: {filename}”) return filename return None def preprocess_image(self, image: np.ndarray, enhance: bool = True) -> np.ndarray: “””Advanced image preprocessing for better OCR accuracy.””” if len(image.shape) == 3: gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) else: gray = image.copy() if enhance: clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8,8)) gray = clahe.apply(gray) gray = cv2.fastNlMeansDenoising(gray) kernel = np.array([[-1,-1,-1], [-1,9,-1], [-1,-1,-1]]) gray = cv2.filter2D(gray, -1, kernel) binary = cv2.adaptiveThreshold( gray, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 11, 2 ) return binary def extract_text(self, image_path: str, preprocess: bool = True) -> Dict: “””Extract text from image with advanced processing.””” print(f” Processing image: {image_path}”) image = cv2.imread(image_path) if image is None: raise ValueError(f”Could not load image: {image_path}”) if preprocess: processed_image = self.preprocess_image(image) else: processed_image = image results = self.reader.readtext(processed_image) extracted_data = { ‘raw_results’: results, ‘filtered_results’: [], ‘full_text’: ”, ‘confidence_stats’: {}, ‘word_count’: 0, ‘line_count’: 0 } high_confidence_text = [] confidences = [] for (bbox, text, confidence) in results: if confidence >= self.confidence_threshold: extracted_data[‘filtered_results’].append({ ‘text’: text, ‘confidence’: confidence, ‘bbox’: bbox }) high_confidence_text.append(text) confidences.append(confidence) extracted_data[‘full_text’] = ‘ ‘.join(high_confidence_text) extracted_data[‘word_count’] = len(extracted_data[‘full_text’].split()) extracted_data[‘line_count’] = len(high_confidence_text) if confidences: extracted_data[‘confidence_stats’] = { ‘mean’: np.mean(confidences), ‘min’: np.min(confidences), ‘max’: np.max(confidences), ‘std’: np.std(confidences) } return extracted_data def visualize_results(self, image_path: str, results: Dict, show_bbox: bool = True): “””Visualize OCR results with bounding boxes.””” image = cv2.imread(image_path) image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) plt.figure(figsize=(15, 10)) if show_bbox: plt.subplot(2, 2, 1) img_with_boxes = image_rgb.copy() for item in results[‘filtered_results’]: bbox = np.array(item[‘bbox’]).astype(int) cv2.polylines(img_with_boxes, [bbox], True, (255, 0, 0), 2) x, y = bbox[0] cv2.putText(img_with_boxes, f”{item[‘confidence’]:.2f}”, (x, y-10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 0, 0), 1) plt.imshow(img_with_boxes) plt.title(“OCR Results with Bounding Boxes”) plt.axis(‘off’) plt.subplot(2, 2, 2) processed = self.preprocess_image(image) plt.imshow(processed, cmap=’gray’) plt.title(“Preprocessed Image”) plt.axis(‘off’) plt.subplot(2, 2, 3) confidences = [item[‘confidence’] for item in results[‘filtered_results’]] if confidences: plt.hist(confidences, bins=20, alpha=0.7, color=’blue’) plt.xlabel(‘Confidence Score’) plt.ylabel(‘Frequency’) plt.title(‘Confidence Score Distribution’) plt.axvline(self.confidence_threshold, color=’red’, linestyle=’–‘, label=f’Threshold: {self.confidence_threshold}’) plt.legend() plt.subplot(2, 2, 4) stats = results[‘confidence_stats’] if stats: labels = [‘Mean’, ‘Min’, ‘Max’] values = [stats[‘mean’], stats[‘min’], stats[‘max’]] plt.bar(labels, values, color=[‘green’, ‘red’, ‘blue’]) plt.ylabel(‘Confidence Score’) plt.title(‘Confidence Statistics’) plt.ylim(0, 1) plt.tight_layout() plt.show() def smart_text_analysis(self, text: str) -> Dict: “””Perform intelligent analysis of extracted text.””” analysis = { ‘language_detection’: ‘unknown’, ‘text_type’: ‘unknown’, ‘key_info’: {}, ‘patterns’: [] } email_pattern = r’b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+.[A-Z|a-z]{2,}b’ phone_pattern = r'(+d{1,3}[-.s]?)?(?d{3})?[-.s]?d{3}[-.s]?d{4}’ url_pattern = r’http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+’ date_pattern = r’bd{1,2}[/-]d{1,2}[/-]d{2,4}b’ patterns = { ’emails’: re.findall(email_pattern, text, re.IGNORECASE), ‘phones’: re.findall(phone_pattern, text), ‘urls’: re.findall(url_pattern, text, re.IGNORECASE), ‘dates’: re.findall(date_pattern, text) } analysis[‘patterns’] = {k: v for k, v in patterns.items() if v} if any(patterns.values()): if patterns.get(’emails’) or patterns.get(‘phones’): analysis[‘text_type’] = ‘contact_info’ elif patterns.get(‘urls’): analysis[‘text_type’] = ‘web_content’ elif patterns.get(‘dates’): analysis[‘text_type’] = ‘document_with_dates’ if re.search(r'[а-яё]’, text.lower()): analysis[‘language_detection’] = ‘russian’ elif re.search(r'[àáâãäåæçèéêëìíîïñòóôõöøùúûüý]’, text.lower()): analysis[‘language_detection’] = ‘romance_language’ elif re.search(r'[一-龯]’, text): analysis[‘language_detection’] = ‘chinese’ elif re.search(r'[ひらがなカタカナ]’, text): analysis[‘language_detection’] = ‘japanese’ elif re.search(r'[a-zA-Z]’, text): analysis[‘language_detection’] = ‘latin_based’ return analysis def process_batch(self, image_folder: str) -> List[Dict]: “””Process multiple images in batch.””” results = [] supported_formats = (‘.png’, ‘.jpg’, ‘.jpeg’, ‘.bmp’, ‘.tiff’) for filename in os.listdir(image_folder): if filename.lower().endswith(supported_formats): image_path = os.path.join(image_folder, filename) try: result = self.extract_text(image_path) result[‘filename’] = filename results.append(result) print(f” Processed: {filename}”) except Exception as e: print(f” Error processing {filename}: {str(e)}”) return results def export_results(self, results: Dict, format: str = ‘json’) -> str: “””Export results in specified format.””” if format.lower() == ‘json’: output = json.dumps(results, indent=2, ensure_ascii=False) filename = ‘ocr_results.json’ elif format.lower() == ‘txt’: output = results[‘full_text’] filename = ‘extracted_text.txt’ else: raise ValueError(“Supported formats: ‘json’, ‘txt'”) with open(filename, ‘w’, encoding=’utf-8′) as f: f.write(output) print(f” Results exported to: {filename}”) return filename We define an AdvancedOCRAgent that we initialize with multilingual EasyOCR and a GPU, and we set a confidence threshold to control output quality. We preprocess images (CLAHE, denoise, sharpen, adaptive threshold), extract text, visualize bounding boxes and confidence, run smart pattern/language analysis, support batch folders, and export results as JSON or TXT. Check out the FULL CODES here. Copy CodeCopiedUse a different Browser def demo_ocr_agent(): “””Demonstrate the OCR agent capabilities.””” print(” Advanced OCR AI Agent Demo”) print(“=” * 50) ocr = AdvancedOCRAgent(languages=[‘en’], gpu=True) image_path = ocr.upload_image() if image_path: try: results = ocr.extract_text(image_path, preprocess=True) print(“n OCR Results:”) print(f”Words detected: {results[‘word_count’]}”) print(f”Lines detected: {results[‘line_count’]}”) print(f”Average confidence: {results[‘confidence_stats’].get(‘mean’, 0):.2f}”) print(“n Extracted Text:”) print(“-” * 30) print(results[‘full_text’]) print(“-” * 30) analysis = ocr.smart_text_analysis(results[‘full_text’]) print(f”n Smart Analysis:”) print(f”Detected text type: {analysis[‘text_type’]}”) print(f”Language hints: {analysis[‘language_detection’]}”) if analysis[‘patterns’]: print(f”Found patterns: {list(analysis[‘patterns’].keys())}”) ocr.visualize_results(image_path, results) ocr.export_results(results, ‘json’) except Exception as e: print(f” Error: {str(e)}”) else: print(“No image uploaded. Please try again.”) if __name__ == “__main__”: demo_ocr_agent() We create a demo function that walks us through the full OCR workflow: we initialize the agent with English and GPU support, upload an image, preprocess it, and extract text with confidence stats. We then display

arXiv:2509.09631v1 Announce Type: cross Abstract: Zero-shot Text-to-Speech (TTS) aims to synthesize high-quality speech that mimics the voice of an unseen speaker using only a short reference sample, requiring not only speaker adaptation but also accurate modeling of prosodic attributes. Recent approaches based on language models, diffusion, and flow matching have shown promising results in zero-shot TTS, but still suffer from slow inference and repetition artifacts. Discrete codec representations have been widely adopted for speech synthesis, and recent works have begun to explore diffusion models in purely discrete settings, suggesting the potential of discrete generative modeling for speech synthesis. However, existing flow-matching methods typically embed these discrete tokens into a continuous space and apply continuous flow matching, which may not fully leverage the advantages of discrete representations. To address these challenges, we introduce DiFlow-TTS, which, to the best of our knowledge, is the first model to explore purely Discrete Flow Matching for speech synthesis. DiFlow-TTS explicitly models factorized speech attributes within a compact and unified architecture. It leverages in-context learning by conditioning on textual content, along with prosodic and acoustic attributes extracted from a reference speech, enabling effective attribute cloning in a zero-shot setting. In addition, the model employs a factorized flow prediction mechanism with distinct heads for prosody and acoustic details, allowing it to learn aspect-specific distributions. Experimental results demonstrate that DiFlow-TTS achieves promising performance in several key metrics, including naturalness, prosody, preservation of speaker style, and energy control. It also maintains a compact model size and achieves low-latency inference, generating speech up to 25.8 times faster than the latest existing baselines.

arXiv:2501.02348v2 Announce Type: replace Abstract: Complex problem-solving requires cognitive flexibility–the capacity to entertain multiple perspectives while preserving their distinctiveness. This flexibility replicates the “wisdom of crowds” within a single individual, allowing them to “think with many minds.” While mental simulation enables imagined deliberation, cognitive constraints limit its effectiveness. We propose synthetic deliberation, a Large Language Model (LLM)-based method that simulates discourse between agents embodying diverse perspectives, as a solution. Using a custom GPT-based model, we showcase its benefits: concurrent processing of multiple viewpoints without cognitive degradation, parallel exploration of perspectives, and precise control over viewpoint synthesis. By externalizing the deliberative process and distributing cognitive labor between parallel search and integration, synthetic deliberation transcends mental simulation’s limitations. This approach shows promise for strategic planning, policymaking, and conflict resolution.

arXiv:2509.09524v1 Announce Type: new Abstract: This system paper presents the DeMeVa team’s approaches to the third edition of the Learning with Disagreements shared task (LeWiDi 2025; Leonardelli et al., 2025). We explore two directions: in-context learning (ICL) with large language models, where we compare example sampling strategies; and label distribution learning (LDL) methods with RoBERTa (Liu et al., 2019b), where we evaluate several fine-tuning methods. Our contributions are twofold: (1) we show that ICL can effectively predict annotator-specific annotations (perspectivist annotations), and that aggregating these predictions into soft labels yields competitive performance; and (2) we argue that LDL methods are promising for soft label predictions and merit further exploration by the perspectivist community.

arXiv:2509.09196v1 Announce Type: new Abstract: Contextual biasing improves rare word recognition of ASR models by prioritizing the output of rare words during decoding. A common approach is Trie-based biasing, which gives “bonus scores” to partial hypothesis (e.g. “Bon”) that may lead to the generation of the rare word (e.g. “Bonham”). If the full word (“Bonham”) isn’t ultimately recognized, the system revokes those earlier bonuses. This revocation is limited to beam search and is computationally expensive, particularly for models with large decoders. To overcome these limitations, we propose adapting ASR models to look ahead and predict multiple steps at once. This avoids the revocation step entirely by better estimating whether a partial hypothesis will lead to the generation of the full rare word. By fine-tuning Whisper with only 10 hours of synthetic data, our method reduces the word error rate on the NSC Part 2 test set from 30.86% to 12.19%.