Uncategorized

Tencent Hunyuan researchers have released HY-MT1.5, a multilingual machine translation family that targets both mobile devices and cloud systems with the same training recipe and metrics. HY-MT1.5 consists of 2 translation models, HY-MT1.5-1.8B and HY-MT1.5-7B, supports mutual translation across 33 languages with 5 ethnic and dialect variations, and is available on GitHub and Hugging Face under open weights. Model family and deployment targets HY-MT1.5-7B is an upgraded version of the WMT25 championship system Hunyuan-MT-7B. It is optimized for explanatory translation and mixed language scenarios, and adds native support for terminology intervention, contextual translation and formatted translation. HY-MT1.5-1.8B is the compact variant. It has less than one third the parameters of HY-MT1.5-7B but delivers comparable translation performance in the reported benchmarks. After quantization, the 1.8B model can run on edge devices and support real time translation. The quantized HY-MT1.5-1.8B operates on devices with about 1 GB of memory and reaches an average response time of about 0.18 seconds for Chinese inputs of around 50 tokens, while surpassing mainstream commercial translation APIs in quality. HY-MT1.5-7B targets server and high end edge deployment, where latency around 0.45 seconds is acceptable in exchange for higher quality. Holistic training framework The research team defines HY-MT1.5 as a translation specific language model trained with a multi stage pipeline. The pipeline has 5 main components: General pre training: The base model is first pre-trained on large scale multilingual text with a language modeling objective. This builds shared representations across languages. MT oriented pre training: The model is then exposed to parallel corpora and translation oriented objectives. This step aligns the generation distribution with real translation tasks rather than open ended text generation. Supervised fine tuning: High quality sentence and document level parallel data is used to fine tune the model with supervised loss. This stage sharpens literal correctness, domain coverage and direction specific behavior, such as ZH to EN versus EN to ZH. On policy distillation from 7B to 1.8B: HY-MT1.5-7B is used as a teacher for HY-MT1.5-1.8B. The research team collects about 1 million monolingual prompts across the 33 languages, runs them through the teacher and uses reverse Kullback Leibler divergence on the student rollouts to match the teacher distribution. This yields a 1.8B student that inherits most of the 7B model’s translation behavior with much lower cost. Reinforcement learning with rubrics based evaluation: In the final stage, both models are optimized with a group relative policy optimization style algorithm and a rubrics based reward model. Human reviewers score translations on multiple axes such as accuracy, fluency, idiomaticity and cultural appropriateness. The reward model distills those scores and guides the policy update. This pipeline is specific to machine translation. It differs from chat oriented LLM training by combining translation centric supervised data, on policy distillation within the translation domain and RL tuned with fine grained translation rubrics. Benchmark results against open and commercial systems HY-MT1.5 is evaluated on Flores 200, WMT25 and a Mandarin to minority language benchmark using XCOMET-XXL and CometKiwi. https://arxiv.org/pdf/2512.24092v1 Key results from the above Table in the report: On Flores 200, HY-MT1.5-7B reaches XCOMET-XXL scores of 0.8690 for ZH to XX, 0.9093 for EN to XX and 0.8098 for XX to XX. It outperforms translation specialized models such as iFLYTEK Translator and Doubao Translator and matches or exceeds medium sized general models like Qwen3-235B-A22B. On WMT25, HY-MT1.5-7B reaches XCOMET-XXL 0.6159. This is about 0.065 higher than Gemini 3.0 Pro and significantly above translation oriented models such as Seed-X-PPO-7B and Tower-Plus-72B. HY-MT1.5-1.8B scores 0.5308, which still exceeds many medium sized general models and translation systems. On Mandarin to minority language pairs, HY-MT1.5-7B achieves 0.6174 in XCOMET-XXL, higher than all baselines including Gemini 3.0 Pro. The 1.8B variant reaches 0.5806 and still surpasses several very large models like DeepSeek-V3.2. In human evaluation on a 0 to 4 scale for Chinese to English and English to Chinese, HY-MT1.5-1.8B achieves an average score of 2.74, which is higher than Baidu, iFLYTEK, Doubao, Microsoft and Google translator systems under the same protocol. Practical features for product use The models expose three prompt driven capabilities that matter in production systems: Terminology intervention: A prompt template lets you inject term mappings such as “混元珠 → Chaos Pearl”. Without the mapping, the model outputs an ambiguous transliteration. With the mapping, it enforces a consistent domain specific term. This is critical for legal, medical or brand constrained content. Context aware translation: A second template accepts a context block plus the sentence to translate. The report shows the word “pilot” misinterpreted as a person when context is absent. When a paragraph about TV series is added, the model correctly translates “pilot” as an episode. Format preserving translation: A third template wraps the source in <source> tags and marks spans with <sn> tags. The instruction forces the model to keep tags and output inside <target> tags. This allows HTML or XML like text to survive translation with structure preserved. These are implemented as prompt formats, so they are available even when you call the public weights through standard LLM stacks. Quantization and edge deployment HY-MT1.5-1.8B is evaluated with FP8 and Int4 post training quantization using GPTQ. https://arxiv.org/pdf/2512.24092v1 The above Table 4 shows: FP8 keeps XCOMET-XXL scores very close to the full precision model, for example 0.8379 versus 0.8361 for ZH to XX. Int4 reduces size further but introduces clear quality drops on Flores 200. On Hugging Face, Tencent publishes both FP8 and GPTQ Int4 variants for HY-MT1.5-1.8B and HY-MT1.5-7B, along with GGUF versions for local inference stacks. Quantization is the mechanism that enables the reported 1 GB memory deployment and low latency on consumer hardware. Key Takeaways HY-MT1.5 is a 2 model translation family, HY-MT1.5-1.8B and HY-MT1.5-7B, supporting mutual translation across 33 languages plus 5 dialect or variant forms, released with open weights on GitHub and Hugging Face. HY-MT1.5-1.8B is a distillation based edge model that runs on about 1 GB memory with around 0.18 seconds latency for 50 token Chinese inputs, while achieving industry leading performance among models of similar size

DeepSeek researchers are trying to solve a precise issue in large language model training. Residual connections made very deep networks trainable, hyper connections widened that residual stream, and training then became unstable at scale. The new method mHC, Manifold Constrained Hyper Connections, keeps the richer topology of hyper connections but locks the mixing behavior on a well defined manifold so that signals remain numerically stable in very deep stacks. https://www.arxiv.org/pdf/2512.24880 From Residual Connections To Hyper Connections Standard residual connections, as in ResNets and Transformers, propagate activations with xl+1​=xl​+F(xl​,Wl​)The identity path preserves magnitude and keeps gradients usable even when you stack many layers. Hyper Connections generalize this structure. Instead of a single residual vector of size C, the model keeps an n stream buffer 𝑥𝑙∈𝑅𝑛×𝐶. Three learned mappings control how each layer reads and writes this buffer: Hlpre selects a mixture of streams as the layer input F is the usual attention or feed forward sublayer Hlpost writes results back into the n stream buffer Hlres​∈Rn×n mixes streams between layers The update has the formxl+1​=Hlres​xl​+Hlpost​⊤F(Hlpre​xl​,Wl​) With n set to 4, this design increases expressivity without a large increase in floating point cost, which is why hyper connections improve downstream performance in language models. Why Hyper Connections Become Unstable The problem appears when you look at the product of residual mixers across many layers. In a 27B mixture of experts model, DeepSeek studies the composite mapping and defines an Amax Gain Magnitude based on maximum row and column sums. This metric measures worst case amplification in the forward and backward signal paths. In the hyper connection model, this gain reaches peaks around 3000, far from the ideal value 1 that you expect from a stable residual path. This means small per layer deviations compound into very large amplification factors across depth. Training logs show loss spikes and unstable gradient norms relative to a baseline residual model. At the same time, keeping a multi stream buffer increases memory traffic for each token, which makes naive scaling of hyper connections unattractive for production large language models. Manifold Constrained Hyper Connections mHC keeps the multi stream residual idea but constrains the dangerous part. The residual mixing matrix Hlres no longer lives in the full n by n space. Instead, it is projected onto the manifold of doubly stochastic matrices, also called the Birkhoff polytope. In that set all entries are non negative and each row and each column sums to 1. DeepSeek team enforces this constraint with the classical Sinkhorn Knopp algorithm from 1967, which alternates row and column normalizations to approximate a doubly stochastic matrix. The research team uses 20 iterations per layer during training, which is enough to keep the mapping close to the target manifold while keeping cost manageable. Under these constraints, Hlres​xl behaves like a convex combination of residual streams. Total feature mass is preserved and the norm is tightly regularized, which eliminates the explosive growth seen in plain hyper connections. The research team also parameterize input and output mappings so that coefficients are non negative, which avoids cancellation between streams and keeps the interpretation as averaging clear. With mHC the composite Amax Gain Magnitude stays bounded and peaks at about 1.6 in the 27B model, compared with peaks near 3000 for the unconstrained variant. That is a reduction of about 3 orders of magnitude in worst case amplification, and it comes from a direct mathematical constraint rather than tuned tricks. Systems Work And Training Overhead Constraining every residual mixer with Sinkhorn style iterations adds cost on paper. The research team addresses this with several systems choices: Fused kernels combine RMSNorm, projections and gating for the mHC mappings so that memory traffic stays low Recompute based activation checkpointing trades compute for memory by recomputing mHC activations during backprop for blocks of layers Integration with a DualPipe like pipeline schedule overlaps communication and recomputation, so that additional work does not stall the training pipeline In large scale in house training runs, mHC with expansion rate n equal to 4 adds about 6.7 percent training time overhead relative to the baseline architecture. That figure already includes both the extra compute from Sinkhorn Knopp and the infrastructure optimizations. https://www.arxiv.org/pdf/2512.24880 Empirical Results The research team trains 3B, 9B and 27B mixture of experts models and evaluates them on a standard language model benchmark suite, including tasks like BBH, DROP, GSM8K, HellaSwag, MMLU, PIQA and TriviaQA. For the 27B model, the reported numbers on a subset of tasks show the pattern clearly: Baseline: BBH 43.8, DROP F1 47.0 With hyper connections: BBH 48.9, DROP 51.6 With mHC: BBH 51.0, DROP 53.9 So hyper connections already provide a gain over the basic residual design, and manifold constrained hyper connections push performance further while restoring stability. Similar trends appear on other benchmarks and across model sizes, and scaling curves suggest that the advantage persists across compute budgets and through the full training trajectory rather than only at convergence. Key Takeaways mHC stabilizes widened residual streams: mHC, Manifold Constrained Hyper Connections, widens the residual pathway into 4 interacting streams like HC, but constrains the residual mixing matrices on a manifold of doubly stochastic matrices, so long range propagation remains norm controlled instead of exploding. Exploding gain is reduced from ≈3000 to ≈1.6: For a 27B MoE model, the Amax Gain Magnitude of the composite residual mapping peaks near 3000 for unconstrained HC, while mHC keeps this metric bounded around 1.6, which removes the exploding residual stream behavior that previously broke training. Sinkhorn Knopp enforces doubly stochastic residual mixing: Each residual mixing matrix is projected with about 20 Sinkhorn Knopp iterations so that rows and columns both sum to 1, making the mapping a convex combination of permutations, which restores an identity like behavior while still allowing rich cross stream communication. Small training overhead, measurable downstream gains: Across 3B, 9B and 27B DeepSeek MoE models, mHC improves benchmark accuracy, for example about plus 2.1 percent on BBH for the 27B model, while adding only about 6.7 percent training time

The most amazing drummer on the internet My daughter introduced me to El Estepario Siberiano’s YouTube channel a few months back, and I have been obsessed ever since. The Spanish drummer (real name: Jorge Garrido) posts videos of himself playing supercharged cover versions of popular tracks, hitting his drums with such jaw-dropping speed and technique that he makes other pro drummers shake their heads in disbelief. The dozens of reaction videos posted by other musicians are a joy in themselves.  EL ESTEPARIO SIBERIANO VIA YOUTUBE Garrido is up-front about the countless hours that it took to get this good. He says he sat behind his kit almost all day, every day for years. At a time when machines appear to do it all, there’s a kind of defiance in that level of human effort. It’s why my favorites are Garrido’s covers of electronic music, where he out-drums the drum machine. Check out his version of Skrillex and Missy Elliot’s “Ra Ta Ta” and tell me it doesn’t put happiness in your heart. Finding signs of life in the uncanny valley Watching Sora ­videos of Michael Jackson stealing a box of chicken nuggets or Sam Altman biting into the pink meat of a flame-grilled Pikachu has given me flashbacks to an Ed Atkins exhibition at Tate Britain I saw a few months ago. Atkins is one of the most influential and unsettling British artists of his generation. He is best known for hyper-detailed CG animations of himself (pore-perfect skin, janky movement) that play with the virtual representation of human emotions.  Still from ED ATKINS PIANOWORK 2 2023COURTESY: THE ARTIST, CABINET GALLERY, LONDON, DÉPENDANCE, BRUSSELS, GLADSTONE GALLERY In The Worm we see a CGI Atkins make a long-distance call to his mother during a covid lockdown. The audio is from a recording of an actual conversation. Are we watching Atkins cry or his avatar? Our attention flickers between two realities. “When an actor breaks character during a scene, it’s known as corpsing,” Atkins has said. “I want everything I make to corpse.” Next to Atkins’s work, generative videos look like cardboard cutouts: lifelike but not alive. A dark and dirty book about a talking dingo What’s it like to be a pet? Australian author Laura Jean McKay’s debut novel, The Animals in That Country, will make you wish you’d never asked. A flu-like pandemic leaves people with the ability to hear what animals are saying. If that sounds too Dr. Dolittle for your tastes, rest assured: These animals are weird and nasty. A lot of the time they don’t even make any sense.  SCRIBE With everybody now talking to their computers, McKay’s book resets the anthropomorphic trap we’ve all fallen into. It’s a brilliant evocation of what a nonhuman mind might contain—and a meditation on the hard limits of communication.

Editor’s note: This article is a part of our series on visualizing the foundations of machine learning.

Recursive Language Models aim to break the usual trade off between context length, accuracy and cost in large language models. Instead of forcing a model to read a giant prompt in one pass, RLMs treat the prompt as an external environment and let the model decide how to inspect it with code, then recursively call itself on smaller pieces. https://arxiv.org/pdf/2512.24601 The Basics The full input is loaded into a Python REPL as a single string variable. The root model, for example GPT-5, never sees that string directly in its context. Instead, it receives a system prompt that explains how to read slices of the variable, write helper functions, spawn sub LLM calls, and combine results. The model returns a final text answer, so the external interface stays identical to a standard chat completion endpoint. The RLM design uses the REPL as a control plane for long context. The environment, usually written in Python, exposes tools such as string slicing, regex search and helper functions like llm_query that call a smaller model instance, for example GPT-5-mini. The root model writes code that calls these helpers to scan, partition and summarize the external context variable. The code can store intermediate results in variables and build up the final answer step by step. This structure makes the prompt size independent from the model context window and turns long context handling into a program synthesis problem. https://arxiv.org/pdf/2512.24601 Where it stands in Evaluation? The research paper evaluates this idea on four long context benchmarks with different computational structure. S-NIAH is a constant complexity needle in a haystack task. BrowseComp-Plus is a multi hop web style question answering benchmark over up to 1,000 documents. OOLONG is a linear complexity long context reasoning task where the model must transform many entries and then aggregate them. OOLONG Pairs increases the difficulty further with quadratic pairwise aggregation over the input. These tasks stress both context length and reasoning depth, not only retrieval. On these benchmarks, RLMs give large accuracy gains over direct LLM calls and common long context agents. For GPT-5 on CodeQA, a long document question answering setup, the base model reaches 24.00 accuracy, a summarization agent reaches 41.33, while RLM reaches 62.00 and the RLM without recursion reaches 66.00. For Qwen3-Coder-480B-A35B, the base model scores 20.00, a CodeAct retrieval agent 52.00, and the RLM 56.00 with a REPL only variant at 44.66. The gains are largest on the hardest setting, OOLONG Pairs. For GPT-5, the direct model is almost unusable with F1 equal to 0.04. Summarization and CodeAct agents sit near 0.01 and 24.67. The full RLM reaches 58.00 F1 and the non recursive REPL variant still achieves 43.93. For Qwen3-Coder, the base model stays below 0.10 F1, while the full RLM reaches 23.11 and the REPL only version 17.34. These numbers show that both the REPL and recursive sub calls are critical on dense quadratic tasks. https://arxiv.org/pdf/2512.24601 BrowseComp-Plus highlights effective context extension. The corpus ranges from about 6M to 11M tokens, which is 2 orders of magnitude beyond the 272k token context window of GPT-5. RLM with GPT 5 maintains strong performance even when given 1,000 documents in the environment variable, while standard GPT-5 baselines degrade as document count grows. On this benchmark, RLM GPT 5 achieves around 91.33 accuracy with an average cost of 0.99 USD per query, while a hypothetical model that reads the full context directly would cost between $1.50 and $2.75 at current pricing. The research paper also analyzes the trajectories of RLM runs. Several behavior patterns emerge. The model often starts with a peek step where it inspects the first few thousand characters of the context. It then uses grep style filtering with regex or keyword search to narrow down relevant lines. For more complex queries, it partitions the context into chunks and calls recursive LMs on each chunk to perform labeling or extraction, followed by programmatic aggregation. On long output tasks, the RLM stores partial outputs in variables and stitches them together, which bypasses output length limits of the base model. The new take from Prime Intellect Prime Intellect team has turned this concept into a concrete environment, RLMEnv, integrated in their verifiers stack and Environments Hub. In their design, the main RLM has only a Python REPL, while sub LLMs receive the heavy tools such as web search or file access. The REPL exposes an llm_batch function so the root model can fan out many sub queries in parallel, and an answer variable where the final solution must be written and flagged as ready. This isolates token heavy tool outputs from the main context and lets the RLM delegate expensive operations to sub models. Prime Intellect evaluates this implementation on four environments. DeepDive tests web research with search and open tools and very verbose pages. Math python exposes a Python REPL for difficult competition style math problems. Oolong reuses the long context benchmark inside RLMEnv. Verbatim copy focuses on exact reproduction of complex strings across content types such as JSON, CSV and mixed codes. Across these environments, GPT-5-mini and the INTELLECT-3-MoE model both gain from the RLM scaffold in success rate and in robustness to very long contexts, especially when tool output would otherwise swamp the model context The research paper’s author team and Prime Intellect team both stress that current implementations are not fully optimized. RLM calls are synchronous, recursion depth is limited and cost distributions have heavy tails due to very long trajectories. The real opportunity is to combine RLM scaffolding with dedicated reinforcement learning so that models learn better chunking, recursion and tool usage policies over time. If that happens, RLMs provide a framework where improvements in base models and in systems design convert directly into more capable long horizon agents that can consume 10M plus token environments without context rot. Key Takeaways Here are 5 concise, technical takeaways you can plug under the article. RLMs reframe long context as an environment variable: Recursive Language Models treat the entire prompt as an external string in

This article is divided into five parts; they are: • An Example of Tensor Parallelism • Setting Up Tensor Parallelism • Preparing Model for Tensor Parallelism • Train a Model with Tensor Parallelism • Combining Tensor Parallelism with FSDP Tensor parallelism originated from the Megatron-LM paper.