{"id":113345,"date":"2026-08-24T00:42:28","date_gmt":"2026-08-24T00:42:28","guid":{"rendered":"https:\/\/youzum.net\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/"},"modified":"2026-08-24T00:42:28","modified_gmt":"2026-08-24T00:42:28","slug":"meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu","status":"publish","type":"post","link":"https:\/\/youzum.net\/de\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/","title":{"rendered":"Meet FreeToken: An Edge-Native MoE Serving Engine that Runs 753B GLM-5.2 on a Single Workstation GPU"},"content":{"rendered":"<p class=\"wp-block-paragraph\">Frontier open-weight models are shipping faster than the hardware assumptions around them. <a href=\"https:\/\/arxiv.org\/abs\/2607.24653\">Kimi-K3<\/a>, <a href=\"https:\/\/huggingface.co\/zai-org\/GLM-5.2\">GLM-5.2<\/a> and <a href=\"https:\/\/huggingface.co\/deepseek-ai\/DeepSeek-V4-Flash-0731\">DeepSeek-V4-Flash<\/a> are closing the capability gap with proprietary systems, but releasing parameters only determines who can <em>obtain<\/em> a model \u2014 not who can afford to <em>run<\/em> it. Serving them still assumes datacenter-class GPU clusters, and as agentic workloads push inference demand up, that cost lands hardest on individual developers and small teams. Meanwhile, more than a hundred million consumer machines already carry discrete GPUs. A team of researchers from UC Berkeley and UT Austin propose <a href=\"https:\/\/arxiv.org\/abs\/2608.16157\">FreeToken<\/a>. The research team argued the missing piece is not hardware but a serving system: it treats a personal machine as a unified, elastic inference platform rather than a small GPU, and continuously maps computation and model state onto whatever GPU, CPU, memory and interconnect bandwidth the machine actually has. The result is a 35B model at interactive speed on an 8 GB laptop GPU, 284B on a gaming desktop, and the 753B GLM-5.2 on a single workstation card.<\/p>\n<h2 class=\"wp-block-heading\"><strong>Is it deployable?<\/strong><\/h2>\n<p class=\"wp-block-paragraph\"><strong>Yes<\/strong>, FreeToken is <a href=\"https:\/\/github.com\/FlashML-org\/FreeToken\">Apache-2.0 on GitHub<\/a>, published on <a href=\"https:\/\/pypi.org\/project\/freetoken\/\">PyPI<\/a> as <code>freetoken<\/code> v0.1.2 (<code>uv pip install \"freetoken[accel]\"<\/code>), and shipped as a one-click desktop app for Windows and Linux at <a href=\"https:\/\/www.flashml.ai\/\">flashml.ai<\/a>. The CLI targets Linux x86_64 with an NVIDIA GPU on driver r580+ (CUDA 13). <code>ft serve<\/code> exposes OpenAI- and Anthropic-compatible endpoints on port 1919, and <code>ft launch claude<\/code> wires up Claude Code, Codex, OpenCode or OpenClaw against your own box.<\/p>\n<p class=\"wp-block-paragraph\">Who it fits: solo developers, startups and SMB engineering teams whose agent token bills already exceed the cost of a GPU they own; enterprises should treat it as an air-gapped or regulated-workload path, not a datacenter replacement. Strongest industry fit: healthcare and legal (data never leaves the machine), defense, finance, and IP-heavy R&amp;D. Typical applications: local coding agents, private code review, offline contract analysis, synthetic-data generation, batch evals. <\/p>\n<h2 class=\"wp-block-heading\"><strong>The gap it targets<\/strong><\/h2>\n<p class=\"wp-block-paragraph\">Mixture-of-Experts makes local frontier inference arithmetically feasible. <a href=\"https:\/\/huggingface.co\/deepseek-ai\/DeepSeek-V4-Flash-0731\">DeepSeek-V4-Flash<\/a> activates 6 of 256 routed experts in each of 43 layers, so only 13B of its 284B parameters participate in any single token. Sparsity does not shrink the expert pool, though \u2014 at FP4 the full set is roughly 140 GB, so inactive experts sit in host memory and enter the execution path on demand.<\/p>\n<p class=\"wp-block-paragraph\"><strong>The research team isolates three failure modes in existing engines (<a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\">llama.cpp<\/a>, <a href=\"https:\/\/github.com\/kvcache-ai\/ktransformers\">KTransformers<\/a>, Ollama, MoE-Infinity):<\/strong><\/p>\n<ul class=\"wp-block-list\">\n<li><strong>Prefill destroys sparsity<\/strong>: Thousands of tokens per layer route to nearly the whole expert set, so a prefill pass streams the entire pool across PCIe \u2014 about two seconds on an RTX 5090, five on PCIe 4.0 desktops, ten or more on the x8 links common in laptops.<\/li>\n<li><strong>Static placement misses decode traffic<\/strong>: llama.cpp assigns MoE tensors at load time; KTransformers pins a \u201chot\u201d subset. Routing shifts every token, so most expert evaluations fall to the CPU while the GPU and the PCIe link sit idle.<\/li>\n<li><strong>Consumer CPUs cannot carry the remainder<\/strong>: Dual-channel DDR5 delivers 80\u201390 GB\/s against the 1\u20131.8 TB\/s an RTX 4090 or 5090 draws from on-package memory.<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">\n<h2 class=\"wp-block-heading\"><strong>Three mechanisms<\/strong><\/h2>\n<ul class=\"wp-block-list\">\n<li><strong>Bandwidth-adaptive execution (the q* policy)<\/strong>: Because DMA transfers and CPU expert execution read from the same host-memory subsystem, a saturated PCIe link leaves a residual bandwidth of <code>B_H \u2212 B_P<\/code>. FreeToken splits each step\u2019s <em>m<\/em> cache misses accordingly: <code>q* \u2248 m \u00d7 B_P \/ B_H<\/code> experts are filled into the GPU cache, the rest are computed in place on the CPU, and the two partial sums merge exactly \u2014 no approximation, no router modification. Both bandwidths are profiled on the deployed machine (<code>ft bench bw<\/code>), which matters: measured B_P:B_H is 52.7:77.3 on an RTX 5090 server but 11.8:47.5 on a 4060 laptop.<\/li>\n<li><strong>Semantic-aware caching<\/strong>: During prefill, full-layer double buffering streams layer <em>l+1<\/em> while the GPU computes layer <em>l<\/em>. Recurrent-state checkpoints are anchored at special-token boundaries \u2014 thinking blocks, tool calls, tool outputs \u2014 precisely where agent harnesses truncate context, so an edit re-prefills only the new suffix. During decode, a shared LRU expert cache spanning all MoE layers follows the router instead of a placement frozen at load time.<\/li>\n<li><strong>Elastic memory management<\/strong>: At scheduler safe points the GPU expert cache is rebuilt under a revised VRAM budget without restarting the engine or reloading the host pool. Experts are read from disk straight into their final host layout, then pinned; no GPU warmup is required because the first request is served with a cold cache.<\/li>\n<\/ul>\n<h2 class=\"wp-block-heading\"><strong>Results<\/strong><\/h2>\n<\/p><p class=\"wp-block-paragraph\">On an RTX 5090, FreeToken sustains 77\u201383 tok\/s on <a href=\"https:\/\/huggingface.co\/Qwen\/Qwen3.6-35B-A3B\">Qwen3.6-35B-A3B<\/a> (BF16) and 22\u201325 tok\/s on DeepSeek-V4-Flash (MXFP4) \u2014 1.5\u20132.3\u00d7 the strongest baseline, with decode staying within 12% of the single-turn rate across three agentic workloads. Worst-case TTFT stays below 44 s in every cell; llama.cpp hits 232 s, Ollama 179 s and KTransformers 946 s somewhere in the matrix, past the point where agent clients time out.<\/p>\n<p class=\"wp-block-paragraph\">At equal cache capacity (37% of the Qwen3.6 pool), the global LRU misses 16% of decode-time expert reads against 41% for KTransformers and 62% for llama.cpp. On an 8 GB RTX 4060 laptop the NVFP4 build serves 35B at 39.3 tok\/s \u2014 above the 33 tok\/s median decode speed measured for Codex in production traces. On a single RTX PRO 6000, <a href=\"https:\/\/huggingface.co\/nvidia\/GLM-5.2-NVFP4\">GLM-5.2<\/a> (753B, 40B active) runs at 14.9 tok\/s versus llama.cpp\u2019s 7.3.<\/p>\n<h2 class=\"wp-block-heading\"><strong>Data Check<\/strong><\/h2>\n<div>\n<div class=\"rc-bar\">\n<span class=\"rc-ttl\">Reality Check \u00b7 FlashML FreeToken<\/span><br \/>\n<span class=\"rc-score\">INFLATION SCORE 59\/100<\/span>\n<\/div>\n<div class=\"rc-meta\">As of Aug 23, 2026 \u00b7 default mode \u00b7 audited: <a href=\"https:\/\/arxiv.org\/abs\/2608.16157\" target=\"_blank\" rel=\"noopener\">arXiv:2608.16157<\/a>, <a href=\"https:\/\/github.com\/FlashML-org\/FreeToken\" target=\"_blank\" rel=\"noopener\">GitHub repo<\/a>, <a href=\"https:\/\/www.flashml.ai\/\" target=\"_blank\" rel=\"noopener\">flashml.ai<\/a><\/div>\n<div class=\"rc-strip\">\n<div class=\"rc-cnt\"><b class=\"c-ok\">3<\/b><span>Verified<\/span><\/div>\n<div class=\"rc-cnt\"><b class=\"c-sr\">9<\/b><span>Self-rep.<\/span><\/div>\n<div class=\"rc-cnt\"><b class=\"c-fl\">4<\/b><span>Misleading<\/span><\/div>\n<div class=\"rc-cnt\"><b class=\"c-no\">0<\/b><span>Contradicted<\/span><\/div>\n<div class=\"rc-cnt\"><b class=\"c-nf\">0<\/b><span>Not found<\/span><\/div>\n<\/div>\n<div class=\"rc-form\">Score formula: 8 \u00d7 misleading + 15 \u00d7 contradicted + 3 \u00d7 self-reported, capped at 100. The score is driven by the self-reported column, not by dishonesty \u2014 the code went public six days before this audit, so no independent reproduction exists yet.<\/div>\n<div class=\"rc-h\">Claim table \u00b7 16 claims<\/div>\n<div class=\"rc-wrap\">\n<table>\n<thead>\n<tr>\n<th class=\"w1\">Claim<\/th>\n<th class=\"w2\">Their number<\/th>\n<th class=\"w3\">Independent check<\/th>\n<th class=\"w4\">Verdict &amp; source<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td class=\"cl w1\">Decode, Qwen3.6-35B-A3B BF16, RTX 5090<\/td>\n<td class=\"w2\" data-l=\"Their number\">77\u201383 tok\/s<\/td>\n<td class=\"w3\" data-l=\"Independent check\">None found<\/td>\n<td class=\"vd w4 c-sr\" data-l=\"Verdict\">SELF-REPORTED<span class=\"src\">Paper Fig 3<\/span><\/td>\n<\/tr>\n<tr>\n<td class=\"cl w1\">Decode, DeepSeek-V4-Flash MXFP4, RTX 5090<\/td>\n<td class=\"w2\" data-l=\"Their number\">22\u201325 tok\/s<\/td>\n<td class=\"w3\" data-l=\"Independent check\">None found<\/td>\n<td class=\"vd w4 c-sr\" data-l=\"Verdict\">SELF-REPORTED<span class=\"src\">Paper Fig 3<\/span><\/td>\n<\/tr>\n<tr>\n<td class=\"cl w1\">Decode speedup vs strongest baseline<\/td>\n<td class=\"w2\" data-l=\"Their number\">1.5\u20132.3\u00d7<\/td>\n<td class=\"w3\" data-l=\"Independent check\">Recomputes exactly from Fig 3<\/td>\n<td class=\"vd w4 c-sr\" data-l=\"Verdict\">SELF-REPORTED<span class=\"src\">Paper \u00a75.2<\/span><\/td>\n<\/tr>\n<tr>\n<td class=\"cl w1\">Decode stability across agent workloads<\/td>\n<td class=\"w2\" data-l=\"Their number\">within 12% of W1<\/td>\n<td class=\"w3\" data-l=\"Independent check\">None found<\/td>\n<td class=\"vd w4 c-sr\" data-l=\"Verdict\">SELF-REPORTED<span class=\"src\">Paper \u00a75.2<\/span><\/td>\n<\/tr>\n<tr>\n<td class=\"cl w1\">Worst-case TTFT vs baselines<\/td>\n<td class=\"w2\" data-l=\"Their number\">&lt;44 s vs 232 \/ 179 \/ 946 s<\/td>\n<td class=\"w3\" data-l=\"Independent check\">None found<\/td>\n<td class=\"vd w4 c-sr\" data-l=\"Verdict\">SELF-REPORTED<span class=\"src\">Paper \u00a75.2<\/span><\/td>\n<\/tr>\n<tr>\n<td class=\"cl w1\">4060 laptop \u201cexceeds Codex median 33 tok\/s\u201d<\/td>\n<td class=\"w2\" data-l=\"Their number\">39.3 vs 33<\/td>\n<td class=\"w3\" data-l=\"Independent check\">TraceLab 33.9 is <em>normalized<\/em>; Codex pure decode median 57.1, w.avg 61.0<\/td>\n<td class=\"vd w4 c-fl\" data-l=\"Verdict\">MISLEADING<span class=\"src\">arXiv:2606.30560<\/span><\/td>\n<\/tr>\n<tr>\n<td class=\"cl w1\">Laptop is \u201c92% of the RTX 4090 rate\u201d<\/td>\n<td class=\"w2\" data-l=\"Their number\">39.3 \/ 42.9<\/td>\n<td class=\"w3\" data-l=\"Independent check\">Arithmetic correct, but 39.3 is NVFP4 and 42.9 is BF16<\/td>\n<td class=\"vd w4 c-fl\" data-l=\"Verdict\">MISLEADING<span class=\"src\">Paper Fig 5<\/span><\/td>\n<\/tr>\n<tr>\n<td class=\"cl w1\">GLM-5.2 753B on one RTX PRO 6000<\/td>\n<td class=\"w2\" data-l=\"Their number\">14.9 vs llama.cpp 7.3<\/td>\n<td class=\"w3\" data-l=\"Independent check\">None found<\/td>\n<td class=\"vd w4 c-sr\" data-l=\"Verdict\">SELF-REPORTED<span class=\"src\">Paper \u00a75.3<\/span><\/td>\n<\/tr>\n<tr>\n<td class=\"cl w1\">Cross-hardware lead, five consumer systems<\/td>\n<td class=\"w2\" data-l=\"Their number\">1.3\u20132.1\u00d7<\/td>\n<td class=\"w3\" data-l=\"Independent check\">Recomputes exactly from Fig 5<\/td>\n<td class=\"vd w4 c-sr\" data-l=\"Verdict\">SELF-REPORTED<span class=\"src\">Paper Fig 5<\/span><\/td>\n<\/tr>\n<tr>\n<td class=\"cl w1\">Decode expert miss rate at equal capacity<\/td>\n<td class=\"w2\" data-l=\"Their number\">16% \/ 39%<\/td>\n<td class=\"w3\" data-l=\"Independent check\">None found; trace <em>replay<\/em>, not live serving<\/td>\n<td class=\"vd w4 c-sr\" data-l=\"Verdict\">SELF-REPORTED<span class=\"src\">Paper Fig 4b<\/span><\/td>\n<\/tr>\n<tr>\n<td class=\"cl w1\">Prefill 8,192-tok chunk; overlap penalty<\/td>\n<td class=\"w2\" data-l=\"Their number\">1.19\u20131.22 s; 19\/25\/26%<\/td>\n<td class=\"w3\" data-l=\"Independent check\">None found<\/td>\n<td class=\"vd w4 c-sr\" data-l=\"Verdict\">SELF-REPORTED<span class=\"src\">Paper Fig 4a<\/span><\/td>\n<\/tr>\n<tr>\n<td class=\"cl w1\">\u201c753B on a single workstation GPU\u201d framing<\/td>\n<td class=\"w2\" data-l=\"Their number\">1 GPU<\/td>\n<td class=\"w3\" data-l=\"Independent check\">True for VRAM; hosts carry 512 GiB and 192 GB DRAM<\/td>\n<td class=\"vd w4 c-fl\" data-l=\"Verdict\">MISLEADING<span class=\"src\">Paper Table 1<\/span><\/td>\n<\/tr>\n<tr>\n<td class=\"cl w1\">Baselines run at 6 CPU threads on rented servers<\/td>\n<td class=\"w2\" data-l=\"Their number\">6 threads<\/td>\n<td class=\"w3\" data-l=\"Independent check\">KTransformers\u2019 core contribution is many-core AMX CPU kernels<\/td>\n<td class=\"vd w4 c-fl\" data-l=\"Verdict\">MISLEADING<span class=\"src\">Paper \u00a75.1<\/span><\/td>\n<\/tr>\n<tr>\n<td class=\"cl w1\">\u201cSupports more than 20 MoE models\u201d<\/td>\n<td class=\"w2\" data-l=\"Their number\">20+<\/td>\n<td class=\"w3\" data-l=\"Independent check\">Public docs\/models.md itemizes ~17 known-good MoE checkpoints<\/td>\n<td class=\"vd w4 c-sr\" data-l=\"Verdict\">SELF-REPORTED<span class=\"src\">repo docs<\/span><\/td>\n<\/tr>\n<tr>\n<td class=\"cl w1\">License and distribution<\/td>\n<td class=\"w2\" data-l=\"Their number\">Apache-2.0, PyPI v0.1.2<\/td>\n<td class=\"w3\" data-l=\"Independent check\">LICENSE file and PyPI JSON API both confirm<\/td>\n<td class=\"vd w4 c-ok\" data-l=\"Verdict\">VERIFIED<span class=\"src\">GitHub, PyPI<\/span><\/td>\n<\/tr>\n<tr>\n<td class=\"cl w1\">Consumer discrete-GPU install base (Steam basis)<\/td>\n<td class=\"w2\" data-l=\"Their number\">~72% NVIDIA; 4060 Laptop 3.81%<\/td>\n<td class=\"w3\" data-l=\"Independent check\">Matches Valve June 2026 survey per multiple outlets<\/td>\n<td class=\"vd w4 c-ok\" data-l=\"Verdict\">VERIFIED<span class=\"src\">Valve, Jul 2026<\/span><\/td>\n<\/tr>\n<tr>\n<td class=\"cl w1\">Internal arithmetic across abstract and \u00a75<\/td>\n<td class=\"w2\" data-l=\"Their number\">all ratios<\/td>\n<td class=\"w3\" data-l=\"Independent check\">Every published ratio recomputes from Figures 3 and 5; zero errors<\/td>\n<td class=\"vd w4 c-ok\" data-l=\"Verdict\">VERIFIED<span class=\"src\">recomputed<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<div class=\"rc-h\">Flags explained<\/div>\n<div class=\"rc-flag\"><b>Denominator games \u2014 the Codex comparison mixes two metrics<\/b><span>FreeToken reports decode throughput and TTFT separately, so its 39.3 tok\/s is a pure decode rate. TraceLab\u2019s 33.9 tok\/s is a normalized rate that folds per-step TTFT into decode; the same paper puts Codex\u2019s pure decode median at 57.1 tok\/s. Like-for-like, 39.3 does not exceed Codex \u2014 it is roughly two thirds of it.<\/span><\/div>\n<div class=\"rc-flag\"><b>Denominator games \u2014 \u201c92% of the RTX 4090 rate\u201d compares 4-bit to 16-bit<\/b><span>The 39.3 tok\/s laptop figure is the NVFP4 build; the 42.9 tok\/s RTX 4090 figure is BF16. Disclosed in the Figure 5 caption, but the prose states the ratio without the precision caveat.<\/span><\/div>\n<div class=\"rc-flag\"><b>Denominator games \u2014 \u201csingle GPU\u201d omits the host requirement<\/b><span>The 753B GLM-5.2 tier sits behind 512 GiB of DDR5 on a Xeon Platinum 8559C; the 284B \u201cgaming desktop\u201d carries 192 GB. One GPU is accurate. One machine at consumer prices is not what those configurations describe.<\/span><\/div>\n<div class=\"rc-flag\"><b>Settings mismatch \u2014 baselines capped at 6 CPU threads<\/b><span>Disclosed and defensible: the paper caps rented dual-socket servers to emulate edge hosts and validates on two real edge machines at full threads. But KTransformers is built around AMX-optimized many-core CPU expert execution, so read its column as \u201cKTransformers on an edge-class host,\u201d not as its ceiling.<\/span><\/div>\n<div class=\"rc-flag rc-good\"><b>Counterweight \u2014 the arithmetic is clean<\/b><span>Every ratio in the abstract and results recomputes correctly: 1.81 \/ 1.87 \/ 2.10 \/ 2.25\u00d7 for Qwen3.6, 1.92 \/ 1.84 \/ 1.52 \/ 1.65\u00d7 for DeepSeek-V4-Flash, 2.04\u00d7 for GLM-5.2. No inflated rounding, no unexplained gaps between figures and prose.<\/span><\/div>\n<div class=\"rc-h\">Key takeaways<\/div>\n<ul class=\"rc-take\">\n<li>The paper is arithmetically clean \u2014 every published ratio recomputes from its own figures.<\/li>\n<li>Nothing is independently reproduced yet; 9 of 16 claims are self-reported by necessity, not evasion.<\/li>\n<li>Sharpest flag: 39.3 tok\/s beats Codex\u2019s normalized 33.9, not its pure decode median of 57.1.<\/li>\n<li>\u201cSingle GPU\u201d headlines quietly require 192\u2013512 GB of host DRAM.<\/li>\n<li>Baseline KTransformers runs at 6 CPU threads, below the many-core AMX config it targets.<\/li>\n<\/ul>\n<div class=\"rc-ft\">Reality Check by Marktechpost \u00b7 verified Aug 23, 2026<\/div>\n<\/div>\n<p class=\"wp-block-paragraph\">\n<h2 class=\"wp-block-heading\"><strong>Key Takeaways<\/strong><\/h2>\n<ul class=\"wp-block-list\">\n<li>FreeToken splits MoE cache misses between PCIe fills and CPU execution using measured bandwidths, not a fixed offload rule.<\/li>\n<li>Expert output stays bit-exact \u2014 no router changes, no expert substitution, no precision relaxation.<\/li>\n<li>1.5\u20132.3\u00d7 decode throughput over llama.cpp, Ollama and KTransformers, with tail TTFT under 44 s.<\/li>\n<li>35B at 39.3 tok\/s on an 8 GB laptop GPU; 753B GLM-5.2 on one workstation GPU.<\/li>\n<li>Apache-2.0, on PyPI and as a Windows\/Linux desktop app \u2014 deployable this afternoon.<\/li>\n<\/ul>\n<\/p><p class=\"wp-block-paragraph\">\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n<\/p><p class=\"wp-block-paragraph\">\n<\/p><p class=\"wp-block-paragraph\">Check out the\u00a0<strong><a href=\"https:\/\/arxiv.org\/pdf\/2608.16157\" target=\"_blank\" rel=\"noreferrer noopener\">PAPER<\/a>, <a href=\"https:\/\/github.com\/FlashML-org\/FreeToken\" target=\"_blank\" rel=\"noreferrer noopener\">GITHUB REPO<\/a> <\/strong>and<strong> <a href=\"https:\/\/github.com\/FlashML-org\/FreeToken\" target=\"_blank\" rel=\"noreferrer noopener\">PROJECT<\/a><\/strong><em>.<\/em>\u00a0Also,\u00a0feel free to follow us on\u00a0<strong><a href=\"https:\/\/x.com\/intent\/follow?screen_name=marktechpost\" target=\"_blank\" rel=\"noopener\"><mark>Twitter<\/mark><\/a><\/strong>\u00a0and don\u2019t forget to join our\u00a0<strong><a href=\"https:\/\/www.reddit.com\/r\/machinelearningnews\/\" target=\"_blank\" rel=\"noopener\">150k+ML SubReddit<\/a><\/strong>\u00a0and Subscribe to\u00a0<strong><a href=\"https:\/\/magic.beehiiv.com\/v1\/f5e63dd4-5653-4f09-83e2-321a8b1ba526?email=%7B%7Bemail%7D%7D\" target=\"_blank\" rel=\"noopener\">our Newsletter<\/a><\/strong>. Wait! are you on telegram?\u00a0<strong><a href=\"https:\/\/t.me\/machinelearningresearchnews\" target=\"_blank\" rel=\"noopener\">now you can join us on telegram as well.<\/a><\/strong><\/p>\n<p class=\"wp-block-paragraph\">Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.?\u00a0<strong><a href=\"https:\/\/forms.gle\/wbash1wF6efRj8G58\" target=\"_blank\" rel=\"noopener\"><mark>Connect with us<\/mark><\/a><\/strong><\/p>\n<p>The post <a href=\"https:\/\/www.marktechpost.com\/2026\/08\/23\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/\">Meet FreeToken: An Edge-Native MoE Serving Engine that Runs 753B GLM-5.2 on a Single Workstation GPU<\/a> appeared first on <a href=\"https:\/\/www.marktechpost.com\/\">MarkTechPost<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"<p>Frontier open-weight models are shipping faster than the hardware assumptions around them. Kimi-K3, GLM-5.2 and DeepSeek-V4-Flash are closing the capability gap with proprietary systems, but releasing parameters only determines who can obtain a model \u2014 not who can afford to run it. Serving them still assumes datacenter-class GPU clusters, and as agentic workloads push inference demand up, that cost lands hardest on individual developers and small teams. Meanwhile, more than a hundred million consumer machines already carry discrete GPUs. A team of researchers from UC Berkeley and UT Austin propose FreeToken. The research team argued the missing piece is not hardware but a serving system: it treats a personal machine as a unified, elastic inference platform rather than a small GPU, and continuously maps computation and model state onto whatever GPU, CPU, memory and interconnect bandwidth the machine actually has. The result is a 35B model at interactive speed on an 8 GB laptop GPU, 284B on a gaming desktop, and the 753B GLM-5.2 on a single workstation card. Is it deployable? Yes, FreeToken is Apache-2.0 on GitHub, published on PyPI as freetoken v0.1.2 (uv pip install &#8220;freetoken[accel]&#8221;), and shipped as a one-click desktop app for Windows and Linux at flashml.ai. The CLI targets Linux x86_64 with an NVIDIA GPU on driver r580+ (CUDA 13). ft serve exposes OpenAI- and Anthropic-compatible endpoints on port 1919, and ft launch claude wires up Claude Code, Codex, OpenCode or OpenClaw against your own box. Who it fits: solo developers, startups and SMB engineering teams whose agent token bills already exceed the cost of a GPU they own; enterprises should treat it as an air-gapped or regulated-workload path, not a datacenter replacement. Strongest industry fit: healthcare and legal (data never leaves the machine), defense, finance, and IP-heavy R&amp;D. Typical applications: local coding agents, private code review, offline contract analysis, synthetic-data generation, batch evals. The gap it targets Mixture-of-Experts makes local frontier inference arithmetically feasible. DeepSeek-V4-Flash activates 6 of 256 routed experts in each of 43 layers, so only 13B of its 284B parameters participate in any single token. Sparsity does not shrink the expert pool, though \u2014 at FP4 the full set is roughly 140 GB, so inactive experts sit in host memory and enter the execution path on demand. The research team isolates three failure modes in existing engines (llama.cpp, KTransformers, Ollama, MoE-Infinity): Prefill destroys sparsity: Thousands of tokens per layer route to nearly the whole expert set, so a prefill pass streams the entire pool across PCIe \u2014 about two seconds on an RTX 5090, five on PCIe 4.0 desktops, ten or more on the x8 links common in laptops. Static placement misses decode traffic: llama.cpp assigns MoE tensors at load time; KTransformers pins a \u201chot\u201d subset. Routing shifts every token, so most expert evaluations fall to the CPU while the GPU and the PCIe link sit idle. Consumer CPUs cannot carry the remainder: Dual-channel DDR5 delivers 80\u201390 GB\/s against the 1\u20131.8 TB\/s an RTX 4090 or 5090 draws from on-package memory. Three mechanisms Bandwidth-adaptive execution (the q* policy): Because DMA transfers and CPU expert execution read from the same host-memory subsystem, a saturated PCIe link leaves a residual bandwidth of B_H \u2212 B_P. FreeToken splits each step\u2019s m cache misses accordingly: q* \u2248 m \u00d7 B_P \/ B_H experts are filled into the GPU cache, the rest are computed in place on the CPU, and the two partial sums merge exactly \u2014 no approximation, no router modification. Both bandwidths are profiled on the deployed machine (ft bench bw), which matters: measured B_P:B_H is 52.7:77.3 on an RTX 5090 server but 11.8:47.5 on a 4060 laptop. Semantic-aware caching: During prefill, full-layer double buffering streams layer l+1 while the GPU computes layer l. Recurrent-state checkpoints are anchored at special-token boundaries \u2014 thinking blocks, tool calls, tool outputs \u2014 precisely where agent harnesses truncate context, so an edit re-prefills only the new suffix. During decode, a shared LRU expert cache spanning all MoE layers follows the router instead of a placement frozen at load time. Elastic memory management: At scheduler safe points the GPU expert cache is rebuilt under a revised VRAM budget without restarting the engine or reloading the host pool. Experts are read from disk straight into their final host layout, then pinned; no GPU warmup is required because the first request is served with a cold cache. Results On an RTX 5090, FreeToken sustains 77\u201383 tok\/s on Qwen3.6-35B-A3B (BF16) and 22\u201325 tok\/s on DeepSeek-V4-Flash (MXFP4) \u2014 1.5\u20132.3\u00d7 the strongest baseline, with decode staying within 12% of the single-turn rate across three agentic workloads. Worst-case TTFT stays below 44 s in every cell; llama.cpp hits 232 s, Ollama 179 s and KTransformers 946 s somewhere in the matrix, past the point where agent clients time out. At equal cache capacity (37% of the Qwen3.6 pool), the global LRU misses 16% of decode-time expert reads against 41% for KTransformers and 62% for llama.cpp. On an 8 GB RTX 4060 laptop the NVFP4 build serves 35B at 39.3 tok\/s \u2014 above the 33 tok\/s median decode speed measured for Codex in production traces. On a single RTX PRO 6000, GLM-5.2 (753B, 40B active) runs at 14.9 tok\/s versus llama.cpp\u2019s 7.3. Data Check Reality Check \u00b7 FlashML FreeToken INFLATION SCORE 59\/100 As of Aug 23, 2026 \u00b7 default mode \u00b7 audited: arXiv:2608.16157, GitHub repo, flashml.ai 3Verified 9Self-rep. 4Misleading 0Contradicted 0Not found Score formula: 8 \u00d7 misleading + 15 \u00d7 contradicted + 3 \u00d7 self-reported, capped at 100. The score is driven by the self-reported column, not by dishonesty \u2014 the code went public six days before this audit, so no independent reproduction exists yet. Claim table \u00b7 16 claims Claim Their number Independent check Verdict &amp; source Decode, Qwen3.6-35B-A3B BF16, RTX 5090 77\u201383 tok\/s None found SELF-REPORTEDPaper Fig 3 Decode, DeepSeek-V4-Flash MXFP4, RTX 5090 22\u201325 tok\/s None found SELF-REPORTEDPaper Fig 3 Decode speedup vs strongest baseline 1.5\u20132.3\u00d7 Recomputes exactly from Fig 3 SELF-REPORTEDPaper \u00a75.2 Decode stability across agent workloads within 12% of W1 None<\/p>","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"pmpro_default_level":"","site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"_pvb_checkbox_block_on_post":false,"footnotes":""},"categories":[52,5,7,1],"tags":[],"class_list":["post-113345","post","type-post","status-publish","format-standard","hentry","category-ai-club","category-committee","category-news","category-uncategorized","pmpro-has-access"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.3 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Meet FreeToken: An Edge-Native MoE Serving Engine that Runs 753B GLM-5.2 on a Single Workstation GPU - YouZum<\/title>\n<meta name=\"description\" content=\"\u0e01\u0e34\u0e08\u0e01\u0e23\u0e23\u0e21\u0e40\u0e01\u0e35\u0e48\u0e22\u0e27\u0e01\u0e31\u0e1a\u0e42\u0e14\u0e23\u0e19\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/youzum.net\/de\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/\" \/>\n<meta property=\"og:locale\" content=\"de_DE\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Meet FreeToken: An Edge-Native MoE Serving Engine that Runs 753B GLM-5.2 on a Single Workstation GPU - YouZum\" \/>\n<meta property=\"og:description\" content=\"\u0e01\u0e34\u0e08\u0e01\u0e23\u0e23\u0e21\u0e40\u0e01\u0e35\u0e48\u0e22\u0e27\u0e01\u0e31\u0e1a\u0e42\u0e14\u0e23\u0e19\" \/>\n<meta property=\"og:url\" content=\"https:\/\/youzum.net\/de\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/\" \/>\n<meta property=\"og:site_name\" content=\"YouZum\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DroneAssociationTH\/\" \/>\n<meta property=\"article:published_time\" content=\"2026-08-24T00:42:28+00:00\" \/>\n<meta name=\"author\" content=\"admin NU\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Verfasst von\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin NU\" \/>\n\t<meta name=\"twitter:label2\" content=\"Gesch\u00e4tzte Lesezeit\" \/>\n\t<meta name=\"twitter:data2\" content=\"8\u00a0Minuten\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/youzum.net\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/youzum.net\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/\"},\"author\":{\"name\":\"admin NU\",\"@id\":\"https:\/\/yousum.gpucore.co\/#\/schema\/person\/97fa48242daf3908e4d9a5f26f4a059c\"},\"headline\":\"Meet FreeToken: An Edge-Native MoE Serving Engine that Runs 753B GLM-5.2 on a Single Workstation GPU\",\"datePublished\":\"2026-08-24T00:42:28+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/youzum.net\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/\"},\"wordCount\":1619,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/yousum.gpucore.co\/#organization\"},\"articleSection\":[\"AI\",\"Committee\",\"News\",\"Uncategorized\"],\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/youzum.net\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/youzum.net\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/\",\"url\":\"https:\/\/youzum.net\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/\",\"name\":\"Meet FreeToken: An Edge-Native MoE Serving Engine that Runs 753B GLM-5.2 on a Single Workstation GPU - YouZum\",\"isPartOf\":{\"@id\":\"https:\/\/yousum.gpucore.co\/#website\"},\"datePublished\":\"2026-08-24T00:42:28+00:00\",\"description\":\"\u0e01\u0e34\u0e08\u0e01\u0e23\u0e23\u0e21\u0e40\u0e01\u0e35\u0e48\u0e22\u0e27\u0e01\u0e31\u0e1a\u0e42\u0e14\u0e23\u0e19\",\"breadcrumb\":{\"@id\":\"https:\/\/youzum.net\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/#breadcrumb\"},\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/youzum.net\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/youzum.net\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/youzum.net\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Meet FreeToken: An Edge-Native MoE Serving Engine that Runs 753B GLM-5.2 on a Single Workstation GPU\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/yousum.gpucore.co\/#website\",\"url\":\"https:\/\/yousum.gpucore.co\/\",\"name\":\"YouSum\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/yousum.gpucore.co\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/yousum.gpucore.co\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"de\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/yousum.gpucore.co\/#organization\",\"name\":\"Drone Association Thailand\",\"url\":\"https:\/\/yousum.gpucore.co\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\/\/yousum.gpucore.co\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/youzum.net\/wp-content\/uploads\/2024\/11\/tranparent-logo.png\",\"contentUrl\":\"https:\/\/youzum.net\/wp-content\/uploads\/2024\/11\/tranparent-logo.png\",\"width\":300,\"height\":300,\"caption\":\"Drone Association Thailand\"},\"image\":{\"@id\":\"https:\/\/yousum.gpucore.co\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/DroneAssociationTH\/\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/yousum.gpucore.co\/#\/schema\/person\/97fa48242daf3908e4d9a5f26f4a059c\",\"name\":\"admin NU\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\/\/yousum.gpucore.co\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/youzum.net\/wp-content\/uploads\/avatars\/2\/1746849356-bpfull.png\",\"contentUrl\":\"https:\/\/youzum.net\/wp-content\/uploads\/avatars\/2\/1746849356-bpfull.png\",\"caption\":\"admin NU\"},\"url\":\"https:\/\/youzum.net\/de\/members\/adminnu\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Meet FreeToken: An Edge-Native MoE Serving Engine that Runs 753B GLM-5.2 on a Single Workstation GPU - YouZum","description":"\u0e01\u0e34\u0e08\u0e01\u0e23\u0e23\u0e21\u0e40\u0e01\u0e35\u0e48\u0e22\u0e27\u0e01\u0e31\u0e1a\u0e42\u0e14\u0e23\u0e19","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/youzum.net\/de\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/","og_locale":"de_DE","og_type":"article","og_title":"Meet FreeToken: An Edge-Native MoE Serving Engine that Runs 753B GLM-5.2 on a Single Workstation GPU - YouZum","og_description":"\u0e01\u0e34\u0e08\u0e01\u0e23\u0e23\u0e21\u0e40\u0e01\u0e35\u0e48\u0e22\u0e27\u0e01\u0e31\u0e1a\u0e42\u0e14\u0e23\u0e19","og_url":"https:\/\/youzum.net\/de\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/","og_site_name":"YouZum","article_publisher":"https:\/\/www.facebook.com\/DroneAssociationTH\/","article_published_time":"2026-08-24T00:42:28+00:00","author":"admin NU","twitter_card":"summary_large_image","twitter_misc":{"Verfasst von":"admin NU","Gesch\u00e4tzte Lesezeit":"8\u00a0Minuten"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/youzum.net\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/#article","isPartOf":{"@id":"https:\/\/youzum.net\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/"},"author":{"name":"admin NU","@id":"https:\/\/yousum.gpucore.co\/#\/schema\/person\/97fa48242daf3908e4d9a5f26f4a059c"},"headline":"Meet FreeToken: An Edge-Native MoE Serving Engine that Runs 753B GLM-5.2 on a Single Workstation GPU","datePublished":"2026-08-24T00:42:28+00:00","mainEntityOfPage":{"@id":"https:\/\/youzum.net\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/"},"wordCount":1619,"commentCount":0,"publisher":{"@id":"https:\/\/yousum.gpucore.co\/#organization"},"articleSection":["AI","Committee","News","Uncategorized"],"inLanguage":"de","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/youzum.net\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/youzum.net\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/","url":"https:\/\/youzum.net\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/","name":"Meet FreeToken: An Edge-Native MoE Serving Engine that Runs 753B GLM-5.2 on a Single Workstation GPU - YouZum","isPartOf":{"@id":"https:\/\/yousum.gpucore.co\/#website"},"datePublished":"2026-08-24T00:42:28+00:00","description":"\u0e01\u0e34\u0e08\u0e01\u0e23\u0e23\u0e21\u0e40\u0e01\u0e35\u0e48\u0e22\u0e27\u0e01\u0e31\u0e1a\u0e42\u0e14\u0e23\u0e19","breadcrumb":{"@id":"https:\/\/youzum.net\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/#breadcrumb"},"inLanguage":"de","potentialAction":[{"@type":"ReadAction","target":["https:\/\/youzum.net\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/youzum.net\/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/youzum.net\/"},{"@type":"ListItem","position":2,"name":"Meet FreeToken: An Edge-Native MoE Serving Engine that Runs 753B GLM-5.2 on a Single Workstation GPU"}]},{"@type":"WebSite","@id":"https:\/\/yousum.gpucore.co\/#website","url":"https:\/\/yousum.gpucore.co\/","name":"YouSum","description":"","publisher":{"@id":"https:\/\/yousum.gpucore.co\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/yousum.gpucore.co\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"de"},{"@type":"Organization","@id":"https:\/\/yousum.gpucore.co\/#organization","name":"Drone Association Thailand","url":"https:\/\/yousum.gpucore.co\/","logo":{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/yousum.gpucore.co\/#\/schema\/logo\/image\/","url":"https:\/\/youzum.net\/wp-content\/uploads\/2024\/11\/tranparent-logo.png","contentUrl":"https:\/\/youzum.net\/wp-content\/uploads\/2024\/11\/tranparent-logo.png","width":300,"height":300,"caption":"Drone Association Thailand"},"image":{"@id":"https:\/\/yousum.gpucore.co\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DroneAssociationTH\/"]},{"@type":"Person","@id":"https:\/\/yousum.gpucore.co\/#\/schema\/person\/97fa48242daf3908e4d9a5f26f4a059c","name":"admin NU","image":{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/yousum.gpucore.co\/#\/schema\/person\/image\/","url":"https:\/\/youzum.net\/wp-content\/uploads\/avatars\/2\/1746849356-bpfull.png","contentUrl":"https:\/\/youzum.net\/wp-content\/uploads\/avatars\/2\/1746849356-bpfull.png","caption":"admin NU"},"url":"https:\/\/youzum.net\/de\/members\/adminnu\/"}]}},"rttpg_featured_image_url":null,"rttpg_author":{"display_name":"admin NU","author_link":"https:\/\/youzum.net\/de\/members\/adminnu\/"},"rttpg_comment":0,"rttpg_category":"<a href=\"https:\/\/youzum.net\/de\/category\/ai-club\/\" rel=\"category tag\">AI<\/a> <a href=\"https:\/\/youzum.net\/de\/category\/committee\/\" rel=\"category tag\">Committee<\/a> <a href=\"https:\/\/youzum.net\/de\/category\/news\/\" rel=\"category tag\">News<\/a> <a href=\"https:\/\/youzum.net\/de\/category\/uncategorized\/\" rel=\"category tag\">Uncategorized<\/a>","rttpg_excerpt":"Frontier open-weight models are shipping faster than the hardware assumptions around them. Kimi-K3, GLM-5.2 and DeepSeek-V4-Flash are closing the capability gap with proprietary systems, but releasing parameters only determines who can obtain a model \u2014 not who can afford to run it. Serving them still assumes datacenter-class GPU clusters, and as agentic workloads push inference&hellip;","_links":{"self":[{"href":"https:\/\/youzum.net\/de\/wp-json\/wp\/v2\/posts\/113345","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/youzum.net\/de\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/youzum.net\/de\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/youzum.net\/de\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/youzum.net\/de\/wp-json\/wp\/v2\/comments?post=113345"}],"version-history":[{"count":0,"href":"https:\/\/youzum.net\/de\/wp-json\/wp\/v2\/posts\/113345\/revisions"}],"wp:attachment":[{"href":"https:\/\/youzum.net\/de\/wp-json\/wp\/v2\/media?parent=113345"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/youzum.net\/de\/wp-json\/wp\/v2\/categories?post=113345"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/youzum.net\/de\/wp-json\/wp\/v2\/tags?post=113345"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}