AI Weekly Review 2026-10-05
Week In Review
The week’s most striking results came from mathematics and the physical sciences, where the question shifted from whether models can do research to which research they are shaped for. Meta’s research group published six papers solving open mathematical problems with Muse Spark, each marking which passages the model drafted and which the human did. A Harvard physicist argued in Claude-shaped science that the productive move is not to make a model behave like a scientist but to hunt for problems whose shape fits what the model already does well, and reported results across physics, ecology, genetics, and linguistics to support it. Both pieces describe the same division of labour: humans supply the conceptual framing and the judgment about what matters, machines supply breadth of technique and tireless exactness.
Underneath the science sat a busy week of plumbing, all of it pointed at the same target of getting more capability per unit of compute. Google released Gemini 4 Argon with a million-token output ceiling and introductory pricing well below its rivals. A training-free decoding trick called LoopCD extracted large gains from looped transformers by contrasting their own early and late passes, and Allen Institute for AI’s Olmo-core 3 published an open stack for training trillion-parameter sparse models. Amazon’s Strands Decider 2B continued a fast-moving shift toward tiny models that return calibrated probabilities over fixed choices instead of prose. Each attacks cost from a different layer: the frontier model, the decoder, the training system, and the small components that surround a large agent.
Two papers converged on memory as the next real constraint on agents. Context Language Models let a model treat its own context as an editable file and decide what deserves to survive, improving accuracy while using fewer floating-point operations. Google Research and Virginia Tech’s WikiSkill keeps an exhaustive wiki of past failures outside the production prompt and distills it into compact executable skills. The shared insight is that an agent’s accumulated experience and its working prompt have opposite requirements, and that separating them cheaply beats stuffing everything into the window.
The infrastructure questions arrived from both the physical and the economic end. Google and Planet put a prototype satellite carrying tensor processing units into orbit to find out whether machine-learning hardware can survive and eventually scale in space. Epoch AI’s estimate of the agent population ran the arithmetic in the other direction, concluding that memory shipped through 2027 could support tens of millions of concurrent frontier agents and that finding enough work for them, rather than building the capacity, may be the harder problem. Taken together with the week’s efficiency results, the two pieces frame the same tension: the supply of machine cognition is growing faster than the catalogue of things we know how to ask it to do.
Items
Google’s Gemini 4 Argon Retakes the Benchmark Lead on Breadth Rather Than Dominance
Google announced Gemini 4 Argon on September 30, its first new flagship since Gemini 3 arrived in November 2025, and positioned it around long-horizon professional work rather than chat. The headline capability is an output ceiling of one million tokens, up from 64,000 in the previous generation. That is a change in kind as much as degree: a model that can emit a million tokens in one response can produce an entire codebase, a full document set, or a lengthy analysis without the orchestration machinery usually needed to stitch together bounded chunks.
VentureBeat reports that Argon leads or ties on 13 of 18 disclosed benchmarks, a wider spread than any competitor currently claims. The most dramatic margin is on Harvey’s legal agent benchmark, where Argon scored 19.6% against 5.4% for GPT-6 Astra and 3.8% for Claude Opus 5.5. It also leads on AutomationBench at 51.3% against 42.5% for Opus 5.5, on the DeepSWE v1.1 coding benchmark at 77.9%, and on the LVBench video understanding test at 91.7%. Those legal-agent numbers are worth reading carefully: a score under 20% means the task remains mostly unsolved, and Argon’s advantage is that it fails less completely than the alternatives.
The lead is not uniform. GPT-6 Astra holds larger margins on FrontierSWE v2 and on Terminal-Bench science tasks, which suggests the frontier is now genuinely multidimensional rather than ordered on a single axis. Google’s framing of breadth over dominance is an accurate description of what the numbers show.
Pricing is the other half of the story. Introductory rates are $2 per million input tokens and $10 per million output, with cached input at $0.10, a 95% discount that rewards workloads which reread the same large corpus. At those rates Argon undercuts GPT-6 Astra by a factor of five and Claude Opus 5.5 by half. Post-introductory pricing doubles to $4 and $20. Availability is staged, beginning with cyber defenders through Google’s Fairwind Program and the United States government’s voluntary pre-release access process, then paid API customers and Google AI Ultra subscribers.
Source: VentureBeat
Mathematicians and Meta’s Muse Spark Co-Author Six Papers on Open Problems
Meta’s research group published six mathematics papers on October 2, each produced by a mathematician working with Muse Spark 1.1 or 1.2 in its Thinking Mode, and each settling a question that was genuinely open. The range is unusually wide for a single release, covering probability, differential equations, group theory, polynomial optimization, arithmetic physics, and non-associative algebra.
Several of the results are the kind of clean verdict that mathematics prizes. In group theory, Joseph Brennan and Milana Golich disproved a 2024 conjecture about semiabelian groups by exhibiting a counterexample with 384 elements. Andres Barei similarly demolished a conjecture about solvable evolution algebras using a three-dimensional counterexample. Counterexample hunting is exactly the sort of task where machine breadth helps most: the search space is combinatorially large, the check is mechanical, and a single hit ends the question permanently.
Other papers are constructive rather than destructive. Aykut Arslan established a sharp threshold for fitting random points to ellipsoids in high dimensions, a result that speaks to the limits of how much data a given geometric model can absorb, and separately answered a 2026 question about when cycle-based relaxations of polynomial optimization problems are exact. Leonard Dinh proved finite-time blow-up for radial negative-energy solutions in a wave model drawn from physics, settling a question posed in 2015. A five-author paper connected number theory to p-adic string theory, extending ideas from Yuri Manin’s 1980s work to a broader class of curves.
The methodological care is as notable as the results. Mathematicians directed the research and the model helped develop proofs, separate mathematicians reviewed the work, and the papers explicitly mark which passages were drafted by the model and which by the researcher. Meta describes the goal as helping researchers “develop mathematical insights that others can understand and build on,” which is a narrower and more defensible claim than autonomous discovery. The provenance marking also creates something the field will need as this becomes common, namely a record that lets reviewers calibrate how much scrutiny each section deserves.
Source: Meta AI Research
Finding the Problems That Fit the Model: A Physicist’s Case for Claude-Shaped Science
Harvard physicist Matthew Schwartz published a guest post on Anthropic’s research blog on October 1 describing what he calls an impedance mismatch between what scientists want from language models and what the models actually do well. His diagnosis is blunt: “Claude and GPT are good at science, but they are not scientists: yes, they are smart, but it can take a lot of hand-holding to produce anything of scientific value.” His response was to stop trying to fix the model and start choosing different problems. “Instead of treating Claude like the collaborator I wanted it to be, I started to treat it like the collaborator it actually is.”
A Claude-shaped problem, in his account, is a quantitative question where the needed mathematical technique already exists somewhere in the literature but is not widely known or implemented in the field where it applies. Those problems reward exactly what a model is good at, namely enormous reading breadth, fluent coding, and patience with high-precision computation. The tooling he built around this is BootLoops, a harness that began as a way to do semi-numerical bootstrap calculations for scattering amplitudes and grew into an extensible toolkit. Its method combines physical constraints with high-precision numerics so that answers are exact rather than approximate: as Schwartz puts it, “if few enough options remain, those numbers pin down the remaining coefficients exactly.”
The results span fields that rarely appear in the same paragraph. In physics and mathematics, the toolkit computed 15 previously unsolved elliptic Feynman integrals and solved Watson’s “final problem” on three-dimensional random walk probabilities. In ecology it solved Etienne’s twenty-year-old neutral biodiversity equation, then used it to show that tree species composition at Barro Colorado Island turns over 4.5 times faster than neutral theory predicts. In population genetics it analysed 5.7 billion mutation pairs from the 1000 Genomes Project and found evidence for gene conversion mechanisms that earlier studies of linked variation had passed over.
The list continues into phylogenetics, where Bayesian evidence calculations became fast and exactly checkable, economics, where replication packages from 4,452 papers comprising some 30,000 routines were converted to open-source code, linguistics, where a database of word-stress patterns across 6,072 languages was assembled, and earth science, with a predictive model of the Great Oxidation Event spanning four glaciations.
Schwartz’s conclusion resists triumphalism. “Much of the technical work of solving it can now be automated. That is liberating, in a way. But humans are still needed for the conceptual part.” He adds that the exercise clarified the other half of the picture: focusing on Claude-shaped science “helps us appreciate what human-shaped science looks like.”
Source: Anthropic
Google and Planet Put Tensor Processing Units Into Orbit
Google announced on October 1 that a prototype satellite for Project Suncatcher, built with Planet and carrying tensor processing units, had launched aboard SpaceX’s Transporter-18 rideshare mission and is operating as expected in orbit. Contact has been confirmed and the payload is functioning as planned. The announcement is deliberately modest about what has been proven so far: no performance data has been released, and the stated purpose is to learn whether space “could one day host scalable machine learning infrastructure.”
What the mission is actually testing is survival. Accelerator hardware designed for climate-controlled data centres faces three hostile conditions in orbit: launch stresses, ionising radiation that corrupts memory and logic, and thermal cycling between sunlight and shadow with no air to carry heat away. Google says the team will gather in-orbit data on how the processors withstand all three, then use it to refine designs. Those are the unknowns that determine whether the idea is an engineering problem or a dead end, and they cannot be settled on the ground.
The appeal of the concept is power and cooling. A satellite in a suitable orbit can receive sunlight nearly continuously, without the land, grid connection, or water that terrestrial data centres compete for. The constraints are equally plain, chiefly the difficulty of moving data to and from orbit at the bandwidths training and inference demand, and the impossibility of sending a technician to replace a failed board.
Google reports that a peer-reviewed paper describing the underlying research has been published in the journal Joule, which places the concept in the energy-systems literature rather than the computing literature. That is a telling choice of venue. The question Suncatcher is ultimately asking is not whether processors can compute in space, which is not in doubt, but whether the full system of solar collection, thermal rejection, and data transport closes favourably against building the same capacity on the ground. One prototype will not answer that. It will establish whether the hardware lasts long enough for the question to be worth asking at scale.
Source: Google
Olmo-core 3 Publishes an Open Stack for Training Trillion-Parameter Sparse Models
The Allen Institute for AI released Olmo-core 3 on October 1, a rebuilt version of its open training framework aimed squarely at mixture-of-experts models at trillion-parameter scale. The practical problem it solves is that a sparse model’s total parameter count can far exceed what any single accelerator can hold, so the model, the optimizer state, and the training pipeline all have to be split across a cluster without the coordination overhead eating the benefit. Olmo-core 3 combines expert parallelism, pipeline parallelism, and a distributed optimizer to do that.
The throughput numbers are substantial. On eight B300 accelerators the framework reached roughly 52,000 tokens per second per GPU, against 19,400 for the previous version, about 2.7 times higher. A scaling test grew the expert pool from 8 to 128 while holding active parameters per token near 3.2 billion, raising total model capacity from 4.6 billion to 47 billion parameters at a cost of less than 5% in throughput. That ratio is the entire argument for sparse models: capacity grows roughly tenfold while the compute spent on each token barely moves.
At the top end, Ai2 benchmarked a 1.2-trillion-parameter configuration across 512 GPUs and sustained 858 teraflops per second per GPU. The team also reports that training in the MXFP8 numerical format delivered 21% higher throughput than a BF16 baseline while cutting peak memory from 103 to 95 gibibytes, a reminder that low-precision formats are now a first-order lever on training cost rather than a late-stage optimization.
The release is fully open source and available on GitHub, intended for researchers to adapt to different hardware and to swap components. That matters because the engineering required to train large sparse models efficiently has been among the least documented parts of frontier practice, typically learned inside well-funded labs and rarely written down. Publishing a working stack with measured numbers moves that knowledge into the commons, where a university group or a small company can start from a known-good baseline instead of rediscovering it.
Source: Allen Institute for AI
Context Language Models Let a Model Edit Its Own Working Memory
A team including Rulin Shao, Nathan Lambert, Mike Lewis, Luke Zettlemoyer, and Pang Wei Koh posted a paper on September 29 introducing Context Language Models, which hand the management of context to the model itself. The usual arrangement wraps a model in external machinery that decides what to retrieve, what to summarise, and what to discard. The paper’s proposal is to treat the context as an editable file the model can rewrite directly, so that it decides what is worth keeping as it goes. The authors note the approach extends naturally to multi-agent settings, where several models share and revise the same working state.
The reported gains come with lower rather than higher compute, which is the result’s most interesting feature. On BrowseComp-Plus the method delivered 11.4% higher accuracy using 21.5% fewer floating-point operations. On a 12-hour EdgeBench run it scored 5% higher with 59% fewer operations. On a 24-hour multi-repository task involving a swarm of agents it produced 65% greater improvement at equal compute. The pattern makes sense: a model that prunes its own context stops paying attention costs on material it has judged irrelevant.
The paper describes two routes to the behaviour. One is in-context adaptation, where natural-language instructions about how to manage the file yield improvements of up to 35.9 accuracy points without any training. The other is parametric learning, where reinforcement learning taught Qwen3.5-9B the skill directly, raising its BrowseComp-Plus performance by 47.6% while using 12% less compute. The authors also report a systems-level companion optimization called Suffix Cache Reuse, which cuts server-side compute by 35% relative to standard SGLang serving by exploiting the structure of these edits.
The broader significance is architectural. Long-context models made it possible to put more into the window; this line of work questions whether putting more in is the right goal. If a model can be trained to curate its own state, then the bottleneck on long-running agent work shifts from context length to context judgment, and the enormous engineering effort currently spent on retrieval pipelines around the model becomes partly redundant.
Source: arXiv
WikiSkill Gives Agents a Searchable Memory of Their Own Failures
Researchers from Google Research and Virginia Tech published WikiSkill, a framework that lets agents accumulate and reuse hard-won experience instead of rediscovering the same failures on every run. The design turns on a tension the authors state directly: “memory wants to be exhaustive and a production prompt wants to be lean.” WikiSkill resolves it by keeping the memory entirely outside the inference agent’s context.
The system has three layers. A raw layer stores immutable execution traces: actions, reasoning, tool calls, outputs, results. A wiki layer digests those traces into structured knowledge, including catalogued failure modes, successful strategies, evolution logs, and records of which skills actually changed outcomes. A skill layer holds executable procedural instructions that link back to the wiki patterns that justify them. Only that last compact layer touches the production prompt, which the paper puts at 45 to 129 lines per cycle, keeping serving costs close to an agent with no memory at all.
Evaluated across five domains covering mathematical reasoning, web search, spreadsheets, document question answering, and interactive household tasks, WikiSkill beat competing approaches named Trace2Skill, EvoSkill, and SkillOpt by margins from 3.3 to 12 percentage points. The more interesting finding is how the benefit scales with model capability: Qwen models improved by 12.3, 17.5, and 23.9 points at 4 billion, 9 billion, and 27 billion parameters respectively. A stronger model extracts more from the same accumulated record, which suggests the wiki is capturing genuine structure rather than patching specific weaknesses.
Skills also transfer between models. Qwen-3.5-9B reached 70.2% accuracy on the ALFWorld household-task benchmark using skills evolved by the larger Qwen-3.6-27B, compared with 63.4% using skills it evolved itself. That result points at an appealing division of labour, where an expensive model does the exploring and writes down what it learns, and cheaper models inherit the lessons. Liyan Tang, a research scientist at Google, is among the co-authors.
Source: VentureBeat
LoopCD Extracts Large Reasoning Gains by Contrasting a Model With Its Earlier Self
A paper posted on October 1 by Weihao Liu, Huangjie Zheng, Tianrong Chen, Rohit Dilip, Richard He Bai, Yizhu Jiao, Yuyang Wang, and Ruixiang Zhang describes LoopCD, a decoding method that needs no training and no second model. Its target is the looped transformer, an architecture that reuses the same block of computation across several recurrent passes instead of stacking distinct layers, trading parameters for repeated computation.
The authors’ observation is that standard decoding throws away most of what a looped model computes. Each pass produces a prediction, but only the last one is read. Earlier passes are systematically weaker, which means a looped model generates weak-strong prediction pairs for free, exactly the ingredient that contrastive decoding normally has to obtain by running a separate smaller model alongside the main one.
LoopCD exploits this in two variants. LoopCD-Logits contrasts the final prediction against an earlier recurrent pass at the cost of one extra output pass. LoopCD-Hidden performs the same contrast in hidden-state space with what the paper describes as zero output overhead. The reported gains are large for a method that changes no weights: Ouro-2.6B-Thinking rose from 61.88% to 73.33% pass@1 on AIME 2024, and the Huginn model went from 22.56% to 31.71% on HumanEval.
The efficiency claim is the more consequential one. Because the contrast sharpens predictions, the authors report they can halve the number of recurrent loops and still match full-depth performance, cutting forward floating-point operations by 22.5% to 48.2%. Looped architectures are attractive precisely because they let a small parameter count buy deep computation, which suits memory-constrained deployment; a free decoding change that nearly halves their runtime cost makes that trade more favourable still.
Source: arXiv
Amazon’s Strands Decider 2B Pushes the Decision-Model Wave Into Open Weights
Amazon released Strands Decider 2B on October 1, a roughly 2-billion-parameter model built on Alibaba’s Qwen that does not write prose. It answers bounded questions by scoring a fixed set of allowed choices and returning probabilities over them in a single pass. The use case is the enormous volume of small judgments that surround any serious agent deployment: route this request, approve or reject this action, rank these candidates, decide whether this output passed.
Those decisions have been made by prompting a general-purpose model to emit a word, which is a slow and poorly calibrated way to get a probability. Strands Decider reports median latency of 106 milliseconds and 296 milliseconds at the 95th percentile on a consumer Nvidia RTX 3090, accuracy around 72% on the public JevBench suite, and a Brier score of 0.35, where lower indicates better-calibrated confidence. The calibration number matters more than the accuracy number, because a decision component whose stated confidence can be trusted lets the surrounding system decide when to escalate to an expensive model.
The release is downloadable from Hugging Face under an Apache 2.0 license with training materials, code, and scripts published alongside, though users must supply their own compute. VentureBeat’s assessment is that Strands does not win on benchmarks; its advantage is reproducibility and openness, which lets a team retrain the component on its own decision boundaries rather than accepting someone else’s.
The speed of the category’s emergence is the striking part. TypeSafe introduced Jev in mid-September, and the field now includes Laya, Kev, Bespoke Nimble, Mapika’s decider family, and FLock’s this-that-model, with Amazon arriving as the first large cloud vendor to ship an open one. A new model class going from a single product to a crowded open-weights market in under three weeks says something about how little architectural novelty the idea required and how much latent demand was waiting for it.
Source: VentureBeat
Epoch AI Counts the Agents the World’s Memory Can Hold
Jason Li published an analysis for Epoch AI on October 2 asking a question the infrastructure debate usually skips: given the hardware actually being shipped, how many AI agents can run at the same time? The method is deliberately simple, multiplying effective hardware supply by serving capacity per unit. Supply is measured in GB300-equivalent inference units derived from high-bandwidth memory shipments from 2025 onward, adjusted for newer hardware; capacity per unit comes from inferring concurrency for closed models from API spending against agent session traces, and from direct serving benchmarks for open models.
The choice to count memory rather than raw compute is the analysis’s central argument. High-bandwidth memory is the binding constraint on concurrency because it must hold both model weights and the key-value caches that grow with every token of context, and memory bandwidth limits how fast tokens can be produced once streaming dominates. Li counts HBM3E and HBM4 generations in 288-gigabyte units matching GB300 specifications, assuming newer HBM4 and HBM4E systems support roughly twice as many concurrent agents per gigabyte.
The headline estimate is that memory shipped through 2027 could support 30 to 170 million concurrent agents running today’s most capable models, or billions using efficient open models such as DeepSeek V4 Pro. Converted into labour terms, and accounting for the fact that an agent works all 168 hours in a week against a human’s 40, that is roughly 140 to 720 million full-time-equivalent workers of capacity.
The conclusion cuts against the prevailing assumption that compute is the scarce input. Li calculates that merely using 20% of projected capacity would require $2.6 to $5.3 trillion a year in API-equivalent spending by the end of 2027, against roughly $1 trillion in projected developer revenue on current growth. The analysis therefore flags oversupply as the live risk: “the compute buildout could run ahead of inference demand.” For that not to happen, adoption and the invention of new uses would have to grow at rates without precedent, which places the burden on the demand side of the industry rather than the fabs.
Source: Epoch AI