Daily Papers · 2026-10-16
📄 论文日报 | 来源：Hugging Face Daily Papers
🗓 Episode：2026.10.15 | ATAS记忆助智能体视频；Player-1合成经验强游戏
1. 任意时间、任意空间：以世界状态为中心的智能体视频生成框架
- 英文标题：Any Time, Any Space: A World-State-Centric Framework for Agentic Video Generation
- arXiv：2610.20571
💡 一句话解读：现有AI把剧本变成视频时，前后镜头常常对不上——换个时间重拍同一场景，布景会变；回放早先事件，细节会乱。ATAS的做法是给每个镜头标注地点、机位和事件时间，生成前从“场景记忆”和“事件记忆”里找参考，生成后再把新片段登记回记忆，从而保证跨镜头一致。
长篇智能体视频生成必须把剧本转化为在非连续请求间仍保持空间与时间一致的镜头。仅靠传递提示词或最近帧，无法为重新访问某场景提供共享空间参照，也无法为回放早先事件提供按时间索引的记录。我们提出“任意时间、任意空间”（ATAS），它将剧本编译为带有明确地点、相机和事件时间引用的镜头请求。生成前，它从已注册静态几何的场景记忆和按场景、事件与时间索引的动态观测事件记忆中检索相关证据；生成后，ATAS重建并注册该片段，仅在审查与状态检查接受该观测时更新记忆，而影片交付不依赖于记忆准入。在四种视频生成器上，ATAS提升了所有报告的跨镜头一致性指标，并在每个骨干模型上取得最高综合基准分。它还在目标图像空间重访上优于所比较的智能体与3D引导方法，在三个事件重捕获标准上领先，并在全部六项标准上获得最高人工评分。视频结果与代码见项目主页：https://lhmd.top/atas/。
📄 English Abstract
Long-form agentic video generation must turn a screenplay into shots that remain spatially and temporally consistent across nonconsecutive requests. Passing prompts or recent frames between shots does not provide a shared spatial frame for revisiting a scene or a time-indexed record for replaying an earlier event. We present Any Time, Any Space (ATAS), which compiles a screenplay into shot requests with explicit location, camera, and event-time references. Before generation, it retrieves relevant evidence from a scene memory of registered static geometry and an event memory of dynamic observations indexed by scene, event, and time. After generation, ATAS reconstructs and registers the clip. It updates memory only when review and state checks accept the observation, while film delivery remains independent of memory admission. Across four video generators, ATAS improves all reported cross-shot consistency measures and achieves the highest aggregate benchmark score for each backbone. It also outperforms the compared agents and 3D-guided methods on target-image spatial revisit, leads three event-recapture criteria, and receives the highest human ratings on all six criteria. The video results and code are available on our project page: https://lhmd.top/atas/.
2. Player-1：从合成经验中学习游戏
- 英文标题：Player-1: Learning to Play from Synthetic Experience
- arXiv：2610.20584
💡 一句话解读：现有游戏AI主要靠模仿人类操作学习，但遇到新情况就抓瞎。本文提出Player-1，先让AI看大量游戏录像入门，再把它放进16个专门设计的虚拟游戏里自己摸索，通过试错和奖励机制提升操作水平，最终在多个游戏测试中成功率大幅提高。
计算机使用智能体在解读屏幕和执行键鼠操作方面已展现出强大能力。然而，游戏因其视觉丰富的世界和多样化的机制，仍具挑战性，是迈向通用数字控制的下一个严苛台阶。当前的游戏智能体学习流程往往依赖大规模演示，对动态游戏状态的覆盖有限，而可扩展的交互式训练环境仍难以获取。我们提出Player-1，一个通过合成经验在演示之外学习游戏控制的训练后框架。我们扩展了计算机使用动作空间，加入相对指针移动、持续输入和复合动作。在监督初始化阶段，我们在6044条精心策划的游戏轨迹上微调计算机使用模型，这些轨迹将录制的动作与任务指令和视觉推理配对。我们构建了Player-1-Gym，包含十六个围绕游戏间共享控制需求设计的合成2D和3D环境。程序化可验证的奖励以及随机化的观察和控制动态支持使用PPO进行多环境强化学习。由此得到的Player-1-9B和Player-1-35B-A3B在MineDojo-300上的成功率分别比其SFT策略提高了27.8和17.7个百分点。Player-1-9B达到38.33%的成功率，而最强评估基线为12.63%，在ViZDoom-10和GameWorld上也比SFT有进一步提升。进一步分析表明，更广泛的合成环境覆盖能改善迁移，而结合多个键盘和鼠标输入的复合动作的使用在强化学习过程中增加。我们证明，从合成经验中学习能显著提升游戏控制能力，超越监督初始化，且这些提升能泛化到强化学习训练套件之外。
📄 English Abstract
Computer-use agents have demonstrated strong capabilities in interpreting screens and executing keyboard and mouse actions. However, games, with their visually rich worlds and diverse mechanics, remain challenging and offer a demanding next step toward general digital control. Current game-agent learning pipelines often rely on large-scale demonstrations with limited coverage of dynamic game states, while scalable environments for interactive training remain difficult to obtain. We introduce Player-1, a post-training framework for learning game control beyond demonstrations through synthetic experience. We extend the computer-use action space with relative pointer motion, sustained inputs, and composite actions. For supervised initialization, we fine-tune computer-use models on 6,044 curated gameplay trajectories that pair recorded actions with task instructions and visual reasoning. We construct Player-1-Gym, sixteen synthetic 2D and 3D environments designed around control demands shared across games. Programmatically verifiable rewards and randomized observations and control dynamics support multi-environment reinforcement learning with PPO. The resulting Player-1-9B and Player-1-35B-A3B improve MineDojo-300 success over their SFT policies by 27.8 and 17.7 percentage points, respectively. Player-1-9B reaches 38.33% success compared with 12.63% for the strongest evaluated baseline, with further gains over SFT on ViZDoom-10 and GameWorld. Further analyses show that broader synthetic environment coverage improves transfer, while the use of composite actions that combine multiple keyboard and mouse inputs increases during RL. We demonstrate that learning from synthetic experience substantially improves game control beyond supervised initialization, with gains that generalize beyond the RL training suite.
3. Doc2Harness：评测大语言模型在编码智能体长时程Harness工程中的表现
- 英文标题：Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents
- arXiv：2610.20344
💡 一句话解读：现有评测只测模型在固定“外挂框架”下的任务表现，没测模型自己搭建这套框架的能力。本文提出Doc2Harness：先让模型根据接口和文档造出一个完整框架，再用脚本模拟模型跑328个测试，从请求、文件、退出码、输出四方面打分。最强模型通过率仅53%，说明长时程框架工程仍很难。
由大语言模型驱动的智能体越来越多地被部署到长时程任务中，它们使用工具并修改真实工作空间。其表现好坏既取决于模型本身，也取决于模型周围的“外挂框架”（harness）。现有基准在固定框架下衡量下游任务成功率，因而框架工程本身未被探索。我们提出 Doc2Harness，一个两阶段基准，直接评测长时程框架工程。第一阶段，一个构建模型在固定构建器框架内，接收仅含接口的契约骨架和一份整合了十二个前沿框架行为的开发文档，然后在隔离容器中构建出完整的候选框架。第二阶段，冻结的候选框架在与构建环境分离的容器中运行，每个测试用例一个容器，并由脚本化的模拟模型端到端驱动。我们通过四个可观测通道对每个测试用例打分，即发送给模型的请求、磁盘上的文件、进程退出码和命令输出；阳性对照排除了空洞通过。在十九个前沿模型上，每个模型在328个测试用例上进行三次独立构建评估，GPT-6 Astra 以53.05%的通过率最高，最弱模型为27.44%，差距约二十六个百分点。即使最强模型也未能通过近一半的测试用例，失败集中在跨领域功能如何拆分到各模块上。据我们所知，Doc2Harness 提供了首个用于衡量长时程框架工程的确定性基准。
📄 English Abstract
Agents driven by large language models are increasingly deployed on long-horizon tasks, where they use tools and modify real workspaces. How well they perform depends as much on the harness around the model as on the model itself. Existing benchmarks measure downstream task success under a fixed harness, leaving harness engineering itself unexplored. We propose Doc2Harness, a two-stage benchmark that evaluates long-horizon harness engineering directly. In Stage 1, a build model operating inside a fixed builder harness receives an interface-only contract skeleton and a development document that consolidates behavior across twelve frontier harnesses, and then constructs a complete candidate harness in an isolated container. In Stage 2, the frozen candidate harness runs in containers separate from the construction environment, one per test case, and is driven end to end by a scripted mock model. We score each test case using four observable channels, namely the requests sent to the model, the files on disk, the process exit code, and the command output; positive controls rule out vacuous passes. Across nineteen frontier models, each evaluated with three independent builds on 328 test cases, GPT-6 Astra achieves the highest pass rate at 53.05%, while the weakest model reaches 27.44%, a gap of roughly twenty-six percentage points. Even the strongest model fails nearly half of the test cases, and failures cluster around how cross-cutting functionality is split across modules. To our knowledge, Doc2Harness provides the first deterministic benchmark for measuring long-horizon harness engineering.
4. 超越领域路由监督：多教师同策略蒸馏的跨教师守护
- 英文标题：Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation
- arXiv：2610.13521
💡 一句话解读：多教师蒸馏中，不同领域的老师各自教学生，但更新时共用一套参数，老师风格差异大时会互相打架，导致某些能力退化。本文提出“跨教师守护”：让非本领域的老师当“裁判”，用老师们的共识强化一致信号、削弱冲突信号，并约束可能冲突的参数更新，从而减少退化、提升整体表现。
多教师同策略蒸馏（MOPD）通过将领域特定数据路由给专门教师，并在学生生成的轨迹上施加密集的 token 级监督，来整合各教师的能力，从而扩展了同策略蒸馏（OPD）。然而，领域路由仅区分不同领域的监督来源；所有领域的更新仍作用于共享参数。当教师在回答风格和领域专长上差异显著时，单独有效的监督在联合优化中仍可能导致持续的梯度冲突和能力退化。为应对这一挑战，我们提出面向 MOPD 的跨教师守护（Road MOPD），它引入非路由教师作为同一学生轨迹上的辅助跨域评判者，并通过 token 级信用分配和梯度更新来协调多教师监督。具体而言，跨教师引导的信用分配利用教师间的一致性来强化被教师共同支持的 token 级修正，并削弱冲突信号；预防性多教师梯度投影利用同一轨迹上的辅助梯度来约束可能冲突的更新，并自适应控制投影强度以保留目标领域的学习信号。实验表明，Road MOPD 缓解了劣势领域的退化，在高冲突下提升超过 5 个百分点，并在低冲突和三个领域的设置中提升了整体性能。
📄 English Abstract
Multi-teacher on-policy distillation (MOPD) extends on-policy distillation (OPD) by routing domain-specific data to specialized teachers, integrating their capabilities through dense token-level supervision on student-generated trajectories. However, domain routing only distinguishes the supervision sources for different domains; updates from all domains still act on shared parameters. When teachers differ substantially in response style and domain expertise, supervision that is effective in isolation may still lead to persistent gradient conflicts and capability degradation during joint optimization. To address this challenge, we propose Cross-Teacher Guarding for MOPD (Road MOPD), which introduces non-routed teachers as auxiliary cross-domain critics on the same student trajectories and coordinates multi-teacher supervision through token-level credit assignment and gradient updates. Specifically, cross-teacher-guided credit assignment uses teacher agreement to strengthen token-level corrections jointly supported by the teachers and attenuate conflicting signals; preventive multi-teacher gradient projection uses auxiliary gradients from the same trajectories to constrain potentially conflicting updates and adaptively controls projection strength to retain target-domain learning signals. In experiments, Road MOPD mitigates degradation in the disadvantaged domain, gains over 5 percentage points under high conflict, and improves overall performance in low-conflict and three-domain settings.
5. 通过批评引导的在线策略蒸馏恢复可靠监督
- 英文标题：Recovering Reliable Supervision via Critique-Guided On-Policy Distillation
- arXiv：2610.20572
💡 一句话解读：在线蒸馏让学生模型跟着老师的指导一步步学，但前提是老师能从学生写错的任何地方把答案救回来——研究发现这个前提不成立：学生一旦出错，老师会跟着错下去。作者提出 CriticOPD：先让老师指出学生的错误，学生据此重写答案，再用老师对修改稿的打分来训练学生，从而恢复可靠的监督信号。
在线策略蒸馏（OPD）让学生模型在自己的输出轨迹上接受教师模型的词元级监督，其假设是教师能从学生写出的任何前缀出发得到正确答案。我们把这一性质称为教师模型的“可恢复性”，并证明它在错误轨迹上并不成立：教师虽然能识别出错误，却会顺着学生的错误继续下去。我们提出 CriticOPD，利用错误识别来恢复可恢复性。教师对错误轨迹进行批评，使学生能够顺着该错误重写其解答。由此得到的输出（将批评作为特权信息一并纳入）由教师打分，而学生在训练时只使用不含批评的修改稿。在两个 Qwen3 学生模型和六个数学推理基准上，CriticOPD 均优于 OPD，在竞赛类题目上提升最大。
📄 English Abstract
On-policy distillation (OPD) trains a student model on its own rollouts with token-level supervision from a teacher, assuming that the teacher can reach a correct answer from any prefix the student writes. We call this property the recoverability of the teacher and show it fails on incorrect rollouts: the teacher follows student errors, although it recognizes them. We propose CriticOPD, using error recognition to restore recoverability. The teacher critiques an incorrect rollout, allowing the student to rewrite its solution following the error. The resulting output, including the critique as privileged information, is scored by the teacher while the student uses the revision without the critique for training. On two Qwen3 students and six mathematical reasoning benchmarks, CriticOPD improves OPD, with the largest gains on competition problems.
6. LatentLoop：采用非对称计算分配扩展循环式 MoE 模型
- 英文标题：LatentLoop: Scaling Looped MoE Models with Asymmetric Compute Allocation
- arXiv：2610.19975
💡 一句话解读：循环式专家混合模型里，同一个模块被重复使用多次，但每次的作用可能不同。这篇论文发现第一次循环的贡献较小，于是提出LatentLoop：压缩第一次循环的序列长度，把省下的计算用来增大模型隐藏层，并让两次循环共享专家权重但使用不同的注意力参数。这样在多个规模下都降低了验证损失，并节省了计算量。
循环式专家混合（MoE）模型将参数复用与稀疏计算相结合，在相同预算下提升性能。先前工作表明，即使共享参数，不同的循环访问也可能扮演不同角色。为了探究不同循环访问的贡献，我们在一个将共享块应用两次的模型中学习单独的残差系数，观察到第一次访问的规模较小。固定系数实验表明，减小第一次访问的规模可以保持或提升性能，而同样减小第二次访问的规模则会损害性能。这种不对称性促使我们探索非对称计算分配。我们提出了LatentLoop，它压缩第一次访问的序列，在第二次访问中保留全分辨率，并重新分配节省的计算来增加隐藏层大小。我们进一步学习第一次访问对完整序列的更新强度，并在不同访问间共享MoE权重，同时使用单独的注意力参数以适应它们不同的序列分辨率。在涵盖13亿到70亿参数、最大预算超过10^21 FLOPs的15种配置中，LatentLoop始终降低验证集负对数似然（NLL）。在拟合的NLL为1.988时，相对于普通MoE和循环基线，预测计算节省分别为19.0%和7.9%。它还在0-shot和5-shot设置下，在所有规模上提升了九个基准测试的平均分数。这些结果表明，非对称计算分配是循环式MoE模型的一个有前景的方向。
📄 English Abstract
Looped Mixture-of-Experts (MoE) models combine parameter reuse with sparse computation to improve performance under matched budgets. Prior work demonstrates that different loop visits can play different roles even when they share parameters. To probe the contributions of different loop visits, we learn separate residual coefficients in a model that applies a shared block twice, observing a smaller scale in the first visit. Fixed coefficient experiments show that reducing the first visit's scale preserves or improves performance, whereas the same reductions to the second hurt it. This asymmetry motivates us to explore asymmetric compute allocation. We propose \textbf{LatentLoop}, which compresses the first visit's sequence, retains full resolution in the second, and reallocates saved computation to increase the hidden size. We further learn how strongly the first visit updates the full sequence, and share MoE weights across visits while using separate attention parameters to accommodate their different sequence resolutions. Across 15 configurations spanning 1.3B–7B parameters, with the largest budget exceeding $10^{21}$ FLOPs, LatentLoop consistently lowers validation NLL. At a fitted NLL of 1.988, predicted compute savings are 19.0% and 7.9% relative to vanilla MoE and the looped baseline. It also improves average scores on nine benchmarks at all sizes in 0-shot and 5-shot settings. These results suggest asymmetric compute allocation is a promising direction for looped MoE models.
7. Cortex：将上下文巩固为参数化记忆
- 英文标题：Cortex: Consolidating Context into Parametric Memory
- arXiv：2610.20399
💡 一句话解读：大模型靠上下文能变聪明，但上下文一删就忘光。Cortex 模仿生物记忆巩固，让模型自己猜上下文可能怎么用，把上下文带来的好处“内化”成参数更新，从而在保留原有能力的同时记住新知识。
上下文能提升大语言模型性能，但移除上下文后这些增益可能丢失。受生物系统巩固启发，我们将记忆巩固（Memory Consolidation, MC）形式化为在保留非目标能力的同时内化这些增益，并将上下文指定的编辑、技能学习和行为引导统一在一个共同目标下。我们引入一个包含五个任务族的测试平台，在单次、批量及顺序更新下衡量巩固与保持效果。我们提出 Cortex，它生成前瞻性查询以预测上下文的用途，并将上下文条件输出蒸馏为可合并的低秩或结构化前馈网络更新。在 4B 至 120B 的稠密和混合专家骨干模型（包括纯文本和多模态模型）上，Cortex 在大多数评估设置中位于经验性的巩固-保持帕累托前沿。在 14B 模型上进行 50 次顺序事实编辑后，Cortex-Structural 恢复了 88.9% 的上下文诱导增益，同时保留了 99.8% 的原始非目标性能。
📄 English Abstract
Context can improve large language model performance, but these gains may be lost after context removal. Inspired by biological systems consolidation, we formulate Memory Consolidation (MC) as internalizing these gains while preserving non-target capabilities, connecting context-specified editing, skill learning, and behavioral steering under a common objective. We introduce a five-task-family testbed measuring consolidation and retention under single, batch, and sequential updates. We propose Cortex, which generates prospective queries to anticipate uses of context and distills context-conditioned outputs into mergeable low-rank or structural feed-forward network updates. Across 4B–120B dense and mixture-of-experts backbones, including both text-only and multimodal models, Cortex lies on the empirical consolidation–retention Pareto frontier in most evaluated settings. After 50 sequential factual edits on a 14B model, Cortex-Structural recovers 88.9% of the context-induced gain while retaining 99.8% of original non-target performance.
8. ActionOPD：蒸馏决策，而非Token
- 英文标题：ActionOPD: Distilling Decisions, Not Tokens
- arXiv：2610.19820
💡 一句话解读：把大模型智能体的能力教给小模型时，该在哪些情况下、按什么粒度对齐师生行为是个难题。现有方法要么只盯着单个词、容易跑偏，要么依赖老师的内部数据。本文提出ActionOPD，改在“动作”层面蒸馏：让学生在自己遇到的状态下采样动作，再和黑盒老师采样的动作比语义是否一致，据此优化学生，无需老师的词概率或相同分词器。
将能力强的语言模型智能体蒸馏到更小的模型中，需要使学生行为与教师行为对齐。这引出一个根本问题：应在哪些状态、以何种粒度进行对齐？离线模仿提供密集但针对具体token的监督，且可能遭遇状态分布不匹配。Token级在线策略蒸馏（OPD）缓解了这种不匹配，但通常需要教师token概率和兼容的分词方式。我们认为语义动作是智能体蒸馏的自然监督单元。为此，我们提出ActionOPD，一种动作级在线策略蒸馏方法，其动机是在学生遇到的状态上最大化期望的教师动作似然。在每个此类状态，ActionOPD从学生和黑盒教师中采样动作，并根据学生动作与采样教师动作的语义一致性给予奖励。通过策略优化，ActionOPD使学生动作级行为与教师决策对齐，而无需复现教师所写的推理过程或获取教师token概率。使用两个教师模型在多样化的精选工具使用任务上训练Qwen3-4B和Qwen3-8B，在四个分布外基准上平均提升3.3至6.9个百分点。ActionOPD随学生规模和教师能力扩展良好，且不要求教师与学生共享同一模型家族或分词器。在$\tau^2$-bench上，ActionOPD取得了与离线蒸馏和token级OPD相当的性能。我们进一步探索了使用同模型教师和特权上下文的Self-ActionOPD。这些结果确立了ActionOPD作为一种有前景的智能体蒸馏方法。
📄 English Abstract
Distilling capable language-model agents into smaller models requires aligning the student's behavior with the teacher's. This raises a fundamental question: at which states and at what granularity should this alignment be performed? Offline imitation provides dense but token-specific supervision and can suffer from state-distribution mismatch. Token-level on-policy distillation (OPD) addresses this mismatch but typically requires teacher token probabilities and compatible tokenization. We argue that semantic actions provide a natural unit of supervision for agent distillation. We introduce ActionOPD, an action-level on-policy distillation method motivated by maximizing expected teacher-action likelihood at student-encountered states. At each such state, ActionOPD samples actions from both the student and a black-box teacher, rewarding student actions according to their semantic agreement with the sampled teacher actions. Through policy optimization, ActionOPD aligns the student's action-level behavior with the teacher's decisions without requiring reproduction of teacher-written reasoning or access to teacher token probabilities. Training Qwen3-4B and Qwen3-8B with two teachers on diverse, curated tool-use tasks yields average gains of 3.3-6.9 percentage points across four out-of-distribution benchmarks. ActionOPD scales favorably with student size and teacher capability, without requiring the teacher and student to share a model family or tokenizer. On $\tau^2$-bench, ActionOPD achieves competitive performance against offline distillation and token-level OPD. We further explore Self-ActionOPD with a same-model teacher and privileged context. These results establish ActionOPD as a promising approach to agent distillation.
9. 更多方法，更好的智能体？智能体强化学习中自蒸馏的现实检验
- 英文标题：More Methods, Better Agents? A Reality Check on Self-Distillation in Agentic Reinforcement Learning
- arXiv：2610.19789
💡 一句话解读：这篇研究要解决的问题是：智能体强化学习里各种“自蒸馏”新方法层出不穷，但大家评测时条件不统一，很难判断谁真的更好。作者把十种代表性方法和基准方法GRPO放在四个任务、三种模型规模下做统一对比，共跑了132次训练、上万个GPU小时，还考察了训练稳定性和计算成本。结论是：这些方法相对基准的收益有限，好成绩可能掩盖训练不稳定，多花钱也不一定换来更好表现。
智能体强化学习（agentic reinforcement learning）中的自蒸馏（self-distillation）已催生出越来越多方法。然而，现有评测往往在不同实验条件下复用已发表的基线分数，很少对同期方法进行直接比较，也较少关注训练稳定性与计算成本。这些局限掩盖了所报告收益的来源，以及各竞争方法的实际优缺点；而智能体训练成本高昂，更凸显了建立统一比较基准的必要性。为弥补这些不足，我们在 ALFWorld、ScienceWorld、Search-QA 和 WebShop 上，跨三种模型规模，将十种代表性方法与 GRPO 进行比较。本研究包含 132 次训练运行，累计记录 12,124.4 个 H800 GPU 小时，并对预训练骨干网络及全部十一种训练方法在八个 rollout 随机种子上进行重复评测。除最终任务表现外，我们还考察了训练进展、稳定性与实测计算成本。评测得出三项关键发现：第一，相对 GRPO 的收益范围仍然有限，没有任何方法能在各种设置下持续优于它；第二，出色的最终表现可能掩盖训练的不稳定；第三，额外的训练成本并不总能转化为更好的性能。我们建立该基准，以支持公平、全面的比较，并引导未来研究走向可靠且高性价比的智能体训练改进。
📄 English Abstract
Self-distillation in agentic reinforcement learning has produced a growing number of methods. However, existing evaluations often reuse published baseline scores under differing experimental conditions, provide few direct comparisons among contemporaneous methods, and give limited attention to training stability and computational cost. These limitations obscure the source of reported gains and the practical strengths and weaknesses of competing methods. Costly agent training further motivates a common basis for comparison. To address these gaps, we compare ten representative methods against GRPO on ALFWorld, ScienceWorld, Search-QA, and WebShop across three model sizes. Our study comprises 132 training runs totaling 12,124.4 recorded H800 GPU-hours, with repeated evaluations over eight rollout seeds for the pretrained backbones and all eleven trained methods. Beyond final task performance, we examine training progress, stability, and measured computational cost. Our evaluation yields three key findings. First, gains over GRPO remain limited in scope, with no method consistently outperforming it across settings. Second, strong final performance can mask unstable training. Third, additional training cost does not consistently translate into better performance. We establish this benchmark to support fair, comprehensive comparison and guide future research toward reliable and cost-effective improvements in agent training. Project page: https://aim-uofa.github.io/Self-Distillation-AgentRL/
10. 感知、维护与学习：多模态智能体记忆的演进
- 英文标题：Seeing, Maintaining, and Learning: The Evolution of Multimodal Agent Memory
- arXiv：2610.19366
💡 一句话解读：多模态智能体要像人一样长期工作，却记不住跨会话的信息、难适应变化。本文提出一个两轴框架：一轴看信息怎么存（编码方式），另一轴看记忆怎么管（固定、动态更新、自我进化三层），并总结设计原则与未来方向。
多模态模型与智能体已出现，它们通过整合多种模态的信息，使AI系统能更像人类那样从真实世界学习并与之交互。然而，随着多模态智能体迈向长时程任务和终身学习场景，如何持续从复杂、异构的多模态数据中获取有用信息仍是一大挑战。应对这一挑战需要有效的记忆机制，以维持跨会话的连续性、适应变化的用户与环境，并通过积累的经验改进记忆管理策略。我们提出了一个面向多模态智能体记忆的双轴分析框架，围绕两个关键问题展开：多模态信息如何形成与表示（轴一：形成与表示），以及其管理如何随积累的经验而适应（轴二：记忆管理）。轴一（形成与表示）刻画保留哪些信息以及如何编码（例如模态特定的嵌入、文本与结构化记录、潜在与参数化表示），以支持后续推理与行动。轴二（记忆管理）区分三个层级：T1静态记忆复用固定记忆；T2动态记忆通过稳定策略更新记忆；T3自进化记忆通过不断演化的策略更新记忆。基于该框架，我们归纳了反复出现的设计模式，并推导出若干原则：在来源保真与抽象之间取得平衡、维护时间有效性与来源追溯、将记忆使用建立在可恢复的证据之上，以及随时间演化记忆策略。此外，我们还讨论了多模态智能体记忆机制面临的挑战与潜在未来方向。
📄 English Abstract
Multimodal models and agents have emerged to enable AI systems to learn from and interact with the real world more like humans by integrating information across multiple modalities. However, enabling multimodal agents to continually acquire useful information from complex, heterogeneous multimodal data remains a significant challenge as they move toward long-horizon tasks and lifelong settings. Addressing this challenge requires effective memory mechanisms that can maintain continuity across sessions, adapt to changing users and environments, and refine memory-management policies through accumulated experience. We introduce a two-axis analytical framework for multimodal agent memory structured around two key questions: how multimodal information is formed and represented (Axis I, Formation and Representation), and how its management adapts with accumulated experience (Axis II, Memory Management). Axis I, Formation and Representation, characterizes what information is retained and how it is encoded (e.g., modality-specific embeddings, textual and structured records, latent and parametric representations) to support subsequent reasoning and action. Axis II, Memory Management, distinguishes three tiers: T1 Static Memory reuses fixed memory; T2 Dynamic Memory updates memory through a stable policy; and T3 Self-Evolving Memory updates memory through an evolving policy. Based on this framework, we synthesize recurring design patterns and derive principles for balancing source fidelity with abstraction, maintaining temporal validity and provenance, grounding memory use in recoverable evidence, and evolving memory policies over time. Furthermore, we discuss the challenges and potential future directions for the memory mechanism of multimodal agents.
11. LensErase：面向擦除任意内容的基础模型
- 英文标题：LensErase: A Foundation Model for Erase Anything
- arXiv：2610.18072
💡 一句话解读：这篇论文要解决的是：现有擦除模型只能处理特定类型的内容，无法通用地擦掉任意东西。它提出LensErase，用超大规模数据集训练，并改造文生图模型，让一个模型就能擦掉物体、文字、水印等各种内容，且效果更好。
我们提出了LensErase，一个通用的基于掩码的“擦除任意内容”基础模型，旨在覆盖广泛的真实世界擦除场景。LensErase支持多样化的物体、视觉文本、水印、相关视觉效果、多物体移除，以及各种物体尺度、宽高比和图像分辨率。首先，为了实现如此广泛的覆盖，我们构建了LensErase-100M，一个包含1.0107亿对源图像-掩码-目标图像样本的大规模数据集，涵盖了多样的擦除场景。其次，我们从文本到图像模型Lens预训练LensErase，将其强大的生成先验适应于基于掩码的擦除。为了高效地整合源图像和擦除掩码，我们提出了Mask-Aware RoPE，一种三维旋转位置编码，直接将掩码信息注入源图像标记，而无需引入单独的掩码标记序列。最后，我们通过监督微调和带有契约引导、区域感知奖励的强化学习来精炼模型，并在推理时应用TraceFusion以保持有效编辑区域外的高保真内容。在六个公开基准和提出的LensErase-Bench上，LensErase在基于参考的客观指标和无参考的VLM评估下均取得了显著改进。我们希望LensErase能够作为擦除相关应用和系统的基础模型。
📄 English Abstract
We introduce LensErase, a general-purpose mask-based Erase Anything foundation model designed to cover a broad spectrum of real-world erase scenarios. LensErase supports diverse objects, visual text, watermarks, associated visual effects, multiple-object removal, and a wide range of object scales, aspect ratios, and image resolutions. First, to enable such broad coverage, we construct LensErase-100M, a large-scale dataset containing 101.07 million paired source-mask-target examples spanning diverse erase scenarios. Second, we pretrain LensErase from the text-to-image model Lens, adapting its strong generative prior to mask-based erasing. To efficiently incorporate the source image and erase mask, we propose Mask-Aware RoPE, a 3D rotary positional encoding that directly injects mask information into the source-image tokens without introducing a separate mask token sequence. Finally, we refine the model through supervised fine-tuning and reinforcement learning with contract-guided, region-aware rewards, and apply TraceFusion at inference to preserve high-fidelity content outside the effective edit region. Across six public benchmarks and the proposed LensErase-Bench, LensErase achieves substantial improvements under both reference-based objective metrics and reference-free VLM evaluation. We hope LensErase can serve as a foundational model for erase-related applications and systems.
12. KV4-RL：利用 INT4 KV 缓存量化解锁 RLVR 更快的异步 Rollout
- 英文标题：KV4-RL: Unlocking Faster Asynchronous Rollouts for RLVR with INT4 KV Cache Quantization
- arXiv：2610.14448
💡 一句话解读：强化学习训练大模型时，生成答案的环节常因个别长回答拖慢整体速度。异步生成能提速，但会集中处理上下文，让缓存压力变大。本文发现把缓存压成4位整数能大幅加速，但会损害学习效果。于是提出KV4-RL：对少数关键位置用8位保护，其余用4位，既保速度又保效果，最终训练提速近两倍且精度几乎不降。
异步 rollout 通过减少长尾生成期间的低利用率，提升了带可验证奖励的强化学习（RLVR）的效率。持续的并发还将上下文处理集中到更少的解码步骤中，增加了每步的 KV 工作负载。我们刻画了这种“KV 集中”现象，并在匹配工作负载的比较中发现，KV 缓存量化比仅权重量化和权重-激活量化能带来显著更大的加速。然而，即使采用 token 级截断重要性采样（TIS），INT4 KV rollout 仍可能损害学习。我们提出 KV4-RL，它将免校准的 INT4 KV 量化与对注意力汇点及近期上下文的选择性 INT8 保护相结合。保护有限数量的位置减少了采样器失配，同时保留了 INT4 的大部分加速。在 Qwen3-8B-Base GRPO 训练的 400 次迭代中，KV4-RL 相比 BF16 rollout 实现了 1.80 倍的累积 rollout 加速和 1.27 倍的累积全迭代加速，在最后 50 次迭代中 rollout 加速达到 1.97 倍。在共同的 BF16 评估器下，对于 Qwen3-4B-Base 和 Qwen3-8B-Base，其最终六个基准的平均准确率与 BF16-rollout 训练的差距在 0.5 个百分点以内。
📄 English Abstract
Asynchronous rollout improves the efficiency of reinforcement learning with verifiable rewards (RLVR) by reducing under-utilization during long-tail generation. Sustained concurrency also concentrates context processing into fewer decode steps, increasing the KV workload per step. We characterize this \emph{KV concentration} and find that KV cache quantization delivers substantially greater acceleration than weight-only and weight-activation quantization in our matched-workload comparisons. However, INT4 KV rollout can degrade learning even with token-level truncated importance sampling (TIS). We propose \emph{KV4-RL}, which combines calibration-free INT4 KV quantization with selective INT8 protection of attention sinks and recent context. Protecting a bounded number of positions reduces sampler mismatch while retaining most of the speedup from INT4. Across 400 iterations of Qwen3-8B-Base GRPO training, KV4-RL achieves $1.80\times$ cumulative rollout speedup and $1.27\times$ cumulative full-iteration speedup over BF16 rollout, reaching $1.97\times$ rollout speedup over the final 50 iterations. Under a common BF16 evaluator, its final six-benchmark mean accuracy is within 0.5 percentage points of BF16-rollout training for both Qwen3-4B-Base and Qwen3-8B-Base.
13. 迈向智能体机器人：模型、数据、环境与执行框架（Harness）
- 英文标题：Towards Agentic Robots: Model, Data, Environment, and Harness
- arXiv：2610.20442
💡 一句话解读：这篇研究要解决的问题是：机器人智能领域发展快但研究零散，缺少统一框架来整合各种智能功能。它提出把机器人看作“智能体”，从模型、数据、环境、执行框架四个维度梳理现有研究，并归纳出十七个类别，为构建更自主的机器人系统提供清晰路线图。
基础模型（Foundation Models）正推动机器人智能发生范式转变，从低层动作预测迈向对数据收集、模型训练、任务规划和真实世界执行的目标导向编排。我们将这一新兴范式称为智能体机器人（agentic robots），其中智能体能够对人类指令进行推理、与机器人环境交互、自主纠正执行错误，并动态调整操作策略。尽管该领域发展迅速，但现有研究之间缺乏良好衔接，研究者对于如何将不同的智能体功能组合成完整的机器人系统尚无一致看法。本文对现有智能体机器人研究进行了统一综述：首先追溯从机器人策略到智能体系统的发展脉络，涵盖视觉-语言-动作模型、世界模型和世界-动作模型；然后阐明智能体机器人的定义与范围，并将现有研究按模型、数据、环境与执行框架（Harness）四个功能域组织为十七个类别。在这些功能域中，我们考察了智能体如何塑造模型学习与机器人决策、收集与整理机器人数据、构建用于学习和评估的世界、任务与测试，以及组织机器人使用的能力。最后，我们讨论了开放挑战并展望了未来研究方向。展望未来，我们希望这项工作有助于更清晰地理解智能体机器人，并支持新方法，使其能力更强、更可靠、更具适应性。
📄 English Abstract
Foundation models are driving a paradigm shift in robotic intelligence, moving beyond low-level action prediction to goal-directed orchestration of data collection, model training, task planning, and real-world execution. We characterize this emerging paradigm as an agentic robots, in which agents reason over human instructions, interact with robotic environments, rectify execution errors autonomously, and dynamically adjust operational strategies. Although this field develops fast, current studies are not well connected. Researchers have no consistent view of how to combine different agentic functions into a complete robot system. This paper provides a unified overview of existing research on agentic robots. We first trace the development from robot policies to agentic systems, covering vision-language-action models, world models, and world-action models. We then clarify the definition and scope of agentic robots and organize existing research into seventeen categories across four functional domains: Model, Data, Environment, and Harness. Across these domains, we examine how agents shape model learning and robot decisions, collect and curate robot data, construct worlds, tasks, and tests for learning and evaluation, and organize capabilities for robot use. Finally, we discuss open challenges and outline directions for future research. Looking forward, we hope this work contributes to a clearer understanding of agentic robots and supports new approaches to making them more capable, reliable, and adaptive.
14. 混合推理智能体
- 英文标题：Hybrid Reasoning Agents
- arXiv：2610.18861
💡 一句话解读：现有AI智能体只能用文字思考，效率低；而纯内部思考又无法用工具、不透明。本文提出混合推理智能体，让AI在内部思考与文字交互间自由切换，既保留工具能力，又让思考可查看，并设计训练方法，性能提升且更省更快。
我们如何构建一个智能体，使其思考超越人类语言，却仍能通过语言在外部环境中行动？现代大语言模型（LLM）智能体仍通过文本进行推理和交互，而潜在推理（latent reasoning）提供了一种高效的内部媒介，但缺乏原生工具访问和推理透明度。我们通过混合推理智能体（HybridAgents）连接这两个方向，它在统一的生成过程中动态地在潜在推理和基于文本的交互之间切换。在架构上，轻量级HybridLink实现了文本和潜在模式之间的无缝转换，而智能体的连续工作记忆在多轮中保留无损的混合上下文。这些设计保留了工具集成的智能体能力，同时使潜在思维可直接检查。基于此架构，我们进一步开发了用于混合推理的专用后训练算法：HybridSFT通过教导主干网络与HybridLink协调，稳定了工具集成的混合生成；然后HybridRL通过学习何时在潜在空间中思考、何时通过文本行动，扩展了推理能力。我们在3个模型规模上，针对涵盖数学、科学、医学、搜索和智能体推理的11个基准评估了HybridAgent。与前沿LLM智能体和涵盖基于文本及潜在方法的智能体框架相比，HybridAgent持续提升性能10.4%，同时实现35.5%-54.7%的令牌减少、1.8倍-2.6倍的推理加速和2.7倍-3.7倍的成本节省。代码和数据可在https://github.com/togethercomputer/HybridAgent获取。
📄 English Abstract
How can we build an agent that thinks beyond human language, yet still acts through it in the external environment? Modern LLM agents still reason and interact through text, while latent reasoning offers an efficient internal medium but lacks native tool access and reasoning transparency. We bridge these two directions with Hybrid Reasoning Agents (HybridAgents), which dynamically navigate between latent reasoning and text-based interaction throughout a unified generation process. Architecturally, a lightweight HybridLink enables seamless transitions between text and latent modes, while the agent's continuous working memory preserves lossless hybrid context across multiple turns. These designs preserve tool-integrated agentic capabilities while making latent thoughts directly inspectable. Building on this architecture, we further develop dedicated post-training algorithms for hybrid reasoning: HybridSFT stabilizes tool-integrated hybrid generation by teaching the backbone to coordinate with HybridLink, then HybridRL scales reasoning by learning when to think in latent space and when to act through text. We evaluate HybridAgent at 3 model scales across 11 benchmarks covering mathematics, science, medicine, search, and agentic reasoning. Compared with frontier LLM Agents and agentic frameworks spanning both text-based and latent methods, HybridAgent consistently improves performance by 10.4%, while delivering 35.5%-54.7% token reduction, 1.8x-2.6x faster inference, and 2.7x-3.7x cost savings. Code and data are available at https://github.com/togethercomputer/HybridAgent.
15. 在正确的潜在空间中进行超分辨率：一种冻结的视觉基础基底
- 英文标题：Super-Resolution in The Right Latent Space: A Frozen Vision-Foundation Substrate
- arXiv：2610.04781
💡 一句话解读：这篇论文要解决的是：真实场景下的超分辨率（把模糊、有损的低清图变高清）容易产生错误细节（幻觉）。作者提出，不要在传统的重建导向的潜在空间或像素空间里做超分，而是换到一个“冻结的视觉基础模型”（DINOv3-L）的中间层特征空间。在这个空间里，退化图像离高清图像所在的流形很近，而且包含从像素细节到语义的层次信息，因此一个4.15亿参数的解码器就能一步映射回高清图，在多个数据集上取得了保真度与感知质量的最佳平衡，速度也很快。
在图像潜在空间中，高分辨率、自然且清晰的图像嵌入形成一个流形。高分辨率图像的退化会将其嵌入推离该流形。因此，真实世界超分辨率（SR）的任务就变成了将退化的嵌入映射回流形——不是流形上的任意位置，而是保留输入所携带信息（包括语义和像素细节）的那个点。目前所有已发表的方法都在面向重建的潜在空间或像素空间中实现这种映射。我们主张这些空间对于SR来说是错误的基底。低分辨率和退化图像嵌入在远离流形的地方，使得映射困难且昂贵。这些基底中缺乏语义信息也使得难以导航到流形上的忠实点，在退化严重时导致严重的幻觉。因此，在合适的潜在空间中进行恢复对SR任务至关重要。我们表明，冻结的DINOv3-L的23个融合层的潜在空间就是这样一个空间，它使SR任务更容易。退化图像嵌入在流形附近。此外，这个基底包含从像素记录到退化鲁棒语义的信息层次，引导模型找到流形上的忠实点。在这个基底上，一个415M的解码器在重建和对抗目标下训练，以将退化的嵌入映射回来并一次性解码到像素空间。所得模型RAESR在RealSR、DRealSR、LSDIR和DIV2K-Val上，在先进的对抗和基于扩散的恢复器中，达到了最佳的保真度-感知权衡，在单张H20 GPU上处理512×512图像仅需37毫秒。在相同配方下将基底换成VAE潜在空间，则在每个指标上都表现不佳。
📄 English Abstract
In an image latent space, the embeddings of high-resolution, natural, and sharp images form a manifold. Degradation of high-resolution images pushes their embeddings off this manifold. Real-world super-resolution (SR) then becomes the task of mapping the degraded embedding back onto this manifold — not anywhere on the manifold, but to the point that preserves what the input still carries, both its semantics and pixel details. Every published method implements this mapping in a reconstruction-oriented latent space or pixel space. We claim these spaces are the wrong substrates for SR. Low-resolution and degraded images are embedded far from the manifold, making the mapping difficult and expensive. The lack of semantic information in these substrates also makes it difficult to navigate to the faithful point on the manifold, causing severe hallucination when degradation is heavy. Thus, restoring in a suitable latent space is crucial to the SR task. We show that the latent space of 23 fused layers of a frozen DINOv3-L is one such space that makes the SR task easier. Degraded images are embedded near the manifold. Moreover, this substrate contains a hierarchy of information, from pixel record to degradation robust semantics, guiding the model to find the faithful point on the manifold. On this substrate, a 415M decoder is trained under reconstruction and adversarial objectives to map the degraded embeddings back and decode to pixel space in one pass. The resulting model, RAESR, attains the best fidelity–perception trade-off among state-of-the-art adversarial and diffusion-based restorers on RealSR, DRealSR, LSDIR and DIV2K-Val, at 37 ms per 512 by 512 image on a single H20 GPU. Swapping the substrate for a VAE latent under an identical recipe loses on every metric.