Daily Papers · 2026-10-23
📄 论文日报 | 来源：Hugging Face Daily Papers
🗓 Episode：2026.10.22 | 流式视频评测揭示时序瓶颈；VLM三维建图仍难精准
1. StreamEval：评估交互式流式视频生成模型
- 英文标题：StreamEval: Evaluating Interactive Streaming Video Generation Models
- arXiv：2610.28402
💡 一句话解读：现有视频生成模型在“边生成边按用户指令实时改动”这件事上缺乏统一评测。作者构建了 StreamEval 基准，覆盖虚拟人对话、虚拟人表演、世界生成、指令编辑和参考编辑五类任务，用300个带脚本的测试案例从执行、状态一致、画质和音画同步四方面打分，并实测37个模型，发现画面好看不等于指令执行可靠。
随着流式视频生成的发展，实现与生成视频的实时交互正变得日益重要。我们提出 StreamEval，这是一个面向交互式流式视频生成的统一基准，涵盖虚拟人对话、虚拟人表演、世界生成、指令编辑和参考编辑五项任务。StreamEval 包含300个时长15至300秒的脚本化测试案例，每个案例规定初始素材、逐步到达的多模态请求，以及对状态变化和内容保持的明确要求。评测从任务执行、状态一致性、视觉保真度和音画协调四个维度进行，并通过十项任务特定指标以0-100分制量化。我们在300个测试案例上评估了37个不同模型，发现高视觉质量和身份保持并不必然伴随可靠的请求执行或持续的状态一致性。尤其请求执行仍是难点：在七个虚拟人表演模型中“跟随（Following）”最高分为36.90，文本条件世界生成中“跟随”最高分为25.93，三个已报告的参考编辑模型中“编辑成功（Edit Success）”最高分仅为15.00。这些发现表明，在时序与内容保持约束下实现可靠的请求执行，是推进全模态实时交互系统的核心挑战。排行榜见 https://streameval.ai。
📄 English Abstract
As streaming video generation advances, enabling real-time interaction with generated video is becoming increasingly important. We introduce StreamEval, a unified benchmark for interactive streaming video generation spanning five tasks: avatar dialogue, avatar performance, world generation, instruction editing, and reference editing. StreamEval comprises 300 scripted test cases lasting 15-300s, each specifying initial assets, incrementally arriving multimodal requests, and explicit requirements for state changes and content preservation. Performance is assessed along four evaluation dimensions: task execution, state consistency, visual fidelity, and audiovisual coordination, instantiated through ten task-specific metrics on a 0-100 scale. Evaluating 37 distinct models across 300 test cases, we find that high visual quality and identity retention do not necessarily coincide with reliable request execution or sustained state consistency. In particular, request execution remains challenging: the highest task-specific scores are 36.90 for Following among seven avatar-performance models, 25.93 for Following in text-conditioned world generation, and 15.00 for Edit Success among three reported reference-editing models. These findings identify reliable request execution under timing and preservation constraints as a central challenge for advancing omni-modal real-time interactive systems. The leaderboard is available at https://streameval.ai
2. SpatialScout：VLM智能体能否主动探索并构建3D场景地图？
- 英文标题：SpatialScout: Can VLM Agents Actively Explore and Map 3D Scenes?
- arXiv：2610.26659
💡 一句话解读：现有空间问答测试难以判断AI是否真能靠主动探索建出准确的三维地图。作者提出SpatialScout基准：让视觉语言模型控制相机在80个室内场景中自主探索，估计3140个目标物体的位置，并同时考察定位精度和覆盖完整度。
空间智能要求智能体主动获取视觉证据，并将观测整合为对三维世界的一致表征。然而，空间问答难以充分证明模型能否通过自主探索构建准确的度量地图。我们提出SpatialScout，一个用于评估视觉语言模型（VLM）智能体在主动物体级三维建图任务上表现的基准。SpatialScout包含80个多样化室内场景、共3140个目标物体。给定物体清单、RGB观测、相机标定和相机位姿，智能体需控制相机探索每个场景，并在统一坐标系中估计物体中心。该任务要求在有限交互预算下协调视角选择、跨视角关联、几何估计与地图修正。评估通过物体尺度的重叠分数同时衡量定位精度与目标覆盖率，并对漏检预测进行惩罚。对前沿VLM的实验揭示了显著局限：表现最强的模型Claude Opus 5.5平均重叠分数仅为41.71%，47.23%的目标超过0.5的交并比（IoU）阈值，仅3.25%超过0.9。这些结果表明，在恢复粗略物体布局与构建完整、精确的度量地图之间仍存在巨大差距。SpatialScout为开发通过主动感知构建可靠三维空间表征的智能体提供了定量测试平台。代码见https://github.com/JesseZZZZZ/SpatialScout。
📄 English Abstract
Spatial intelligence requires agents to actively acquire visual evidence and integrate observations into a coherent representation of the three-dimensional world. Yet spatial question answering offers limited evidence of whether models can construct accurate metric maps through autonomous exploration. We introduce SpatialScout, a benchmark for evaluating vision-language model (VLM) agents on active object-level 3D mapping. SpatialScout comprises 80 diverse indoor scenes containing 3,140 target objects. Given an object inventory, RGB observations, camera calibration, and camera poses, agents control a camera to explore each scene and estimate object centers in a shared coordinate frame. The task requires coordinating viewpoint selection, cross-view association, geometric estimation, and map revision under bounded interaction budgets. Evaluation jointly measures localization accuracy and target coverage through object-scale overlap scores that penalize missing predictions. Experiments with frontier VLMs reveal substantial limitations: the strongest evaluated model, Claude Opus 5.5, achieves a mean overlap score of 41.71%, with 47.23% of targets exceeding an intersection-over-union (IoU) threshold of 0.5 and only 3.25% exceeding 0.9. These results reveal a substantial gap between recovering coarse object layouts and constructing complete, precise metric maps. SpatialScout provides a quantitative testbed for developing agents that build reliable 3D spatial representations through active perception. Code is available at https://github.com/JesseZZZZZ/SpatialScout.
3. 展望未来：自回归视频生成中的未来演化建模
- 英文标题：Looking Ahead: Modeling Future Evolution in Autoregressive Video Generation
- arXiv：2610.29989
💡 一句话解读：视频生成模型常靠前后画面高度相似“偷懒”，画面看着连贯却未必真懂场景如何演变。作者提出“未来块预测”：让模型跳过中间画面去预测更靠后的片段，迫使它学习场景变化规律，且不增加推理成本。
自回归视频扩散模型逐块生成视频，但能生成看似合理的后续内容，并不保证模型捕捉到了场景的演化方式。在教师强制（teacher forcing）训练下，强时间冗余可能使局部视觉连续性成为当前块去噪的捷径，从而降低建模视觉演化的压力。我们提出未来块预测（Future Chunk Prediction, FCP），这是一种与教师强制去噪联合优化的辅助目标，可在无法获取中间视觉上下文的情况下预测时间上相隔的块。FCP 直接从训练视频中获取监督，无需额外的预训练视觉编码器，且不改变自回归推理过程。我们的方法在总体训练计算预算更低的情况下，实现了优于标准教师强制自回归训练的物理合理性，凸显了未来块监督在建模视觉演化方面的有效性与效率。有趣的是，我们的可视化结果表明，FCP 促进了更具结构化的表示空间，这可能对构建自回归世界模型具有价值。
📄 English Abstract
Autoregressive video diffusion models generate videos chunk by chunk, yet producing plausible continuations does not guarantee that they capture how scenes evolve. Under teacher forcing, strong temporal redundancy may allow local visual continuity to serve as a shortcut for current-chunk denoising, reducing the pressure to model visual evolution. We introduce Future Chunk Prediction (FCP), an auxiliary objective jointly optimized with teacher-forced denoising that predicts temporally separated chunks without access to the intervening visual context. Drawing supervision directly from training videos, FCP requires no additional pretrained visual encoder and leaves autoregressive inference unchanged. Our method achieves superior physical plausibility over standard teacher-forced autoregressive training with a lower overall training compute budget, highlighting the effectiveness and efficiency of future-chunk supervision for modeling visual evolution. Intriguingly, our visualizations suggest that FCP fosters a more structured representation space, which may be valuable for building autoregressive world models.
4. 面向智能体的缓存工程
- 英文标题：Cache Engineering for Agents
- arXiv：2610.28577
💡 一句话解读：大语言模型智能体执行长任务时，历史记录越积越多，缓存读写和存储成为主要开销，而删改历史又会导致缓存失效、需要重新计算。本文提出“缓存工程”：让智能体直接丢弃或调整缓存内容，而不改动历史记录本身，并配套设计服务引擎AKV来正确复用缓存，从而在保持准确率的同时降低成本、提升速度。
大语言模型（LLM）智能体能够处理日益复杂的任务，在较长的观测与动作轨迹中运行。随着轨迹增长，键值（KV）缓存的输入输出与存储成为主要开销；轨迹还可能超出LLM的最大上下文长度，因而需要控制活跃缓存的增长。已有工作通过上下文工程（裁剪或摘要历史）来减小缓存，但修改轨迹前缀会使后续缓存状态失效，并触发对保留上下文的昂贵重预填充（re-prefill）；若保留上下文过少以控制重预填充成本，任务准确率又会下降。我们提出“缓存工程”这一新方案，使智能体无需修改轨迹或重预填充保留上下文，即可直接控制其活跃KV缓存：智能体可主动使用drop操作将选定缓存排除在后续注意力之外，并用repos操作调整保留键的位置编码。我们形式化了这些缓存操作，并指出历史依赖性是一大服务挑战：相同token序列在不同缓存操作序列下会产生不同缓存状态，使前缀匹配不足以正确复用缓存。为此我们协同设计了AKV服务引擎，对外提供与现有服务API兼容的无状态接口，内部则管理有状态的缓存复用、驱逐与恢复。实验表明，与常规的阈值触发摘要相比，工具响应滚动丢弃（rolling-drop）缓存策略使基准平均分提高1.7分、平均成本降低27.7%；借助AKV，输出吞吐量最高提升至1.2倍，P95请求延迟加速1.5倍。缓存策略的递归自我改进在专家设计基础上进一步降低成本7.1%。
📄 English Abstract
Large Language Model (LLM) agents can handle increasingly complex tasks, navigating long trajectories of observations and actions. As these trajectories grow, key-value (KV) cache I/O and storage become the dominant cost. Trajectories can also exceed the LLM's maximum context length, requiring control over active cache growth. Prior work uses context engineering to reduce cache size by trimming or summarizing history. However, modifying the trajectory prefix invalidates subsequent cached states and triggers costly re-prefilling of retained context. Task accuracy can also suffer when too little context is retained to limit re-prefill costs. We propose cache engineering as a new scheme. It allows agents to directly control their active KV cache without modifying the trajectory or re-prefilling retained context. Specifically, agents can proactively use drop to exclude selected cache from future attention and repos to adjust retained keys' position encodings. We formalize these cache operations and identify history dependence as a serving challenge: identical token sequences can yield different cache states under different cache operation sequences, making prefix matching insufficient for correct cache reuse. Thus, we co-design the AKV serving engine. It provides a stateless interface compatible with existing serving APIs, while internally managing stateful cache reuse, eviction, and restoration. Experiments show that a tool-response rolling-drop cache policy improves average benchmark scores by 1.7 points and reduces average cost by 27.7%, compared with conventional threshold-triggered summarization. With AKV, it achieves up to 1.2x the output throughput and a 1.5x speedup in P95 request latency. Recursive self-improvement of cache policies further reduces cost by 7.1% over expert design. Our code is at https://github.com/thu-nics/AKV.
5. DONIC：基于人类演示的通用灵巧双手跟踪控制
- 英文标题：DONIC: Generalist Dexterous Bimanual Tracking Control from Human Demonstrations
- arXiv：2610.29997
💡 一句话解读：现有灵巧操作模型大多只能单手、场景有限，无法应对生活中常见的双手协同操作。作者构建了双手交互数据集BiHOI，并提出DONIC通用双手跟踪器，把强化学习与模仿学习结合、循环利用成功经验，让一个策略学会多种双手协同动作。
尽管灵巧参考跟踪近期取得进展，现有模型主要关注单手操作和有限的交互范围，忽视了现实生活中常见的双手灵巧行为。富含接触的双手操作数据稀缺，且协调接触的复杂度更高，使开发通用双手跟踪器尤为困难。为此，我们提出：1）DONIC，首个跨操作类别与人类动作来源联合训练的通用双手跟踪器；2）BiHOI，一个包含1575条序列、覆盖229个物体并强调持续双手交互的多视角数据集，其中重建的交互被重定向为机器人参考。DONIC将近端策略优化与行为克隆结合在反馈回路中，把验证成功的执行回收为训练示范；以物体为中心的单手奖励在依据各手参考接触角色进行监督的同时保持物体运动反馈；Clip-Centered Advantages去除依赖参考的优势偏移，同时保留轨迹内差异。这些设计使单一策略能够学习跨异构参考的协调操作。无需微调，单个检查点在训练物体的未见轨迹上达到86.79%的成功率，在留出物体上达到57.32%，分别超过最先进的双手跟踪基线77.36和45.73个百分点。
📄 English Abstract
Despite recent progress in dexterous reference tracking, existing models focus primarily on single-hand manipulation and a limited range of interactions, leaving the bimanual dexterous behaviors common in real life unaddressed. The scarcity of contact-rich bimanual manipulation data and the increased complexity of coordinating contacts make developing a generalist bimanual tracker particularly challenging. Therefore, we present 1) DONIC, the first generalist bimanual tracker jointly trained across operation categories and human-motion sources, together with 2) BiHOI, a multi-view dataset of 1,575 sequences spanning 229 objects and emphasizing sustained bimanual interaction, where reconstructed interactions are retargeted into robot references. DONIC combines proximal policy optimization with behavioral cloning in a feedback loop that recycles verified successful executions into training demonstrations. An object-centered per-hand reward maintains object-motion feedback while supervising each hand according to its reference contact role. Clip-Centered Advantages remove reference-dependent advantage offsets while preserving within-trajectory differences. Together, these designs enable one single policy to learn coordinated manipulation across heterogeneous references. Without finetuning, a single checkpoint achieves 86.79% success on unseen trajectories of training objects and 57.32% on held-out objects, surpassing the state-of-the-art bimanual tracking baselines by 77.36 and 45.73 percentage points.
6. 真实环境中的智能体世界模型：计算机使用智能体环境合成
- 英文标题：Agent World Model in the Wild: Computer-Use Agent Environment Synthesis
- arXiv：2610.26828
💡 一句话解读：训练电脑操作智能体若直接在真实系统里试错，既难扩展又有安全风险。本研究提出 WildAWM：用开源模型自动合成可运行、可控制的虚拟电脑环境与任务，同时支持图形界面、工具调用和命令行，并用执行检验保证任务可解、够难、评分可靠，训练出的智能体能泛化到多种未见任务。
智能体学习正从依赖人类数据转向通过与世界交互来学习，但直接在真实计算机系统中扩展此类学习颇具挑战，并可能带来重大安全风险。为此，我们提出 Agent World Model in the Wild（WildAWM），一个可扩展的流程，用于合成可控环境与任务，以训练通用计算机使用智能体（CUA）。现有方法通常只针对单一智能体交互接口，而我们构建的可执行环境同时支持图形用户界面、模型上下文协议（MCP）服务和命令行界面，使智能体能够灵活组合视觉交互、工具使用与代码执行。WildAWM 完全由开源模型驱动，表明无需专有前沿模型也能生成高质量的计算机使用环境。除环境外，我们还通过引入基于执行的质量门控——结合异构智能体多轮试跑、交叉评判一致性检查以及对判断稳健性的反事实探测——来解决任务合成中反复出现的三个难题：确保可解性、保持足够难度以及产出可靠的验证器。该流程最终合成了 341 个长程环境-任务对，解决它们可能需要超过 100 个智能体步骤，且即便对前沿模型仍具挑战性，GPT-6-Astra 仅能解决其中 60%。我们仅在这些合成世界中训练了两个不同规模的 Qwen 模型，二者在涵盖网页、claw 与计算机使用智能体的 8 个分布外基准上均取得显著提升，表明在 WildAWM 合成世界中的学习能广泛泛化到未见过的计算机使用任务。代码已公开于 https://github.com/MSR-Orchard/WildAWM。
📄 English Abstract
Agent learning is shifting from learning from human data toward learning through interaction with the world. However, scaling such learning directly in real-world computer systems is challenging and can introduce substantial security risks. We therefore propose Agent World Model in the Wild (WildAWM), a scalable pipeline for synthesizing controllable environments and tasks for training general computer-use agents (CUAs). Whereas existing approaches typically specialize in a single agent interaction interface, we construct executable environments that jointly support graphical user interfaces, model context protocol services, and command-line interfaces, allowing agents to flexibly combine visual interaction, tool use, and code execution. WildAWM is entirely powered by open-source models, demonstrating that high-quality computer-use environments can be generated without proprietary frontier models. Beyond environments, we address three recurring challenges in task synthesis: ensuring solvability, maintaining sufficient difficulty, and producing reliable verifiers, by introducing execution-based quality gates that combine heterogeneous agent rollouts, cross-judge consistency checks, and counterfactual probes of judgment robustness. The resulting pipeline synthesizes 341 long-horizon environment-task pairs that can require over 100 agent steps to solve and remain challenging even for frontier models, with GPT-6-Astra solving only 60% of them. We train two Qwen models of different sizes exclusively in these synthetic worlds. Both models achieve substantial gains across 8 out-of-distribution benchmarks spanning web, claw, and computer-use agents, demonstrating that learning in WildAWM's synthesized worlds generalizes broadly to unseen computer-use tasks. The code is publicly available at https://github.com/MSR-Orchard/WildAWM.
7. 光谱-空间解释
- 英文标题：Spectral-Spatial Interpretation
- arXiv：2610.21114
💡 一句话解读：现有图像解释方法要么只画模型看哪里，要么只给全局通道打分，对多光谱数据不够用，因为某个波段的重要性往往只出现在局部区域。作者提出SSI方法，对每个像素位置单独做通道间的公平归因，得到一张位置×通道的完整解释矩阵，比单一热图或通道排名更细致。
事后图像解释通常用空间热图显示模型看哪里，或用全局通道分数表明哪些通道重要。这两种分离的视角对多光谱数据有局限，因为通道相关性往往在空间上局部化且具有物理意义。我们提出光谱-空间解释（SSI），一种在位置-通道参与者（g,c）上的全网格归因方法。对每个空间位置，SSI固定观测到的周围上下文，在该位置的通道上定义条件博弈，并对此局部博弈应用Shapley值，从而得到完整的光谱-空间归因矩阵，而非单一热图或全局通道排名。SSI具有清晰的局部解释：它在每个位置内满足Shapley公平性，且聚合其通道归因可精确恢复将该位置替换为基线所得到的位置级上下文对比。SSI也不同于上下文平均的Owen式分层核算，后者使用相同的位置-通道分组但对空间外部联盟取平均。在HLS Burn Scar上，SSI在保留归因指标上优于MC-Owen估计器，而评估预算仅约为其6%。在CUB-200、EuroSAT和HLS Burn Scar上，在逐通道均值和零基线两种设置下，SSI在稠密、平坦和秩1基线上取得最佳的联合删除与插入AUC，获得最强的光谱命中率和通道删除效果，并在两个带标注数据集上领先空间定位。
📄 English Abstract
Post-hoc image explanations usually show where a model looks using spatial heatmaps or indicate which channels matter using global channel scores. These separate views are limiting for multispectral data because channel relevance is often spatially localized and physically meaningful. We introduce Spectral-Spatial Interpretation (SSI), a full-grid attribution method over position-channel players (g,c). For each spatial position, SSI fixes the observed surrounding context, defines a conditional game over the channels at that position, and applies the Shapley value to this local game. The result is a full spectral-spatial attribution matrix rather than a single heatmap or a global channel ranking. SSI has a clear local interpretation. It satisfies Shapley fairness within each position, and aggregating its channel attributions exactly recovers the position-level contextual contrast obtained by replacing that position with the baseline. SSI also differs from context-averaged Owen-style hierarchical accounting, which uses the same position-channel grouping but averages over external spatial coalitions. On HLS Burn Scar, SSI outperforms an MC-Owen estimator across retained attribution metrics while using only about 6% of its evaluation budget. Across CUB-200, EuroSAT, and HLS Burn Scar under both per-channel mean and zero baselines, SSI achieves the best joint deletion and insertion AUC over dense, flat, and rank-1 baselines, yields the strongest spectral hit rates and channel deletion, and leads spatial localization on both annotated datasets. Code is available at https://github.com/AtheavenMa/Spectral-Spatial-Interpretation.
8. PrismaCoder：通过分解式评分细则合成突破开放多模态代码智能的边界
- 英文标题：PrismaCoder: Pushing the Boundaries of Open Multimodal Code Intelligence through Decomposed Rubric Synthesis
- arXiv：2610.20538
💡 一句话解读：现有开源多模态代码模型落后于闭源模型，瓶颈在于训练数据质量难判断——一段代码既要程序正确、结构良好，还要渲染后视觉正确，而现有方法（执行结果、代码相似度、专用验证器）都覆盖不全。作者提出按可检查标准拆解评分的合成评分细则，用它同时筛选数据和作为强化学习奖励，构建大规模数据集并训练出两个开源模型。
多模态代码智能将程序的符号世界与其执行所产生的视觉世界连接起来。随着模型越来越多地被要求生成图表、网页、动画和半结构化文档，人们对这一能力的需求正迅速增长。近期，专有模型已大幅领先，甚至仅凭文本描述或截图就能生成可交互产物。然而其数据与训练方式并未公开，开源进展受制于可用数据的数量与质量，以及生产这些数据的流程与方法。质量是两者中更难的一项，因为一个样本必须作为程序是正确的、构建良好的，而且最重要的是渲染后视觉上正确，而现有信号无法同时覆盖这三点：执行通常只返回通过或失败，代码相似度无法映射到视觉输出，任务专用验证器又难以泛化。为推进开源进展，我们从数据侧入手，提出 PrismaForge——一个以合成评分细则（rubric）为核心的分层数据流水线，该细则将这些因素分解为可检查的标准。同一套评分细则既作过滤器又作奖励：它先筛选监督语料，再驱动视觉编码任务上的强化学习。借助 PrismaForge 进行数据合成与过滤，我们构建了 PrismaCoder-2M，这是目前最多样化且规模最大的开源多模态代码智能数据集。在模型侧，我们基于该数据训练并发布了 PrismaCoder-9B 和 PrismaCoder-35B，二者大幅领先开源最先进水平，并可媲美领先的专有系统。我们的分析进一步表明，所提出的基于评分细则的流水线能提升不同模型骨干的多模态编码性能。代码与数据已发布于 https://qiushisun.github.io/PrismaCoder-Home/。
📄 English Abstract
Multimodal code intelligence connects the symbolic world of programs with the visual world their execution creates. Interest in this capability is burgeoning, as models are increasingly asked to produce charts, webpages, animations, and semi-structured documents. Recently, proprietary models have pushed far ahead, producing interactive artifacts even from a text description or a screenshot alone. Yet their data and training remain closed, and open progress is held back by the quantity and quality of available data and by the pipelines and methods for producing it. Quality is the harder of the two, since a sample must be correct as a program, well built, and above all visually right once rendered. No existing signal covers all three. Execution often returns just a pass or fail, code similarity does not map to the visual output, and task-specific verifiers do not generalize. To push open progress further, we start from the data side with PrismaForge, a hierarchical data pipeline centered on a synthesized rubric that decomposes these factors into checkable criteria. The same rubric serves as both filter and reward: it curates the supervised corpus, then drives reinforcement learning on visual-coding tasks. Leveraging PrismaForge for data synthesis and filtering, we build PrismaCoder-2M, the most diverse open dataset for multimodal code intelligence and the largest to date. On the model side, we train PrismaCoder-9B and PrismaCoder-35B on this data and release both. They lead the open state of the art by a wide margin and rival leading proprietary systems. Our analyses further show that the proposed rubric-based pipeline improves multimodal coding performance across model backbones. We release our code and data at https://qiushisun.github.io/PrismaCoder-Home/.
9. 苏格拉底式技能学习：交错教学与实践以培养可迁移的智能体技能
- 英文标题：Socratic Skill Learning: Interleaving Teaching and Practice for Transferable Agent Skills
- arXiv：2610.17860
💡 一句话解读：论文要解决智能体学到的技能难以迁移到新任务的问题。现有方法只靠成功尝试，缺少对使用条件的检验。作者提出苏格拉底式技能学习：让专家引导、学生动手实践并编写技能，交替进行教学与练习，从而学到更可迁移的技能。
智能体技能（Agent Skills）通过使任务知识可在会话间复用来改进大语言模型智能体。学习可迁移的技能需要关于操作流程及其使用条件的证据，而一次成功的任务尝试可能并未检验这些条件。我们提出苏格拉底式技能学习（SSL），将专家教学与学生实践交错进行。教育者引导探究，而学生始终是唯一的环境操作者和技能作者，能够质疑、检验并调整建议。为了测试学习任务之外的重用，我们构建了 SkillTransferBench，包含 35 个任务族和 408 个可执行案例，将公开的变体规范与在全新求解器会话中进行的留出评估配对。在主要的 22 个任务族、155 个案例的队列上，固定窗口 SSL 达到 0.801 的效用值，而无技能时为 0.723，适配的 Trace2Skill（轨迹蒸馏）为 0.761，适配的 SKILL-KD（对比蒸馏）为 0.751。在共享任务族上的三对模型中，交错指导优于专家编写和事后教学。过程分析将指导与学生写作和包修订联系起来；案例研究表明学生测试并完善了所提出的流程以供后续重用。随着对新课程的需求下降，固定窗口审查的成本增加。问题引导 SSL 跟踪技能修订中未解决的问题以选择和聚焦审查，在主要队列上达到 0.805 的效用值，同时比固定窗口 SSL 减少 61.7% 的教育者 token 和 16.9% 的总获取 token。
📄 English Abstract
Agent Skills improve LLM agents by making task knowledge reusable across sessions. Learning transferable Skills requires evidence about a procedure and its conditions of use, which a successful task attempt can leave untested. We introduce Socratic Skill Learning (SSL), which interleaves expert teaching with Student practice. An Educator guides investigation while the Student remains the sole environment operator and Skill author, able to question, test, and adapt advice. To test reuse beyond the learning task, we construct SkillTransferBench, with 35 task families and 408 executable cases, pairing public variation specifications with held-out evaluation in fresh solver sessions. On the main 22-family, 155-case cohort, Fixed-Window SSL reaches 0.801 utility, versus 0.723 without a Skill, 0.761 for adapted Trace2Skill (trajectory distillation), and 0.751 for adapted SKILL-KD (contrastive distillation). Across three model pairs on shared families, interleaved guidance outperforms expert authoring and post-hoc teaching. Process analysis links guidance to Student writing and package revision; a case study shows the Student testing and refining a proposed procedure for later reuse. As demand for new lessons falls, the cost of fixed-window review grows. Issue-Guided SSL tracks unresolved questions across Skill revisions to select and focus review, reaching 0.805 utility with 61.7% fewer Educator tokens and 16.9% fewer total acquisition tokens than Fixed-Window SSL on the main cohort.
10. 游戏中的规模化：面向多样虚拟世界具身智能体的持续预训练
- 英文标题：Scaling in Games: Continued Pre-Training For Embodied Agents In Diverse Virtual Worlds
- arXiv：2610.28698
💡 一句话解读：研究想弄清：在物理世界有效的“数据越多、越多样、模型越大就越好”的规律，是否也适用于视觉风格、规则和操作各不相同的虚拟游戏世界。作者用120种配置系统训练不同规模模型，发现多样性只对相近世界有帮助，并拟合出经验缩放规律。
在物理环境中，扩展数据、多样性和模型容量已提升具身学习效果，因为多样经验仍受共同物理规律约束。这些收益能否延伸到视觉风格、动态、控制和目标各异的虚拟世界仍不清楚。我们通过跨数据量、多样性和模型规模三个维度的持续预训练（CPT）受控研究考察该问题，覆盖0.8B至27B四种模型规模、50至1600小时六种数据预算及从仅Minecraft到多世界语料的五种混合。我们使用三级验证损失和从语料内世界到未见世界的游戏基准评估性能与泛化。结果表明：更多数据提升语料内性能，多样性改善向相关世界的迁移但对远距离世界增益有限，模型规模提升数据效率但在单世界训练下对远距离世界泛化更差；我们观察到早期“接口阶段”具有广泛跨世界收益，后期“世界阶段”收益收窄至语料内和相关世界，并拟合出验证损失与数据量、多样性及模型规模的经验缩放规律；我们的模型在3D世界中与Minecraft专家模型和前沿模型相当，在实时设置中表现最佳。然而，模仿的局限和远距离世界的持续差距表明，构建通用具身智能体需要超越持续预训练的更丰富训练阶段。
📄 English Abstract
Scaling data, diversity and model capacity has improved embodied learning in physical environments, where diverse experiences remain grounded in shared physical laws. Whether these gains extend to virtual worlds with different visual styles, dynamics, controls, and objectives remains unclear. We investigate this question through a controlled study of continued pre-training (CPT) across three axes: data volume, diversity, and model size. Our 120 training configurations span four model sizes from 0.8B to 27B, six data budgets from 50 to 1,600 hours, and five mixtures ranging from Minecraft-only data to diverse multi-world corpora. We assess performance and generalization using three-level validation losses and gameplay benchmarks from in-corpus worlds to unseen worlds. Our results show: (1) more data improves in-corpus performance, diversity improves transfer to related worlds with limited gains in distant worlds, and model size improves data efficiency but generalizes worse to distant worlds under single-world training; (2) we observe an early interface phase with broad cross-world benefits and a later world phase with gains narrowed to in-corpus and related worlds, and fit an empirical scaling law relating validation loss to data volume, diversity and model size; and (3) our models are competitive with Minecraft specialists and frontier models in 3D worlds, and perform best in real-time settings. However, the limits of imitation and persistent gaps in distant worlds suggest that building generalist embodied agents requires richer training stages beyond continued pre-training.
11. AgenticRoboRL：将编程智能体后训练引入智能体机器人
- 英文标题：AgenticRoboRL: Bringing Coding-Agent Post-Training to Agentic Robotics
- arXiv：2610.28589
💡 一句话解读：现有机器人智能系统靠闭源大模型当“大脑”做规划，但无法提升这个大脑本身的能力。作者借鉴编程智能体的后训练思路，提出 AgenticRoboRL 框架，用监督微调加多轮强化学习训练机器人规划器，并通过适配器兼容不同执行方式，使9B参数规划器成功率从18.1%升到71.0%。
智能体机器人系统将规划器与机器人执行框架（harness）结合，以完成复杂操作任务。近期工作通过改进执行框架和外部记忆来提升这类系统，但依赖强大的闭源模型作为规划器，且并未提升规划器自身的能力。相比之下，编程智能体通过后训练模型本身已取得显著进步。因此，我们研究后训练是否也能改进机器人规划器。然而，在不同机器人执行框架上应用后训练需要适配不同的执行机制，包括VLA技能调用和控制代码生成。为此，我们提出 AgenticRoboRL，一个可跨不同执行框架对机器人规划器进行后训练的框架。其训练方案结合了监督微调与多轮强化学习。为适配不同执行机制，针对各执行框架的适配器将记录下来的规划器交互与任务结果关联起来，从而在保持执行组件固定的同时实现统一的强化学习训练流程。在 LIBERO-PRO 上，AgenticRoboRL 在无外部记忆的情况下将9B参数规划器的成功率从18.1%提升至71.0%，接近 Codex（GPT-5.5）配合外部记忆所达到的72.1%。这些结果表明，后训练是改进机器人规划器的有效方法。我们将发布该框架及训练好的规划器检查点。
📄 English Abstract
Agentic robotics systems combine planners with robot harnesses to perform complex manipulation tasks. Recent work improves these systems through better harnesses and external memory, but relies on powerful closed-source models as planners and does not improve the planners' own capabilities. In contrast, coding agents have improved substantially by post-training the model itself. We therefore investigate whether post-training can also improve robot planners. However, applying it across robot harnesses requires accommodating different execution mechanisms, including VLA skill invocation and control-code generation. To this end, we introduce AgenticRoboRL, a framework for post-training robot planners across different harnesses. The training recipe combines supervised fine-tuning with multi-turn reinforcement learning. To accommodate different execution mechanisms, harness-specific adapters connect recorded planner interactions with task outcomes, enabling a common RL training pipeline while keeping execution components fixed. On LIBERO-PRO, AgenticRoboRL raises the success rate of a 9B-parameter planner from 18.1% to 71.0% without external memory, approaching the 72.1% achieved by Codex (GPT-5.5) with external memory. These results demonstrate post-training as an effective approach for improving robot planners. We will release the framework and trained planner checkpoints.
12. 以记忆为中心的AI智能体持续学习
- 英文标题：Memory-Centric Continual Learning for AI Agents
- arXiv：2610.29357
💡 一句话解读：现有AI智能体的记忆系统大多只是“存了能查出来”，并不代表它真的从经验中学到了东西、以后做得更好。本文提出“以记忆为中心的持续学习”（MCCL），用一套分类框架、评估方法和规模分析框架，把记忆从被动查询改造成能推动智能体持续自我改进的机制。
由基础模型驱动的自主智能体能够推理、规划和使用工具，但其固定权重和上下文长度限制使其难以在长期部署中持续从经验中学习。外部记忆系统试图弥补这一缺口，但大多只是被动的检索缓存，评估标准主要是能否回忆已存储的事实或过往交互；记忆能否驱动真正的持续学习仍是一个开放问题。本文提出以记忆为中心的持续学习（MCCL），主张应以记忆是否改善未来行动而非仅以检索效果来评判记忆。MCCL包含三部分：一是三轴分类法，按状态存储位置（substrates）、记忆形式（assets）和管理方式（policies）组织记忆，并定义双环生命周期——内环更新记忆状态，外环依据任务反馈优化管理策略；二是评估协议，衡量干扰下的保持、对变化的适应和任务迁移，以区分真实的行为增益与单纯的上下文扩展；三是记忆扩展框架，对非参数化与参数化记忆建模智能体性能如何随经验、存储容量和计算预算扩展。这些组件共同将智能体记忆从被动查找重新定义为持续学习，为通向自我改进的自主智能体提供了有原则的路径。
📄 English Abstract
Autonomous agents powered by foundation models can reason, plan, and use tools, but their fixed weights and context limits hinder sustained learning from experience over long-term deployment. While external memory systems attempt to bridge this gap, they mostly act as passive retrieval caches, evaluated primarily by whether they can recall stored facts or past interactions. Whether memory can drive genuine continual learning remains an open question. In this paper, we introduce Memory-Centric Continual Learning (MCCL), arguing that memory should be judged by whether it improves future actions, not merely what it retrieves. MCCL has three main parts. First, a three-axis taxonomy organizes memory by where state is stored (substrates), what form it takes (assets), and how it is managed (policies). This defines a two-loop lifecycle that updates memory state in an inner loop while refining management policies in an outer loop based on task feedback. Second, an evaluation protocol measures retention under interference, adaptation to change, and task transfer to isolate genuine behavioral gains from simple context expansion. Third, a memory scaling framework models how agent performance scales with experience, storage capacity, and compute budgets across non-parametric and parametric memory. Together, these components reframe agent memory from passive lookup to continuous learning, offering a principled route toward self-improving autonomous agents.
13. 多模态具身智能体：从计算机使用到机器人使用的统一能力中心视角
- 英文标题：Multimodal Embodied Agents: A Unified Capability-centric Perspective from Computer-Use to Robot-Use
- arXiv：2610.30108
💡 一句话解读：研究要解决的问题是：多模态智能体从“用电脑”转到“用机器人”时，到底发生了什么变化、该如何统一理解。作者提出 PAPAV 框架，把这类智能体的能力归纳为感知、预判、规划、执行、验证五项，并据此比较电脑智能体与机器人系统，指出物理约束让预判和验证能力被忽视。
多模态智能体（MMAs）通过推理、记忆、工具和反馈维持交互，最典型的是计算机使用智能体；而机器人系统（RSs）则在物理动力学下耦合感知与执行。我们将多模态具身智能体（MMEAs）定义为将多模态任务推理与物理动作相结合、并根据由此产生的反馈修正决策的目标导向系统。这引出一个核心问题：当多模态智能体从计算机使用转向机器人使用时，会发生什么变化？我们提出 PAPAV，一个以能力为中心的统一框架，包含五种反复出现的功能：感知当前状态、预判动作效果、规划可行路线、通过接口或身体执行、验证结果。这些能力按功能而非架构定义，使 PAPAV 能识别三者的共同点，进而将 MMEAs 与 MMAs 和 RSs 进行比较。从部分可观测性到结果不可验证性这五项物理约束，使得预先固定的选择更少、纠错空间更小。在 64 个基准测试中，显式评估主要集中在“执行”上；只有 5 个评估“预判”，3 个评估“验证”，二者在很大程度上被任务成功率所掩盖。这些约束也框定了我们识别出的开放挑战。因此，PAPAV 为设计可靠的物理智能体、并衡量全部五项能力（而非仅任务成功率）的进展提供了共同基础。
📄 English Abstract
Multimodal agents (MMAs) sustain interaction through reasoning, memory, tools, and feedback, most visibly as computer-use agents, while robotic systems (RSs) couple sensing and actuation under physical dynamics. We define multimodal embodied agents (MMEAs) as goal-directed systems that couple multimodal task reasoning with physical action and revise decisions from the resulting feedback. This raises a central question: what changes when a multimodal agent moves from computer-use to robot-use? We introduce PAPAV, a unified capability-centric framework of five recurring functions: Perceive the current state, Anticipate action effects, Plan a feasible course, Act through an interface or body, and Verify the outcome. Defined by function rather than architecture, these capabilities let PAPAV identify what all three share and then compare MMEAs with MMAs and with RSs. Five physical constraints, from partial observability to unverifiable outcomes, leave fewer choices fixed in advance and less room to reverse errors. Across 64 benchmarks, explicit evaluation centers on Act; only 5 assess Anticipate and 3 assess Verify, leaving both largely hidden behind task success. These constraints also frame the open challenges we identify. PAPAV therefore provides a common basis for designing reliable physical agents and measuring progress across all five capabilities rather than task success alone.
14. TraceForge：用自身思考进行蒸馏
- 英文标题：TraceForge: Distilling with Your Own Thinking
- arXiv：2610.29680
💡 一句话解读：前沿大模型只返回答案、隐藏推理过程，导致无法用其输出蒸馏训练。现有破解手段违反服务条款。TraceForge 改为：拿题目和老师答案，用生成器模仿学生自家风格补写推理，再用判别器把关，从而合法地造出可训练数据。
用一个 LLM 为另一个 LLM 生成训练数据，是最通用的训练方式之一，且仍是常见做法。一些前沿团队已对自家模型关闭这条路：返回的只是答案或简短摘要，不含背后的详细推理轨迹。这有点讽刺：他们自己正是靠蒸馏人类知识建起来的，蒸馏开源模型也依然司空见惯；限制只针对他们自己的输出。近期工作通过跨模型重放加密块来恢复隐藏推理，虽有效但违反服务商条款，无法作为长期产品基础。我们构建了替代方案：以上下文和教师答案为输入，按学生自身模型家族的风格自行生成推理轨迹。训练分两阶段，先监督后对抗：生成器学习从上下文和答案重建该家族自己的推理轨迹，判别器学习判断推理轨迹有多真实。应用于教师答案时，生成器以该家族风格写出一条轨迹。独立评判器按 0 到 1 打分评估轨迹真实性：我们的平均 0.68，真实轨迹 0.66，判别器从四个候选中选出的达 0.82；这些是真实性分数而非正确率。我们在五个基准上固定其他条件测试该思路：通用任务上五个训练语料大致相当；编程任务上，生成的推理优于 Fable 5 自身的思考内容。在相同 5000 道题和相同教师答案下，最终检查点 HumanEval+ 提升 3.7 分、MBPP+ 提升 7.1 分，且 MBPP+ 的领先在每个检查点都保持。我们推测，用学生家族风格写出的推理更易于其学习。
📄 English Abstract
Using an LLM to generate training data for another is among the most general ways to train one, and it remains common practice. Some frontier teams have closed that route for their own models: what comes back is an answer, or a short summary, without the detailed reasoning trace behind it. That is a little ironic: they were themselves built on distilling human knowledge, and distilling an open-source model is still entirely routine; the restriction applies to their own outputs alone. Recent work recovers the hidden reasoning by replaying encrypted blocks across models. It somehow works, but it breaks the providers' terms of service, so it is not something a product can be built on long-term. We built the alternative. We take the context and the teacher's answer as input, and generate the reasoning trace ourselves, in the style of the student's own model family. Training is two-stage, supervised and then adversarial: a generator learns to regenerate the family's own reasoning traces from their context and answer, and a critic learns how real a reasoning trace looks. Applied to a teacher's answer, the generator writes one in the family's style. An independent judge rates how likely traces are to be genuine on a scale from 0 to 1: ours average 0.68, real traces 0.66, and traces selected by the critic from four candidates 0.82. These are authenticity scores, not correctness rates. We test the idea on five benchmarks with everything else held fixed. On general tasks the five training corpora are about the same; on coding, the generated reasoning beats Fable 5's own thinking content. With the same 5,000 questions and the same teacher answers, HumanEval+ improves by 3.7 points and MBPP+ by 7.1 at the final checkpoint, and the MBPP+ lead holds at every checkpoint. We suspect that reasoning written in the student's family style is easier for it to learn from.
15. 正确的屏幕，错误的转移：世界模型作为GUI智能体的验证器
- 英文标题：Right Screen, Wrong Transition: World Models as Verifiers for GUI Agents
- arXiv：2610.11942
💡 一句话解读：这篇研究要解决的是：图形界面智能体如何判断一次操作后的屏幕变化是否安全。现有方法只检查屏幕本身，但同一个屏幕可能来自正常操作或恶意攻击。作者提出一种世界模型，它直接预测下一步屏幕的内部表示，通过比较预测和实际观察的差异来验证转移是否安全，速度快且准确率高。
点击“登录”后出现登录界面是预期的；而点击“查看订单”后出现同一界面则是攻击。因此，对于GUI智能体而言，安全性是转移（transition）的属性而非屏幕的属性，仅检查屏幕的监控器可以通过重用合法屏幕被绕过。判断转移需要对动作后应有结果的预期。现有的GUI世界模型提供了这种预期，但其输出为文本、代码或图像，因此需要第二个模型来将预期与观察到的屏幕进行比对。我们认为，用于验证的世界模型应在观测编码的空间中进行预测，并提出了LGWM——一种无解码器、以动作为条件的世界模型，它直接预测下一屏幕的表示，并在185万条真实GUI转移数据上训练而无需语义标注。验证简化为向量比较，且同一信号可揭示不匹配是否有害。我们在RSWT-BENCH上评估，该诊断基准中每个凭证屏幕都同时出现在合法和劫持转移下，因此仅看屏幕的检测器按构造即为随机水平。无需训练的打分在每次决策17毫秒下达到0.987 AUC，与最强的闭源VLM相当，且比生成式GUI世界模型高约十个AUC点，延迟低三个数量级以上。残差方向在区分有害与良性违规时达到0.953 AUC，而提示的VLM接近随机。进一步分析表明，预测是可用的未来状态而非异常分数。世界模型主要用作模拟器或规划器；我们的结果指向第三种角色——验证，而在表示空间中进行预测是其自然设计。
📄 English Abstract
A login screen that appears after a tap on Sign in is expected; the same screen after a tap on View order is an attack. For GUI agents, safety is therefore a property of the transition rather than of the screen, and a monitor that inspects only screens can be defeated by reusing a legitimate one. Judging a transition requires an expectation of what should have followed the action. Existing GUI world models provide one, but they output it as text, code, or images, so checking it against the observed screen requires a second model to judge the two. We argue that a world model meant for verification should instead predict in the space in which observations are encoded, and present LGWM, a decoder-free, action-conditioned world model that predicts the representation of the next screen directly, trained without semantic annotation on 1.85M real GUI transitions. Verification reduces to a vector comparison, and the same signal reveals whether a mismatch is harmful. We evaluate on RSWT-BENCH, a diagnostic where each credential screen appears under both a legitimate and a hijacked transition, so detectors that see only the screen are at chance by construction. The training-free score reaches 0.987 AUC at 17 ms per decision, on par with the strongest closed-source VLMs and about ten AUC points above generative GUI world models at over three orders of magnitude lower latency. The residual direction reaches 0.953 AUC at separating harmful from benign violations, where prompted VLMs are near chance. Further analyses show that the prediction is a usable future state rather than an anomaly score. World models have mostly served as simulators or planners; our results point to a third role, verification, for which predicting in representation space is the natural design.