Daily Papers · 2026-10-21
📄 论文日报 | 来源：Hugging Face Daily Papers
🗓 Episode：
1. 从机群经验中学习：一种面向机器人的集体学习范式
- 英文标题：Learning from Fleet Experience: A Collective Learning Paradigm for Robots
- arXiv：2610.25736
💡 一句话解读：机器人部署时出错往往要靠人工远程纠正，但机器人一多就顾不过来。本研究让所有机器人共享各自的执行经验来改进同一个策略：用失败场景去检索机群中成功的轨迹片段作为学习监督，并配一套容错异步训练系统协调收集与更新，在五项真实操作任务上成功率最高达99.2%。
机器人在部署过程中产生的经验，为通过从自身执行中学习来提升策略可靠性提供了机会。然而，通过人工纠正进行学习需要遥操作员监控执行并在策略失败时介入，随着机器人机群规模扩大，这一要求难以满足。我们提出集体经验学习（CEL），一种让已部署机器人共享执行经验以改进共同策略的学习范式。我们通过失败引导的经验检索微调（FERT）实现 CEL：它利用机器人执行中的失败情境，从机群共享的成功经验中检索相关轨迹片段，并将其动作作为策略更新的监督信号，无需收集额外的专家示范或纠正动作。容错异步训练系统（FATS）在机器人机群出现局部中断时协调经验收集、策略优化与策略分发。在五项真实世界操作任务中，所得策略成功率达到93.3%–99.2%，每项任务在未参与训练数据收集的机器人上评估120次。在保持每台机器人收集时间不变的情况下，更大的机群提供更多经验，FERT 在所评估的规模上总体取得更高成功率。在多阶段手机包装任务中，使用96台机器人学习将成功率从79.2%提升至98.3%。这些结果支持将集体经验作为可扩展机器人学习的实用基础，并为在部署期间无需收集额外专家纠正动作即可提升策略可靠性提供了一条路径。
📄 English Abstract
Experience generated during deployment offers an opportunity for robots to improve policy reliability through learning from their own executions. However, learning through human corrections requires teleoperators to monitor execution and intervene when policies fail, a demand that is difficult to meet as robot fleets grow. We introduce collective experience learning (CEL), a learning paradigm in which deployed robots share their execution experience to improve a common policy. We realize CEL through failure-guided experience retrieval fine-tuning (FERT), which uses failure contexts from robot executions to retrieve relevant trajectory segments from the fleet’s shared successful experience and uses their actions as supervision for policy updates, without collecting additional expert demonstrations or corrective actions. A fault-tolerant asynchronous training system (FATS) coordinates experience collection, policy optimization and policy distribution despite local interruptions within the robot fleet. Across five real-world manipulation tasks, the resulting policies achieve success rates of 93.3-99.2%, evaluated over 120 trials per task on robots excluded from training data collection. With collection time per robot held fixed, larger fleets provide more experience and FERT generally achieves higher success rates across the evaluated sizes. On multi-stage phone packaging, learning with 96 robots increases success from 79.2% to 98.3%. These results support collective experience as a practical basis for scalable robot learning and a path towards improving policy reliability during deployment without collecting additional expert corrective actions.
2. MemT-1.0：迈向记忆原生演化
- 英文标题：MemT-1.0: Towards Memory-Native Evolution
- arXiv：2610.26825
💡 一句话解读：现有大模型记不住长期信息，只能靠外挂检索或超长上下文硬撑，既慢又占空间。作者提出MemT-1.0，把“记忆”直接做进模型内部计算，用门控记忆更新和混合存储扩容，并配两阶段训练，让模型边用边记、还能记住操作经验。
记忆原生演化要求将记忆变为基础模型的内在能力，使其能够持续积累并利用信息。为此，我们提出MemT-1.0，一种将记忆内化为模型计算一部分的记忆基础模型新架构。MemT-1.0基于Metis，采用带稳定token级记忆更新机制的Gated DeltaNet-2，并进一步引入混合存储（Mixture-of-Storage）以扩展原生记忆容量。经过审慎的主干准备，我们构建了由持续预训练与后训练组成的两阶段训练流程，并辅以34B token的训练数据以及高效的训练与推理基础设施。我们发布了包含MemT-1.0-Nano与MemT-1.0-Flash的模型系列，并对记忆能力与通用能力进行了全面评估。结果表明，它们在平均表现上优于参数化记忆基线，与检索增强方法相比仍具竞争力，同时基本保持了通用能力。在128K token的历史长度下，MemT-1.0-Nano相较完整上下文KV缓存实现约1.86倍端到端推理加速，并将持久化存储降低66倍。通过内化检索到的程序性经验，MemT-1.0提升了智能体任务的成功率，为迈向记忆原生演化提供了切实一步。
📄 English Abstract
Memory-native evolution requires making memory an intrinsic capability of foundation models, enabling them to continuously accumulate and utilize information. To this end, we introduce MemT-1.0, a new architecture of memory foundation models that internalize memory as part of model computation. Building on Metis, MemT-1.0 employs Gated DeltaNet-2 with a stable token-level memory update mechanism within each turn. We further introduce Mixture-of-Storage to scale native memory capacity. Following a deliberate backbone preparation, we develop a two-stage training pipeline consisting of continued pre-training and post-training, supported by 34B tokens of training data and efficient training and inference infrastructure. We release a model family comprising MemT-1.0-Nano and MemT-1.0-Flash, and conduct a comprehensive evaluation of memory and general capabilities. The results show that they outperform the parametric memory baselines on average and remain competitive with retrieval-augmented methods, while largely preserving general capabilities. At a history length of 128K tokens, MemT-1.0-Nano achieves approximately $1.86\times$ end-to-end inference speedup and a $66\times$ reduction in persistent storage relative to full context KV cache. By internalizing retrieved procedural experience, MemT-1.0 improves task success on agentic tasks, providing a practical step toward memory-native evolution.
3. EgoXperience：从人类经验中学习世界锚定的自我中心表征
- 英文标题：EgoXperience: Learning World-Grounded Egocentric Representations from Human Experience
- arXiv：2610.26625
💡 一句话解读：现有方法难以把第一人称视频里的相机运动、周围场景和手与物体的互动放进同一个一致的三维世界。作者构建了自动标注流程与数据集，并训练出模型，只需一段视频和首帧物体轮廓，就能一次性重建出相机、场景、手和物体在同一坐标系中的位置。
从第一人称（自我中心）视频中重建人类活动，不仅需要恢复各个三维组成部分，还要求观察者运动、周围几何结构与手-物体交互在一致的世界中彼此吻合。实现这种一致性既需要联合重建，也需要对齐的监督信号，而现有方法与数据集对二者的支持都有限。我们提出 EgoXperience，一套涵盖数据采集、自动标注、统一建模与基准评测的全栈范式；其核心是自动标注引擎 EgoX-Engine，它将经过几何验证的物体跟踪与基于 SLAM 的世界对齐相结合，为相机、场景、手和物体生成一致的监督。由此得到 EgoX-Data-6M——一个 50 小时、包含室内外任务驱动操作与移动的数据集，并带有秒级任务描述；我们从中精选一小时数据构成 EgoX-Bench，用于评测联合重建。为将监督转化为重建能力，我们提出 EgoX 模型，使用 EgoX-Data-6M 及其他公开数据集的补充标注训练。给定单目第一人称视频和首帧物体掩码，该模型通过一次前向推理即可在共享世界坐标系中联合重建相机、场景、手和物体的度量三维结构。大量实验表明，其在场景估计和世界空间手-物体位姿恢复上达到最先进水平：与最强基线相比，EgoX 将相机、手和深度误差分别降低 68.9%、65.6% 和 42.9%，物体位姿得分提升 28.0%。
📄 English Abstract
Reconstructing human activity from egocentric video requires more than recovering individual 3D components: the observer’s motion, surrounding geometry, and hand-object interactions must fit together in a consistent world. Achieving this coherence requires both joint reconstruction and aligned supervision, yet existing methods and datasets offer limited support for either. We present EgoXperience, a full-stack paradigm that addresses these needs through data collection, automatic annotation, unified modeling, and benchmarking. Central to EgoXperience is an automatic annotation engine, EgoX-Engine, that couples geometrically verified object tracking with SLAM-based world alignment, yielding consistent supervision for the camera, scene, hands, and objects. This yields EgoX-Data-6M, a 50-hour dataset of task-driven manipulation and locomotion within and across indoor and outdoor environments, with second-level task captions. We carefully select one hour across these activity categories as EgoX-Bench to evaluate joint reconstruction. To translate this supervision into reconstruction, we introduce EgoX, trained with EgoX-Data-6M and complementary annotations from additional public datasets. Given monocular egocentric video and first-frame object masks, our model achieves joint metric reconstruction of the camera, scene, hands, and objects in a shared world coordinate frame with a single forward pass. Extensive experiments demonstrate state-of-the-art performance across scene estimation and world-space hand-object pose recovery. Against the strongest evaluated baselines, EgoX reduces camera, hand, and depth errors by 68.9%, 65.6%, and 42.9%, and improves object pose scores by 28.0%.
4. 扩展用于大脑编码的参数高效微调
- 英文标题：Scaling Parameter-Efficient Fine-Tuning for Brain Encoding
- arXiv：2610.26802
💡 一句话解读：脑编码模型需要预测大脑对刺激的神经反应，但神经数据又贵又少，从头训练或完整微调大模型都很困难。作者用参数高效微调（只训练极少量参数）把超大预训练模型改造成脑编码模型，并利用模型结构一次预测多个时间点来省算力，成功适配了千亿参数级模型，在挑战赛上取得最佳成绩。
脑编码模型旨在预测神经对刺激的反应。最常用的方法是从冻结的预训练骨干网络中拟合读出层。由于神经数据昂贵且有限，用神经数据从头训练大模型或在任务预训练后对其进行完全微调已被证明十分困难。我们表明，参数高效微调（PEFT）使我们能够用少量可训练参数有效地将超大预训练骨干网络适配于脑编码。为使其在计算上可行，我们利用骨干网络的因果结构，在一次前向传播中预测多个时间点，而非使用滑动窗口，从而显著降低特征提取成本。这使我们能够高效微调参数量高达104B的多模态骨干网络，据我们所知这是适配于脑编码的最大模型。采用低秩适配器（LoRA）和轻量读出层训练后，我们的方法在九种不同的多模态骨干网络上持续提升脑编码性能。在与先前最先进方法（SoTA）相同的Qwen3-Omni骨干网络上，我们的方法BrainLoRA在Algonauts 2025挑战赛上取得了已发表的最佳性能。我们还表明，适配后的多模态骨干网络仍具有可解释性：模态归因恢复了以往工作常通过分别构建单模态编码器而获得的典型感觉拓扑结构，且神经适配增加了音频输入的贡献以及骨干网络内部的跨模态混合。为扩展适配模型的数量，我们开发了一种多LoRA内核，可加速在共享骨干网络上联合训练多个LoRA。这使我们能够构建一个包含200多个适配模型的集成，在撰写本文时于公开的Algonauts赛后排行榜上排名第一。总之，我们的结果凸显了PEFT作为一种高效且强大的脑编码范式。
📄 English Abstract
Brain encoding models aim to predict neural responses to stimuli. The most widely used approach fits readouts from frozen pretrained backbones. Using neural data to train large models from scratch or fully fine-tune them after task-pretraining has proven difficult because neural data is costly and limited. We show that parameter-efficient fine-tuning (PEFT) enables us to effectively adapt very large pretrained backbones for brain encoding with a small number of trainable parameters. To make it computationally practical, we exploit the backbone’s causal structure to predict multiple timepoints in a single pass instead of using sliding windows, significantly reducing feature-extraction cost. This enables efficient fine-tuning of multimodal backbones with up to 104B parameters, the largest model adapted for brain encoding to our knowledge. Trained with a low-rank adapter (LoRA) and a lightweight readout, our approach consistently improves brain encoding performance across nine diverse multimodal backbones. On the same Qwen3-Omni backbone as the prior state-of-the-art (SoTA), our method BrainLoRA achieves the best published performance on the Algonauts 2025 Challenge. We also show the adapted multimodal backbones remain interpretable: modality attribution recovers the canonical sensory topography that prior work often obtains by construction from separate unimodal encoders, and neural adaptation increases the contribution of audio inputs and cross-modal mixing within the backbone. To scale out the number of adapted models, we develop a multi-LoRA kernel that accelerates joint training of multiple LoRAs on a shared backbone. This enables an ensemble of 200+ adapted models that ranks first on the public Algonauts post-challenge leaderboard at the time of writing. Together, our results highlight PEFT as an efficient and powerful paradigm for brain encoding.
5. Holi-World：从人类自我中心经验构建整体世界智能
- 英文标题：Holi-World: Building Holistic World Intelligence from Human-Ego Experience
- arXiv：2610.26549
💡 一句话解读：现有AI学习世界知识时，需要大量人工标注，成本高、规模上不去。本研究把人们日常第一视角拍摄的视频自动加工成学习材料：先自动提取场景结构和物体变化，再生成约8万条问答来训练视觉语言模型，并建立独立测试基准，让机器更好地理解动态世界。
人类的第一视角经验为学习世界如何组织、如何通过交互而变化提供了丰富基础。要在规模上实现这一潜力，需要将不断增长的录制数据转化为学习资源，在跨视角与跨时间上连接场景结构、物体语义和交互历史，同时减少对昂贵人工标注的依赖。我们提出HOLI-WORLD，一个连接经验采集、世界表示、自动化数据构建、模型学习和独立评估的集成生态系统。为支持规模化学习，HOLIWORLD-Foundry结合互补的静态与动态感知，自动构建世界记录，将持久的场景结构和演变的物体状态与交互历史及支撑观测相连接。HOLIWORLD-Data将这些来自约100个自采集场景的记录组织为有依据的学习资源，包括约8万条用于训练HOLIWORLD-VL-4B的问答对。对留出录制数据进行的独立人工标注构成HOLIWORLD-Bench，支持评估标注质量以及学习到的空间与时间理解能力。在ScanNet、ScanNet++、Aria Digital Twin和HOLIWORLD-Bench上的实验表明，该方法在静态感知和动态跟踪方面表现强劲，而在HOLIWORLD-Data上训练显著提升了HOLIWORLD-VL-4B在HOLIWORLD-Bench和EgoDynamic4D上的空间与时间推理能力。HOLIWORLD为从人类经验的连续流中扩展学习资源、推进机器对动态世界的理解提供了实用基础。
📄 English Abstract
Human egocentric experience offers a rich foundation for learning how the world is organized and how it changes through interaction. Realizing this potential at scale requires turning growing volumes of recordings into learning resources that connect scene structure, object semantics, and interaction histories across viewpoints and time, while reducing dependence on costly manual annotation. We introduce HOLI-WORLD, an integrated ecosystem connecting experience capture, world representation, automated data construction, model learning, and independent evaluation. To sup- port learning at scale, HOLIWORLD-Foundry combines complementary static and dynamic perception to automatically construct world records that connect persistent scene structure and evolving object states with interaction histories and supporting observations. HOLIWORLD-Data organizes these records from approximately 100 self-collected scenes into grounded learning resources, including approximately 80K QA pairs used to train HOLIWORLD-VL-4B. Independent human annotations of heldout recordings form HOLIWORLD-Bench, supporting evaluation of both annotation quality and learned spatial and temporal understanding. Experiments on ScanNet, ScanNet++, Aria Digital Twin, and HOLIWORLD-Bench demonstrate strong performance in static perception and dynamic tracking, while training on HOLIWORLD-Data substantially improves HOLIWORLD-VL-4B’s spatial and temporal reasoning on HOLI- WORLD-Bench and EgoDynamic4D. HOLIWORLD provides a practical foundation for expanding learning resources from the continuous flow of human experience and advancing machine understanding of the dynamic world.
6. OpenVLHarness：面向真实世界的视觉智能体工具使用与多模态记忆
- 英文标题：OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal Memory
- arXiv：2610.26860
💡 一句话解读：多模态大模型光靠“想”解决不了需要查资料、量数据、看图的任务，加长思考时间也收效甚微。作者提出 OpenVLHarness：给模型配一套工具接口和可随时调取的多模态记忆，让它能取证据、看证据、组合证据，在22个测试上平均提升约10分。
多模态大语言模型（MLLMs）在视觉推理上进展迅速，但许多任务所需的证据难以仅靠内部推理获得：我们在22个基准上的对照研究表明，单纯增加基础模型的推理预算只能带来有限提升。这类任务需要借助异构工具获得有依据的感知、外部知识和数值测量，且工具输出必须能被模型理解并在各推理步骤间保留。我们提出 OpenVLHarness，一个围绕能力层与持久多模态记忆组织的多模态推理框架（harness）。能力层将任务级工具接口与证据渲染相结合，为异构后端提供一致的访问方式，并将其输出转换为模型可读的观察结果；持久多模态记忆则在每次查询过程中把相应的图像、结构化数据和网页引用保留为可寻址的工件。编码接口可在已存储的数据上执行生成的程序，从而实现组合多个工具结果的计算。这些机制共同支持“获取—检查—组合”多模态证据的迭代过程。在22个基准、八种开源与专有基础模型上的评估显示，OpenVLHarness 相比匹配的无工具基线平均提升9.7–12.6分，其中 Qwen3-VL-8B 提升11.2分、Kimi K3 提升9.7分、GPT-6 Astra 提升9.9分。在中等推理强度下使用 GPT-6 Luna 时，OpenVLHarness 比在最高推理强度下运行的基础模型、Codex 和 Visual Sketchpad 高出7.7–8.5分，而模型 token 成本至多为其37%。
📄 English Abstract
Multimodal large language models (MLLMs) have made rapid progress on visual reasoning, yet many tasks require evidence that is difficult to obtain through internal reasoning alone: our controlled study on 22 benchmarks shows that simply increasing the inference budget of the base model yields only modest gains. Such tasks demand grounded perception, external knowledge, and numerical measurements obtained through heterogeneous tools, whose outputs must be made interpretable to the model and retained across reasoning steps. We introduce OpenVLHarness, a multimodal reasoning harness organized around a capability layer and persistent multimodal memory. The capability layer combines task-level tool interfaces with evidence rendering, providing consistent access to heterogeneous backends and converting their outputs into model-readable observations. Persistent multimodal memory retains the corresponding images, structured data, and web references as addressable artifacts throughout each query. A coding interface executes generated programs over stored data, enabling computations that combine results from multiple tools. Together, these mechanisms support an iterative process of acquiring, inspecting, and composing multimodal evidence. Evaluated on 22 benchmarks with eight open and proprietary backbones, OpenVLHarness improves average scores by 9.7-12.6 points over matched no-tool baselines, including +11.2 with Qwen3-VL-8B, +9.7 with Kimi K3, and +9.9 with GPT-6 Astra. With GPT-6 Luna at medium reasoning effort, OpenVLHarness outperforms the base model, Codex, and Visual Sketchpad at maximum effort by 7.7-8.5 points while incurring at most 37% of their model-token cost.
7. ScaleNav：用于语言条件局部导航的可扩展互联网视频预训练
- 英文标题：ScaleNav: Scalable Internet Video Pretraining for Language-Conditioned Local Navigation
- arXiv：2610.25690
💡 一句话解读：机器人要按一句话指令走到眼前看到的目标并停下，但训练这种能力缺真实数据：真机采集贵，仿真数据搬到现实又不好用。作者改用网上大量第一人称视频，先让模型学会“朝目标走时画面怎么变”，再用仿真数据教它输出可执行动作，最后整体微调，在仿真和真实四足机器人上都验证了效果。
本文研究语言条件局部导航：机器人需导航至初始观测中可见的、由文本指定的目标，并在所描述位置停下。端到端导航策略需要多样化训练数据才能跨场景泛化，但真实机器人数据采集成本高，仿真数据又存在仿真到现实的差距。互联网第一人称视频提供了可扩展的真实世界导航经验来源。我们的关键思路是从互联网第一人称视频中学习目标导向的视觉动态，并利用这些动态支持机器人动作生成。为此，我们提出 ScaleNav——一个统一的世界-动作框架，采用三阶段训练：首先在这些视频上进行指令条件视频预训练，学习相机接近指定目标时观测如何演变；随后用带动作标签的仿真数据进行动作对齐，训练模型预测可执行的动作块；最后联合微调所有可训练组件以完成导航。ScaleNav 在 Short-Horizon OVON 的单视角适配版本上达到 86.5% 的成功率，在我们的 HabitatGS-LN 基准上达到 94.9%，在两个基准上均优于所评估的基线。我们还在真实四足机器人上部署并评估了 ScaleNav，结果支持该模型泛化到真实世界导航的能力。
📄 English Abstract
We study language-conditioned local navigation, where a robot must navigate to a text-specified target visible in its initial observation and stop at the described location. End-to-end navigation policies require diverse training data to generalize across scenes, yet real-robot data is costly to collect and simulated data leaves a sim-to-real gap. Internet egocentric videos offer a scalable source of real-world navigation experience. Our key insight is to learn goal-directed visual dynamics from Internet egocentric videos, and use these dynamics to support robot action generation. To this end, we propose ScaleNav, a unified world-action framework with three-stage training. We first perform instruction-conditioned video pretraining on these videos to learn how observations evolve as the camera approaches instructed targets across diverse real-world scenes. Action alignment then uses action-labeled simulated data to train the model to predict executable action chunks. Joint fine-tuning subsequently optimizes all trainable components together for navigation. ScaleNav achieves success rates of 86.5% on a single-view adaptation of Short-Horizon OVON and 94.9% on our HabitatGS-LN benchmark, outperforming the evaluated baselines on both benchmarks. We further deploy and evaluate ScaleNav on a real quadruped robot, with results supporting the model’s ability to generalize to real-world navigation. Code and videos are available at this https URL
8. SpecMRL：在参数化频谱中重新设计套娃表示学习
- 英文标题：SpecMRL: Redesign Matryoshka Representation Learning within Parametric Spectrum
- arXiv：2610.26824
💡 一句话解读：现有套娃式表示学习（MRL）让模型输出可长可短的特征向量，但没保证短向量部分足够好用。作者提出 SpecMRL，把特征空间按重要性重新组织成频谱形式，让截取前段时更有效。实验显示各压缩尺寸都优于原方法，仅用一半维度就超过原方法全维度表现。
套娃表示学习（MRL）通过联合优化嵌套前缀来支持灵活的表征尺寸。然而，该设计缺乏显式约束来确保前段部分提供合适的紧凑编码。为此，我们提出 SpecMRL，它重新组织特征空间以实现更有效的前缀读取。具体而言，我们用一个按特征重要性组织的、由参数导出的频谱基（spectral basis）扩展 MRL；其余部分遵循标准 MRL，前缀读取在学习到的频谱空间中进行。在 MTEB 上的实验表明，SpecMRL 在所有评估的缩减维度上都优于 MRL，且在压缩更激进时增益更大。这些结果表明，学习到的频谱基能更好地优先安排信息量大的方向，从而实现更有效的表征压缩。值得注意的是，SpecMRL 拓展了 MRL 的性能前沿，仅用一半的嵌入维度就超过了全维度的 MRL。
📄 English Abstract
Matryoshka Representation Learning (MRL) is designed to support flexible representation sizes by jointly optimizing nested prefixes. While effective, this design lacks an explicit constraint to ensure that the leading portion provides a suitable compact encoding. To address this, we propose SpecMRL, which reorganizes the feature space to enable more effective prefix readout. Specifically, we extend MRL with a parameter-derived spectral basis organized with feature importance. SpecMRL otherwise follows standard MRL, with prefix readout performed in the learned spectral space. Experiments on MTEB show that SpecMRL outperforms MRL at all evaluated reduced dimensions, with larger gains under more aggressive compression. These results suggest that the learned spectral basis better prioritizes informative directions, enabling more effective representation compression. Notably, SpecMRL extends the performance frontier of MRL, surpassing full-dimensional MRL with only half the embedding dimensions.
9. 通过执行框架与策略优化实现自进化编程智能体
- 英文标题：Self-Evolving Coding Agents via Harness-Policy Optimization
- arXiv：2610.22791
💡 一句话解读：编程智能体的表现同时取决于“大脑”（策略模型）和“工具与流程”（执行框架），但以往只优化其中一方，导致两者不匹配。本文提出 Self-HPO：让智能体每轮先分析自己的失败、提出新的执行框架，再在新框架下用强化学习更新策略，交替迭代、互相促进，从而显著提升解题成功率。
大语言模型（LLM）智能体在编程任务中展现出可观潜力，其表现由策略模型与中介其与环境交互的执行框架（harness）共同决定。事实上，执行框架与策略相互促进、协同演化，只优化其中一个组件是次优的——为某一模型检查点调优的执行框架可能无法针对更新后策略的失败模式，而策略学习又受限于固定执行框架所产生的轨迹。为此，我们提出 Self-HPO，一种面向编程智能体的递归自我改进框架，交替进行执行框架优化与策略优化，并以不断演化的策略自身作为执行框架的提出者。具体而言，每一轮中当前策略先分析自身失败并提出新的执行框架候选（H 步），随后在选定的执行框架下通过强化学习（RL）更新策略（P 步），训练中识别出的困难任务构成下一轮执行框架搜索集，使执行框架优化始终针对策略不断演化的失败模式。在三个编程基准上，Self-HPO 持续优于仅优化执行框架和仅优化策略的基线，将 Qwen3.5-35B-A3B 的平均解决率从 57.8% 提升至 64.8%，且增益可泛化到分布外场景。进一步分析表明，两个优化后的组件即使与各自的原始版本搭配仍能保持增益，而二者的联合增益超过各自增益之和。Self-HPO 还在基础智能体之上提升了可靠性（Pass@1）并扩展了能力前沿（Pass@k）。此外，我们表明，若以更强的外部模型（如 Opus 4.8）作为执行框架提出者，同一框架可带来进一步的性能提升。
📄 English Abstract
Large language model (LLM) agents have shown promising potential in coding tasks, and their performance is jointly determined by the policy model and the harness that mediates their interaction with the environment. Indeed, harness and policy are mutually reinforcing and evolving only one component is suboptimal—a harness tuned for one checkpoint may not target the failure modes of an updated policy, while policy learning remains limited to the trajectories elicited by a fixed harness. In light of this, we propose Self-HPO, a recursive self-improving framework for coding agents that alternates harness and policy optimization, using the evolving policy as its own harness proposer. Specifically, in each round, the current policy first analyzes its own failures and proposes new harness candidates (H-step). The policy is then updated through reinforcement learning (RL) under the selected harness (P-step), and hard tasks identified during training form the next harness search set, keeping harness optimization targeted to the policy’s evolving failure modes. Across three coding benchmarks, Self-HPO consistently outperforms harness-only and policy-only baselines, improving Qwen3.5-35B-A3B’s average resolve rate from 57.8% to 64.8%, with gains generalizing to out-of-distribution settings. Further analysis shows that both optimized components retain gains even when paired with their base counterparts, while their joint gain exceeds the sum of their individual gains. Self-HPO also improves reliability (Pass@1) and expands the capability frontier (Pass@k) over the base agent. Additionally, we show that with a stronger external model (e.g., Opus 4.8) as the harness proposer, the same framework yields a further performance boost.
10. MT Forcing：面向稳定且一致的多轮图像编辑的自回归扩散方法
- 英文标题：MT Forcing: Towards Stable and Consistent Multi-Turn Image Editing with Autoregressive Diffusion
- arXiv：2610.25404
💡 一句话解读：多轮图像编辑中，改几次后画面就会变差、内容跑偏。作者提出MT Forcing训练方法，用一次性完成所有编辑的“老师”来指导模型，并加入跨轮一致性约束，让多轮编辑更稳定、更一致。
多轮图像编辑旨在根据一系列指令修改图像，从而为用户提供更有效的交互式视觉内容创作。然而，现有方法往往在仅几轮编辑后就出现明显漂移，表现为两方面：1）视觉质量不稳定，伪影累积、细节退化；2）图像内容不一致，保留区域和先前编辑被意外改变。为解决这些挑战，我们提出MT Forcing，一种面向基于自回归扩散的统一多模态模型（UMMs）的多轮图像编辑训练新范式。具体而言，我们提出单轮分布匹配蒸馏（Single-Turn DMD），用一个在单轮内对源图像施加累积编辑的教师模型来监督学生的自展开，从而提供视觉质量指导且不产生递归误差累积。此外，我们引入教师强制引导（Teacher-Forcing Guidance），在预热阶段监督参考轨迹上依赖历史的转移，并在蒸馏阶段作为正则化项，以促进跨轮内容一致性，超越仅靠逐轮分布匹配所能保证的程度。大量实验表明，我们的方法显著提升了UMMs的多轮图像编辑能力，带来更稳定的视觉质量和更强的跨轮一致性。
📄 English Abstract
Multi-turn image editing aims to modify an image according to a sequence of instructions, enabling more effective interactive visual content creation for users. However, existing methods often exhibit noticeable drift after only a few editing turns, manifesting in two aspects: 1) unstable visual quality, with accumulating artifacts and degraded details; and 2) inconsistent image content, with unintended changes to preserved regions and prior edits. To address these challenges, we propose MT Forcing, a novel training paradigm for multi-turn image editing with unified multimodal models (UMMs) based on autoregressive diffusion. Specifically, we propose Single-Turn Distribution Matching Distillation (Single-Turn DMD), which supervises student self-rollouts with a teacher that applies cumulative edits to the source image in a single turn, providing visual-quality guidance without recursive error accumulation. Additionally, we introduce Teacher-Forcing Guidance, which supervises history-dependent transitions on reference trajectories during warmup and as a regularizer during distillation, encouraging cross-turn content consistency beyond what per-turn distribution matching alone can ensure. Extensive experiments demonstrate that our approach significantly improves the multi-turn image editing capabilities of UMMs, yielding more stable visual quality and stronger cross-turn consistency.
11. OR-License：评估LLM智能体作为真实世界运筹学实践者
- 英文标题：OR-License: Evaluating LLM Agents as Real-World Operations Research Practitioners
- arXiv：2610.24087
💡 一句话解读：现有测试都从整理好的题目出发，看不出大模型能否直接读懂客户原始材料并做出专业运筹学方案。作者用10个行业、31个真实付费项目（总额368万美元）建了端到端测试，保留原始需求和交付格式，并配一个只提供业务事实、不给专业知识的模拟客户。结果显示顶尖模型在部分目标值上超过专家，但综合质量仍不及专家，主要卡在建模和实现环节。
大型语言模型（LLM）智能体能否不经专家预先重构，直接从客户原始材料给出专家级运筹学（OR）方案？现有OR基准多从自包含描述或预设优化任务开始，未能体现将原始业务材料转化为经专业验证方案的过程。我们提出OR-License，这是一个端到端基准，涵盖来自付费OR项目（总额368万美元）的10个工业场景共31个实例，在脱敏后保留原始需求、输入数据、交付格式和信息边界。为研究信息获取，我们引入ORSim——一个受控客户模拟器，只披露业务事实而不提供OR专业知识。隐藏评估检查可行性，并将目标质量与基准构建前客户已接受的原始专家方案比较。部署调整质量（DAQ）对实例的专家相对质量取平均，不可行交付记为零。在11个使用编码智能体框架的前沿LLM中，若干模型在条件目标质量上超过专家参考，但无一在DAQ上匹敌；最强的Claude Code主配置严格可行性达88.39%，DAQ为0.87（专家参考为1）。客户信息获取不仅限于澄清，还延伸到方案开发与验证，部分后期阶段确认问题解决的比例更高。轨迹分析识别出建模与实现失败，而在可评估任务上，任务均衡的研究链完成率在各模型间为24%至45%。这些发现表明，可靠的长程智能体需要持续消解信息缺口、忠实建模问题并进行有证据关联的自我改进。
📄 English Abstract
Can large language model (LLM) agents deliver expert-level operations research (OR) solutions directly from original client materials, without prior expert reformulation? Existing OR benchmarks typically begin with self-contained descriptions or predefined optimization tasks, underrepresenting the translation of original business materials into professionally validated solutions. We introduce OR-License, an end-to-end benchmark of 31 instances across 10 industrial scenarios from paid OR engagements totaling USD 3.68 million. It preserves original requirements, input data, deliverable formats, and information boundaries after de-identification. To study information acquisition, we introduce ORSim, a controlled client simulator that discloses business facts without supplying OR expertise. Hidden evaluation checks feasibility and compares objective quality against original expert solutions accepted by clients before benchmark construction. Deployment-Adjusted Quality (DAQ) averages expert-relative quality across instances, assigning zero to infeasible deliveries. Across 11 frontier LLMs using coding-agent harnesses, several exceed the expert reference in conditional objective quality, but none matches its DAQ; the strongest primary Claude Code configuration achieves 88.39% strict feasibility and a DAQ of 0.87 against the expert reference of 1. Client information acquisition extends beyond clarification into solution development and validation, with some later stages showing higher confirmed issue-resolution proportions. Trajectory analysis identifies formulation and implementation failures, while task-equalized research-chain completion rates range from 24% to 45% across models on evaluable tasks. These findings motivate sustained information-gap resolution, faithful problem formulation, and evidence-linked self-improvement for reliable long-horizon agents.
12. CoworkEnv：面向长时程协作智能体的训练环境合成
- 英文标题：CoworkEnv: Synthesizing Training Environments for Long-Horizon Cowork Agents
- arXiv：2610.24960
💡 一句话解读：想让AI像同事一样多轮协作、调用工具并遵守流程，但缺少训练素材。作者做了个自动生成训练环境的框架CoworkEnv：按工作流要求造任务、自动校验并修复，再用教师示范和学生试错分别做监督微调和强化学习，训练出小型模型PocketWorker，在协作评测上明显提分。
AI同事必须协调工具、跨轮次沟通，并在遵守工作流约束的同时对变化的工作区状态作出反应。为这些能力构建训练经验，需要连贯的请求、服务记录、工具接口和评分检查。我们提出CoworkEnv，一个以能力为导向的框架，用于为AI同事合成可执行的训练环境。工作流需求指导任务构建，带反馈修复的程序化验证则检查所生成的规格与评分行为。教师执行产生精选示范用于监督微调（SFT），学生交互则提供用于强化学习（RL）的轨迹（rollouts）与奖励。利用这些资源，我们训练了紧凑的PocketWorker模型。对于Qwen3-VL-8B-Instruct，SFT在Claw-Eval上将平均分从46.6提升到64.1，其中多轮任务提升31.9分。仅用SFT扩展到ClawMark，表明其适用于多种同事基准。我们进一步比较了在改变奖励处理和信用分配的不同配置下的RL训练动态。训练奖励的提升为所构建环境支持策略优化提供了初步证据。
📄 English Abstract
AI coworkers must coordinate tools, communicate across turns, and act on changing workspace state while respecting workflow constraints. Constructing training experience for these capabilities requires coherent requests, service records, tool interfaces, and grading checks. We present CoworkEnv, a capability-guided framework for synthesizing executable training environments for AI coworkers. Workflow requirements guide task construction, and programmatic validation with feedback repair checks generated specifications and grading behavior. Teacher executions yield curated demonstrations for supervised fine-tuning (SFT), while student interactions provide rollouts and rewards for reinforcement learning (RL). Using these resources, we train compact PocketWorker models. For Qwen3-VL-8B-Instruct, SFT raises the mean score from 46.6 to 64.1 on Claw-Eval, including a 31.9-point gain on Multi-turn tasks. An SFT-only extension to ClawMark demonstrates applicability across coworker benchmarks. We further compare RL training dynamics across configurations that vary reward processing and credit assignment. Improving training rewards provide preliminary evidence that the constructed environments support policy optimization.
13. AnyGoal：论世界模型在专家轨迹之外的泛化
- 英文标题：AnyGoal: On World Model Generalization Beyond Expert Rollouts
- arXiv：2610.26844
💡 一句话解读：现有世界模型在专家演示之外的新任务上表现不佳，因为它们只会模仿见过的状态-动作组合。作者构建了AnyGoal基准，通过组合熟悉的状态和动作来测试模型能否泛化，并发现当前模型普遍失败。
世界模型的目标是捕捉环境的动态变化，这使得智能体能够借助世界模型进行规划和模拟行动方案。然而，要使这些用途可靠，学习到的动态必须在专家轨迹中观察到的特定状态-动作组合之外表现良好。尽管如此，数据覆盖、动态泛化与下游规划性能之间的关系仍未得到充分探索。我们通过组合泛化的视角研究这一关系：世界模型能否将熟悉的状态和动作因素重新组合成有效的动态？为了回答这个问题，我们引入了AnyGoal，一个基于广泛使用的世界模型环境构建的目标偏移任务变体基准，并附带一个开源库。每个任务的设计都在一个已学习底层动态的世界模型的能力范围内：单个状态因素和动作仍在其训练支持范围内，但它们的组合在专家轨迹中代表性不足。在多个环境和世界模型家族中，我们发现当前模型在这些重新组合下持续表现出失败。通过检查不同模型家族、交互机制和评估指标下的动态和表示失败，我们研究了当前世界模型在哪些方面能够泛化，以及在目标偏移的预测和规划方面仍存在哪些挑战。总之，这个框架、基准和分析为评估世界模型向新规划任务泛化的进展提供了基础。
📄 English Abstract
The goal of a world model is to capture the dynamics of an environment. This has the great appeal that it allows agents to plan and simulate a course of actions with the help of a world model. For these uses to be reliable, however, the learned dynamics must behave well beyond the specific state-action combinations observed in expert rollouts. Despite this, the relationship between data coverage, dynamics generalization, and downstream planning performance remains underexplored. We study this relationship through the lens of compositional generalization: can a world model recombine familiar state and action factors into valid dynamics? To answer this, we introduce AnyGoal, a benchmark of goal-shifted task variants built from widely used world-model environments, together with an open-source library. Each task is designed to be within reach of a world model that has learned the underlying dynamics: individual state factors and actions remain within their training support, but their combinations are underrepresented in expert trajectories. Across multiple environments and world model families, we find that current models consistently exhibit failures under these recombinations. By examining dynamics and representation failures across model families, interaction regimes, and evaluation metrics, we study where current world models generalize and where challenges remain for goal-shifted prediction and planning. Together, this framework, benchmark, and analysis provide a basis for evaluating progress toward world models that generalize to new planning tasks.
14. RoAR-VAE：通过鲁棒预测提升潜变量可扩散性
- 英文标题：RoAR-VAE: Improving Latent Diffusability through Robust Prediction
- arXiv：2610.26216
💡 一句话解读：图像生成模型需要先把图片压缩成“潜变量”再学习生成，但压缩后的信息若不好学，生成效果就差。本文发现：只要让模型能从被破坏的潜变量里恢复语义和原图，就能提升可学性，据此提出 RoAR-VAE，兼顾重建质量与生成性能。
一个有效的视觉分词器（visual tokenizer）必须在保持图像保真度的同时，产生下游扩散模型能够高效学习的潜变量（latents）。已有工作通过语义对齐或分词器—扩散联合训练来提升潜变量可扩散性（latent diffusability）。我们重新审视代表性方法 REPA-E，发现在匹配设置下，去掉其扩散目标和专用生成模块、仅保留从损坏潜变量进行语义预测，即可获得相当的下游生成性能。这一观察引出“鲁棒预测”的一般原则：与生成相关的信息应能从损坏的潜变量中恢复。我们以两个互补目标实现该原则：RobustAlign 从损坏潜变量预测预训练语义特征以保留高层结构，RobustRecon 重建干净图像以保留细粒度空间与外观信息，二者共同构成 RoAR-VAE。分析表明，鲁棒预测通过让有用信息在高方差潜变量方向上更易获取（扩散模型往往更早学习这些方向）来提升可扩散性。实验上，在相同训练预算下，RoAR-VAE 在生成基准上的平均性能比最强基线分词器高 2.19 分，小文本重建准确率比 FLUX.1-VAE 高 16.93 个百分点，表明其在重建保真度与下游生成可学习性之间取得了良好平衡。
📄 English Abstract
An effective visual tokenizer must preserve image fidelity while producing latents that downstream diffusion models can learn efficiently. Prior work improves latent diffusability through semantic alignment or joint tokenizer–diffusion training. Revisiting REPA-E, a representative approach, we find that removing its diffusion objective and dedicated generative blocks while retaining semantic prediction from corrupted latents yields comparable downstream generation performance under a matched setting. This observation motivates a general principle of robust prediction: generation-relevant information should remain recoverable from corrupted latents. We instantiate this principle with two complementary objectives. RobustAlign predicts pretrained semantic features from corrupted latents to preserve high-level structure, while RobustRecon reconstructs the clean image to retain fine-grained spatial and appearance information. Together, they form Robust Alignment and Reconstruction VAE (RoAR-VAE). Our analysis suggests that robust prediction improves diffusability by making useful information more accessible in high-variance latent directions, which diffusion models tend to learn earlier. Empirically, RoAR-VAE improves average generation benchmark performance by 2.19 points over the strongest baseline tokenizer under the same training budget, while improving small-text reconstruction accuracy by 16.93 percentage points over FLUX.1-VAE. These results demonstrate that RoAR-VAE achieves a favorable balance between reconstruction fidelity and downstream generative learnability.
15. InfiniDream：通过程序化生成开放式照片级真实感3D世界
- 英文标题：InfiniDream: Open-Ended Photorealistic 3D Worlds Through Procedural Generation
- arXiv：2610.26853
💡 一句话解读：现有AI生成3D内容要么太简单、要么依赖人工写好的固定模板，难以自由创造新类型资产。作者提出InfiniDream：让视觉语言模型借助已有程序化算法生成可编辑、多样化的照片级真实感3D资产，并搭建从文字直接生成大规模3D世界的流程，实现精确摆放和动画。
程序化生成在合成数据创建和游戏开发中发挥关键作用，相比静态网格表示具有更强的可控性、可编辑性和多样性。然而，基于视觉语言模型（VLM）的代码生成往往只能产出较简单的3D资产，而成熟的照片级真实感程序化系统需要大量人工开发。现有的VLM引导程序化方法在很大程度上仍受限于预定义的工厂和API，限制了其创建新颖资产类型的能力。我们提出InfiniDream，一个由VLM驱动的开放式程序化3D世界生成框架。首先，我们提出符号约束资产生成（Symbolically Constrained Asset Generation），利用现有程序化算法创建新颖的照片级真实感资产，同时保持可编辑性和基于种子的多样性。其次，我们开发了一条端到端的文本到世界（text-to-world）流程，将语义场景规划与确定性视锥引导网格放置（Deterministic Frustum-Guided Grid Placement）相结合，构建具有精确物体放置、程序化填充和动画的大规模3D世界。在100个物体和20个场景生成任务上的大量评估表明，该方法在四项感知指标上均有提升。多样性分析、消融实验和人类偏好评估进一步验证了该框架的有效性。
📄 English Abstract
Procedural generation plays a key role in synthetic data creation and game development, offering greater controllability, editability, and diversity than static mesh representations. However, VLM-based code generation often produces relatively simple 3D assets, while sophisticated photorealistic procedural systems require extensive manual development. Existing VLM-guided procedural methods remain largely constrained by predefined factories and APIs, limiting their ability to create novel asset types. We introduce InfiniDream, a VLM-driven framework for open-ended procedural 3D world generation. First, we propose Symbolically Constrained Asset Generation, which leverages existing procedural algorithms to create novel, photorealistic assets while preserving editability and seed-based diversity. Second, we develop an end-to-end text-to-world pipeline that combines semantic scene planning with Deterministic Frustum-Guided Grid Placement to construct large-scale 3D worlds with precise object placement, procedural population, and animation. Extensive evaluations on 100 object and 20 scene generation tasks demonstrate improvements across four perceptual metrics. Diversity analysis, ablation studies, and human preference evaluations further validate the effectiveness of our framework. Project page: this https URL