Daily Papers · 2026-09-19
📄 论文日报 | 来源：Hugging Face Daily Papers
🗓 Episode：2026.09.18 | V4.1-Flash压缩KV提效；SoL-Pi优化智能体降本
1. DeepSeek-V4.1-Flash：将 KV 缓存压缩推向极限
- 英文标题：DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
- arXiv：2609.19969v1
💡 一句话解读：长任务智能体让模型要读的输入越来越长，算得慢、存不下、传得贵成了降本瓶颈。本文提出 DeepSeek-V4.1-Flash：一种多模态混合专家模型，支持百万级上下文，通过压缩注意力复用和低精度缓存把 KV 缓存压到极小，同时性能更强。
长时程智能体（long-horizon agents）的广泛采用使模型工作负载日益偏向输入密集型。尽管已有工作大幅降低了长上下文计算成本，预填充（prefill）仍然计算昂贵，且庞大的 KV 缓存持续给 HBM 与 SSD 容量及数据传输带宽带来压力。这些计算、存储与带宽需求共同构成了进一步降低部署成本的主要瓶颈。为应对这一挑战，我们提出 DeepSeek-V4.1-Flash，一种多模态混合专家（MoE）模型，主干参数 552B，支持最长一百万 token 的上下文。借助其因果编码器-解码器（CED）架构，该模型在解码时每 token 激活 16B 参数，而在预填充时仅激活 8B 参数，显著提升了智能体工作负载的成本效率。为将 KV 缓存压缩推向极限，DeepSeek-V4.1-Flash 将压缩稀疏注意力 2（CSA2）中的跨层 KV 缓存复用与 FP4 KV 缓存相结合。这些设计将其全局 KV 缓存占用（始终位于 HBM）降至每 token 890 字节，约为 DeepSeek-V4-Flash 相应占用的 1/4。此外，通过名为 SWA Bounded Replay 的专门部署优化，DeepSeek-V4.1-Flash 将其持久化 KV 缓存占用（始终位于 SSD 或主机内存）降至 DeepSeek-V4-Flash 的约 1/8。尽管 KV 缓存占用大幅缩小，该模型性能仍显著优于基线。此外，我们精简了 DeepSeek-V4 架构并引入若干高效架构扩展。我们在包含 45T token 的多模态语料上预训练 DeepSeek-V4.1-Flash，并进行全面后训练，在多种基于文本与多模态的智能体场景中取得强劲表现。模型检查点见 https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash。
📄 English Abstract
The widespread adoption of long-horizon agents has made model workloads increasingly input-heavy. Although prior work has substantially reduced the cost of long-context computation, prefill remains computationally expensive, and large KV caches continue to strain HBM and SSD capacity and data-transfer bandwidth. Together, these compute, storage, and bandwidth demands constitute the primary bottleneck to further lowering deployment costs. To address this challenge, we introduce DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts (MoE) model with 552B backbone parameters and support for contexts of up to one million tokens. With its Causal Encoder-Decoder (CED) architecture, the model activates 16B parameters per token during decode but only 8B parameters during prefill, substantially improving cost efficiency for agentic workloads. To push the limits of KV cache compression, DeepSeek-V4.1-Flash combines cross-layer KV cache reuse in Compressed Sparse Attention 2 (CSA2) with FP4 KV caching. These designs reduce its global KV cache footprint (always in HBM) to 890 bytes per token, roughly 1/4 of the corresponding footprint of DeepSeek-V4-Flash. Further, through a dedicated deployment optimization known as SWA Bounded Replay, DeepSeek-V4.1-Flash reduces its persistent KV cache footprint (always on SSD or in host memory) to roughly 1/8 of that of DeepSeek-V4-Flash. Despite its much smaller KV cache footprint, the model delivers substantially better performance than the baseline. In addition, we streamline the DeepSeek-V4 architecture and introduce several efficient architectural extensions. We pretrain DeepSeek-V4.1-Flash on a multimodal corpus comprising 45T tokens and conduct comprehensive post-training, yielding strong performance across diverse text-based and multimodal agentic scenarios. Model checkpoints are available at https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash.
2. SoL-Pi：递归扩展自动化研究循环以实现高效智能体执行框架
- 英文标题：SoL-Pi: Recursively Scaling Auto-Research Loops for Efficient Agent Harness
- arXiv：2609.20519v1
💡 一句话解读：这篇研究要解决的是：让AI编程助手在长时间自主干活时，别浪费太多“字数”（token），否则成本高、难以持续自我提升。方法是让AI在多种环境里反复自动试验，筛选出四种好用的机制，组成SoL-Pi框架。结果在51项测试中，效果不输原版，但token用量少了近一半，API费用省了约三分之一。
随着编程智能体（coding agents）从有监督的代码补全转向无人值守、全天候的探索，其工作也从孤立的预测扩展为包含推理、工具使用和反馈的长轨迹。因此，token效率对于规模化递归自我改进变得至关重要。我们在框架（harness）层面采用受递归自我改进（RSI）启发的方法，在越来越多样的环境中扩展自动研究循环以进行框架部署。在这种规模下，该过程产生了可复用的改进，这些改进能够迁移到其开发环境之外，推动自动框架发现走向生产级成果。有四种机制在筛选中胜出并构成SoL-Pi，涵盖动作执行、上下文压缩、观察处理和委托阅读。在51项任务的EdgeBench评估中，SoL-Pi在GPT-5.6 Sol和Opus 5上取得了与Pi相当的性能，同时将记录的token流量减少了44.7-49.0%，API成本降低了约三分之一。换言之，相对于原生Codex和Claude Code框架，估计每小时可节省8.75-13.50美元，相对于Pi可节省4.36-5.71美元。
📄 English Abstract
As coding agents move from supervised code completion to unattended, around-the-clock exploration, their work expands from isolated predictions into long trajectories of reasoning, tool use, and feedback. Token efficiency therefore becomes important for scaling recursive self-improvement. We take an RSI-inspired approach at the harness layer, scaling auto-research loops across increasingly numerous and diverse environments for harness rollouts. At this scale, the process yields reusable improvements that transfer beyond their development setting, moving automated harness discovery toward production-level outcomes. Four mechanisms survive selection and form SoL-Pi, spanning action execution, context compaction, observation handling, and delegated reading. On the 51-task EdgeBench evaluation, SoL-Pi achieves performance comparable to Pi across GPT-5.6 Sol and Opus 5 while reducing recorded token traffic by 44.7-49.0% and API cost by about one third. In other words, estimated hourly savings are $8.75-$13.50 relative to native Codex and Claude Code harnesses, and $4.36-$5.71 relative to Pi.
3. 当 EOS token 不一致：理解在线策略蒸馏中的长度膨胀
- 英文标题：When EOS Tokens Disagree: Understanding Length Inflation in On-Policy Distillation
- arXiv：2609.20511v1
💡 一句话解读：在线策略蒸馏中，学生模型回答会变得异常冗长，甚至耗尽生成预算。研究发现，学生和老师模型对“结束符”（EOS token）的偏好不一致是重要原因：双方可能用不同的结束符表示“说完了”。把功能相同的结束符统一视为同一个停止动作，能显著缓解这种长度膨胀。
我们研究了在线策略蒸馏（OPD）中的长度膨胀问题，即学生模型的回答可能变得过长，甚至耗尽生成预算。我们发现，基础学生模型与后训练教师模型之间的终止符不匹配（termination-token mismatch）是导致该行为的一个重要原因。在 Qwen3、Llama 和 Gemma 上，即使两个模型声明的停止集合相同，它们也可能将停止概率放在不同的 EOS token 上。这种不匹配会抑制学生偏好的终止动作，同时无法可靠地传递教师偏好的替代动作。我们表明，仅对齐解码停止集合是不够的，而将功能等价的 EOS token 视为共享的语义停止动作，可显著缓解所有三个模型家族中由不匹配引发的长度膨胀。为了进一步理解终止行为在训练中如何演变，我们研究了不同 K2-Horizon 训练阶段的 OPD。这种分阶段分析表明，终止偏好可能在训练过程中发生显著变化，同时揭示了 OPD 运行后期出现的一种独特的长度膨胀，这种膨胀在终止对齐后仍然存在。总之，这些结果将终止不匹配确定为 OPD 长度动态的一个重要但非全部来源。我们发布了包含所提出的终止处理修正的实现。
📄 English Abstract
We study length inflation in on-policy distillation (OPD), where student responses can become excessively long and even exhaust the generation budget. We identify \emph{termination-token mismatch} between base students and post-trained teachers as an important source of this behavior. Across Qwen3, Llama, and Gemma, the two models can place their stopping probability on different EOS tokens, even when their declared stopping sets are identical. This mismatch can suppress the student’s preferred termination action without reliably transferring the teacher-preferred alternative. We show that aligning the decoding stopping set alone is insufficient, while treating functionally equivalent EOS tokens as a shared semantic stopping action substantially mitigates mismatch-induced length inflation across all three model families. To further understand how termination behavior evolves over training, we study OPD across different K2-Horizon training stages. This stage-wise analysis shows that termination preferences can shift substantially during training, while also revealing a distinct length inflation late in the OPD run that persists beyond termination alignment. Together, these results identify termination mismatch as an important, but not exhaustive, source of OPD length dynamics. We release an implementation incorporating the proposed termination-handling corrections.
4. 面向编码智能体的 Harness 设计实证研究
- 英文标题：An Empirical Study of Harness Design for Coding Agents
- arXiv：2609.20804v1
💡 一句话解读：这篇论文要解决的问题是：现有的编码智能体测试工具（harness）通常被当作一个整体来评估，导致我们不知道其中哪个部分真正有用。作者用一个轻量级框架，固定执行流程，只改变规划、动作空间和上下文管理三个组件，在多个模型和基准上测试了176种配置。结论是：上下文管理在预算紧张时最有用，先规则裁剪再摘要最划算，规划对弱模型提准确率、对强模型省成本，而只给bash接口对熟练模型更省钱。
编码测试框架（coding harness）决定了自主编码智能体如何将模型能力转化为长周期软件工程表现，但现有工作通常将测试框架作为整体系统来评估，导致单个组件的有效性不明确。为了实现组件级比较，我们用一个轻量级编码测试框架来研究这个问题，其执行循环固定，而三个组件可变：规划、动作空间和上下文管理。在四个模型上，基于 SWE-Bench Verified 和 Terminal-Bench 2.1，我们评估了176个匹配设置，涵盖五种上下文管理策略、四种上下文窗口预算，以及对规划和动作空间的针对性消融。我们发现：（1）随着上下文窗口预算收紧，上下文管理变得越来越有价值，其大部分收益来自防止上下文溢出失败。（2）在基于LLM的摘要之前先进行基于规则的省略，在上下文管理策略中提供了最强的整体效率，而让省略内容可恢复则增加了模型很少使用的机制，且没有带来准确率提升。（3）规划从弱模型的准确率支架转变为强模型的成本节省器，准确率变化不大。（4）预定义工具能提升bash能力较弱的模型的表现，而具备bash能力的模型可以仅用bash接口有效运行，并实现显著更低的成本，尤其是在以命令行为中心的任务上。轨迹级分析解释了这些效应：上下文管理延长了执行轨迹而不显著改变智能体行为，规划改变了轨迹停止的位置，动作空间改变了代码编写的粒度。这些发现为模型和预算感知的测试框架设计提供了信息，并为评估未来测试框架组件提供了一个模块化框架。
📄 English Abstract
Coding harnesses shape how autonomous coding agents translate model capabilities into long-horizon software-engineering performance, yet existing work typically evaluates harnesses as monolithic systems, leaving the effectiveness of individual components unclear. To enable component-level comparisons, we study this question with a lightweight coding harness whose execution loop is fixed while three components are varied: planning, action space, and context management. Across four models evaluated on SWE-Bench Verified and Terminal-Bench 2.1, we evaluate 176 matched settings spanning five context-management strategies, four context-window budgets, and targeted ablations of planning and action space. We find that: (1) Context management becomes increasingly valuable as the context-window budget tightens, with most of its benefit coming from preventing context-overflow failures. (2) Staging rule-based elision before LLM-based summarization provides the strongest overall efficiency among the context-management strategies, whereas making elided content recoverable adds machinery that models rarely use and yields no accuracy gain. (3) Planning shifts from an accuracy scaffold for weaker models to a cost saver for stronger models, with little change in accuracy. (4) Predefined tools improve performance for models with weaker bash proficiency, whereas bash-capable models can operate effectively with a bash-only interface and achieve substantially lower cost, especially on command-line-centric tasks. Trajectory-level analysis explains these effects: context management extends execution trajectories without substantially altering agent behavior, planning changes where trajectories stop, and the action space changes the granularity at which code is written. These findings inform model- and budget-aware harness design and provide a modular framework for evaluating future harness components.
5. JEPA-Anything：跨不同世界学习预测模型
- 英文标题：JEPA-Anything: Learning Predictive Models across Different Worlds
- arXiv：2609.20800v1
💡 一句话解读：当前预测模型通常只适用于单一领域，换个场景就不灵了。本文提出JEPA-Anything，通过把预测目标拆成互补的因子分别学习再组合，在视觉、生物、天气等七个领域都取得了更好效果，甚至能指导生物实验。
世界建模使智能体能够预测后果、指导干预并从交互中学习。然而，预测模型仍局限于特定领域：能否有一种通用的学习原则支持跨截然不同系统的世界建模？我们提出JEPA-Anything，一个基于正交预测因子分解（OPF）的领域无关框架。该框架扩展了联合嵌入预测架构（JEPA），将潜在目标分解为互补因子，通过专用路径学习这些因子，并在共享预测设计中重新组合。我们在七个领域评估JEPA-Anything：视觉、生物学、临床轨迹、控制、分子动力学、物理场和天气。实验涵盖表示学习、干预预测、分布外泛化和长时程动力学，包括10个匹配的动力学任务、超过1000个临床事件的预测以及四个系统中100步的分子推演。与匹配的JEPA基线相比，JEPA-Anything在所有10个动力学任务上提升了报告指标，并将Interventional Pong上的单次干预预测误差降低了34.8%。在所有四个系统中，它在比较方法中实现了一步和100步分子误差最低。除预测外，一个因子提名的生物干预在细胞共培养、患者来源类器官、肿瘤片段和小鼠中获得了实验支持；潜在轨道模式恢复了开普勒标度指数，拟合斜率为-1.4991。这些结果支持跨异构世界的共同因子化预测原则，将世界建模与干预及实验基础的科学发现联系起来。代码：https://github.com/Gen-Verse/JEPA-Anything
📄 English Abstract
World modeling enables intelligence to anticipate consequences, guide interventions, and learn from interaction. Yet predictive models remain domain-specific: can a common learning principle support world modeling across radically different systems? We introduce JEPA-Anything, a domain-agnostic framework based on orthogonal predictive factorization (OPF). Extending joint-embedding predictive architectures, OPF decomposes latent targets into complementary factors, learns them through dedicated pathways, and recombines them within a shared predictive design. We evaluate JEPA-Anything across seven domains: vision, biology, clinical trajectories, control, molecular dynamics, physical fields, and weather. Experiments span representation learning, intervention prediction, out-of-distribution generalization, and long-horizon dynamics, including 10 matched dynamics tasks, forecasting of over 1,000 clinical events, and 100-step molecular rollouts across four systems. Against matched JEPA baselines, JEPA-Anything improves reported metrics on all 10 dynamics tasks and reduces single-intervention prediction error on Interventional Pong by 34.8%. It achieves the lowest one-step and 100-step molecular errors among compared methods in all four systems. Beyond prediction, a factor-nominated biological intervention receives experimental support in cell co-cultures, patient-derived organoids, tumor fragments, and mice; latent orbital modes recover the Keplerian scaling exponent with a fitted slope of -1.4991. These results support a common factorized predictive principle across heterogeneous worlds, connecting world modeling with intervention and experimentally grounded scientific discovery. Code: https://github.com/Gen-Verse/JEPA-Anything
6. RiskChainBench：面向混淆平台消息还原与证据支撑网络调查的基准
- 英文标题：RiskChainBench: A Benchmark for Obfuscated Platform Message Restoration and Evidence-Grounded Web Investigation
- arXiv：2609.16900v2
💡 一句话解读：现有研究把“看懂暗语”和“判断网站风险”分开测试，看不出前者对后者的影响。本文提出一个基准，让模型先还原用表情、谐音等伪装的消息和网址，再像人一样打开网站调查并写出带证据的风险报告，从而把两步连起来评估。
平台滥用活动利用表情符号、谐音、字符拆分和冗余符号隐藏重定向指令，再通过伪装链接将用户引向与色情、欺诈、赌博或非法交易相关的服务。现有基准分别评估混淆文本和风险网页，掩盖了目标还原如何影响下游证据获取。我们提出 RiskChainBench，将来自 600 个源会话的 3,600 条合成 token-文本还原输入与 600 个对应的人工标注本地网页环境配对。模型首先还原消息、操作意图和目的地；随后同一底层模型作为视觉语言模型驱动的网页代理，调查正确关联的网站，并在不依赖消息侧语义或域名声誉线索的情况下生成冻结的、引用证据的风险报告。我们分别对还原和正确路由的网页调查进行评分，并通过将冻结的主入口预测作为门控应用于同一任务 2 结果，离线组合两者。人工标签决定任务正确性，而固定的多模态证据评判器评估忠实性、充分性、完整性和一致性。在十个模型中，入口 Top-1 准确率范围为 35.2% 至 95.2%，网页决策准确率范围为 26.3% 至 62.8%；领先系统在入口还原、完整重建、网站决策和细粒度分类上各有不同。执行失败占网页运行的 31.9%，而决策后类型错误仅占 0.9%，表明稳定的探索和风险判断是主要瓶颈。我们发布了基准、协议和可重置的本地沙盒。
📄 English Abstract
Platform abuse campaigns conceal redirection instructions with emojis, homophones, character decomposition, and redundant symbols, then route users through disguised links to services associated with pornography, fraud, gambling, or illicit transactions. Existing benchmarks evaluate obfuscated text and risky webpages separately, obscuring how target recovery affects downstream evidence acquisition. We introduce RiskChainBench, pairing 3,600 synthetic token-text restoration inputs from 600 source sessions with 600 corresponding human-labeled local web environments. A model first restores the message, operational intent, and destination; the same underlying model then acts as a VLM-driven web agent that investigates the correctly associated website and produces a frozen, evidence-cited risk report without message-side semantics or domain-reputation cues. We score restoration and correct-routing web investigation separately and compose them offline by applying the frozen primary-entry prediction as a gate to the same Task 2 result. Human labels determine task correctness, while a fixed multimodal evidence judge assesses faithfulness, sufficiency, completeness, and consistency. Across ten models, Entry Top-1 ranges from 35.2% to 95.2% and web decision accuracy from 26.3% to 62.8%; the leading systems differ across entry recovery, full reconstruction, website decisions, and fine-grained typing. Execution failures account for 31.9% of web runs, whereas post-decision type errors account for only 0.9%, identifying stable exploration and risk judgment as the principal bottlenecks. We release the benchmark, protocol, and resettable local sandbox.
7. RetireOPD：面向智能体强化学习的自退场在线策略蒸馏
- 英文标题：RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learning
- arXiv：2609.20784v1
💡 一句话解读：多轮智能体用强化学习训练时，每轮只得到一个总分奖励，学起来很慢。现有方法让一个拥有特权的“老师”模型提供细粒度指导，但老师不一定可靠，且指导效果分阶段。本文提出RetireOPD：先训练一个带技能的教师模型，再让学生模型同时用强化学习和蒸馏学习，当学生与教师差距不再缩小且达到教师成功率一定比例时，学生自动停止向教师学习，之后只靠强化学习。在多个模型和任务上，该方法显著提升了成功率。
使用强化学习（RL）训练的多轮智能体每条轨迹仅获得一个标量奖励，这促使自在线策略蒸馏（OPD）从具有特权任务技能的自教师模型提供密集的令牌级监督，让无技能的学生模型内化这些技能。然而，这一方案在智能体任务中被两个发现所削弱：仅凭特权信息并不总能使教师模型可靠，且教师监督的收益具有阶段依赖性。因此，我们提出RetireOPD（自退场在线策略蒸馏），它首先用环境奖励优化一个解耦的、以技能为条件的教师模型，然后联合使用RL和OPD训练一个无技能的学生模型。RetireOPD不遵循预定义的蒸馏计划，而是采用自适应退场：一旦学生与教师之间的差异不再缩小，且学生达到教师成功率的目标比例，学生便自行放弃教师，之后仅用RL继续训练。在从1.5B到7B的Qwen2.5模型上，RetireOPD将ALFWorld成功率较RL基线提升14.1%至18.8%，将WebShop准确率提升11.8%至19.0%，并在所有设置中超越了自身的技能条件教师模型。
📄 English Abstract
Multi-turn agents trained with reinforcement learning (RL) receive a single scalar reward per trajectory, which motivates self on-policy distillation (OPD) to supply dense token-level supervision from a self-teacher with privileged task skills, letting a skill-free student internalize them. This recipe, however, is undermined by two findings in agentic tasks: privileged information alone does not always make a teacher reliable, and the benefit of teacher supervision is stage-dependent. We therefore propose RetireOPD (Self-Retiring On-Policy Distillation), which first optimizes a decoupled, skill-conditioned teacher with environment rewards and then trains a skill-free student jointly with RL and OPD. Rather than following a predefined distillation schedule, RetireOPD adopts Adaptive Retirement: the student drops the teacher on its own once their discrepancy stops shrinking and it reaches a target fraction of the teacher’s success rate, after which training proceeds with RL alone. Across Qwen2.5 models from 1.5B to 7B, RetireOPD improves ALFWorld success rate over RL baseline by 14.1% to 18.8% and WebShop accuracy by 11.8% to 19.0%, and surpasses its own skill-conditioned teacher in every setting.
8. WeVisDoc：从覆盖到能力，实现鲁棒的端到端文档解析
- 英文标题：WeVisDoc: From Coverage to Capability for Robust End-to-End Document Parsing
- arXiv：2609.20423v1
💡 一句话解读：文档解析模型常因训练数据偏向常见、干净的文档而在复杂版式上表现不佳，且单纯增加数据种类并不能精准修补弱点。为此，作者提出 WeVisDoc 两阶段框架：先用多样数据和模拟退化扩大覆盖面，再用探针测试找出剩余错误，针对性补充数据并调整训练重点，从而显著提升解析鲁棒性。
文档解析将文档图像转换为结构化内容，需要在多样版式和采集条件下保持可靠性能。然而，训练语料偏向常见文档类型和干净的数字页面，而仅扩大覆盖范围并不能说明如何解决解析器剩余的弱点。我们提出 WeVisDoc，一个面向鲁棒端到端文档解析的两阶段、以数据为中心的框架。第一阶段通过异构数据和保持结构的退化合成，拓宽语义、结构和外观覆盖范围。第二阶段使用留出探针，在固定的视觉-结构聚类内测量第一阶段解析器的残余错误。这些诊断结果指导有针对性的数据构建和目标 token 预算的重新分配。WeVisDoc-4B 在 OmniDocBench v1.6 上取得 95.38 的总体得分，在 PureDocBench 三个赛道上的平均总体得分为 75.54，在所有四个设置中均位列所比较的端到端解析器之首。与第一阶段相比，第二阶段在两个基准上均提升了 2B 和 4B 模型的总体得分，在退化的 PureDocBench 赛道上提升更大，其中 4B 模型在真实退化赛道上提升了 4.03 分。
📄 English Abstract
Document parsing converts document images into structured content and requires reliable performance across diverse layouts and acquisition conditions. Yet training corpora are biased toward common document types and clean digital pages, while expanding coverage alone does not specify how to address a parser’s remaining weaknesses. We present WeVisDoc, a two-stage data-centric framework for robust end-to-end document parsing. Stage I broadens semantic, structural, and appearance coverage through heterogeneous data and structure-preserving degradation synthesis. Stage II uses a held-out probe to measure the Stage I parser’s residual errors within fixed visual-structural clusters. These diagnostics guide targeted data construction and reallocation of the target-token budget. WeVisDoc-4B achieves an Overall score of 95.38 on OmniDocBench v1.6 and a mean Overall score of 75.54 across the three PureDocBench tracks, ranking first among the compared end-to-end parsers in all four settings. Compared with Stage I, Stage II improves Overall scores for the 2B and 4B models on both benchmarks, with larger gains on the degraded PureDocBench tracks, including a 4.03-point gain for the 4B model on the Real Degraded track.
9. 反思、修订、复用：面向GUI智能体的免训练技能演化
- 英文标题：Reflect, Revise, Reuse: Training-Free Skill Evolution for GUI Agents
- arXiv：2609.17653v1
💡 一句话解读：这篇研究要解决的是：图形界面智能体在执行长任务时，常因弹窗、加载延迟、控件移位而失败，而现有技能库是部署前写死的静态内容，无法适应这些变化。作者提出免训练框架 EvoSkill-GUI，把技能做成可编辑的多文件包，通过“反思—修订—复用”循环，让智能体在部署后根据执行反馈自行修改技能，从而越用越好。
GUI 智能体需要在动态图形界面上执行长时程任务，而弹窗、延迟加载和控件移位常常使执行前固定的计划失效。近期的智能体技能框架通过封装可复用的程序性知识来缓解这一问题，但现有技能设计大多没有针对 GUI 执行动态，且把技能视为部署前产生的静态产物，而非能在部署中不断改进的活的知识。我们认为，GUI 智能体需要的不是更好的静态技能，而是能在部署时根据执行反馈修订、且无需额外训练的技能。我们提出 EvoSkill-GUI，一个免训练框架，其中每个技能是一个结构化的多文件包，包含检索元数据、可执行计划、备用定位、失败恢复规则、无障碍工具和失败案例。EvoSkill-GUI 通过“反思—修订—复用”循环运行：执行器在 rollout 中进行即时修订，隔离的评论器在严格信息隔离下诊断失败轨迹，执行器则通过受限工具接口编辑特定技能文件。在 MobileWorld、AndroidWorld 和 OSWorld 这三个涵盖移动与桌面平台的主流 GUI 基准上，EvoSkill-GUI 无需任何训练即可持续提升多个基础模型，最大增益分别为 +16.2%、+6.0% 和 +10.5%，且演化后的技能库能继续惠及相关任务，而非被从头重建。代码见 https://github.com/ZJU-REAL/EvoSkill-GUI。
📄 English Abstract
GUI agents execute long-horizon tasks on dynamic graphical user interfaces, where pop-ups, delayed loads, and relocated widgets routinely invalidate plans fixed before execution. Recent agent-skill frameworks encapsulate reusable procedural knowledge to mitigate this, yet existing skill designs are largely developed without targeting GUI execution dynamics and treat skills as static artifacts produced before deployment rather than living procedural knowledge that improves through it. We argue that what GUI agents need is not better static skills, but skills that can be revised from execution feedback at deployment time, without additional training. We propose \textbf{EvoSkill-GUI}, a training-free framework in which each skill is a structured multi-file package containing retrieval metadata, executable plans, backup localization, failure-recovery rules, accessibility utilities, and failure cases. EvoSkill-GUI operates through a \textbf{\emph{reflect-revise-reuse}} loop: the executor performs instant in-rollout revisions, an isolated critic diagnoses failed trajectories under strict information isolation, and the executor edits specific skill files through a restricted tool interface. Across MobileWorld, AndroidWorld, and OSWorld, three mainstream GUI benchmarks spanning mobile and desktop platforms, EvoSkill-GUI consistently improves multiple base models without any training, with maximum gains of $+16.2%$, $+6.0%$, and $+10.5%$ respectively, and evolved skill libraries continue to benefit related tasks rather than being rebuilt from scratch. Our code is available at https://github.com/ZJU-REAL/EvoSkill-GUI.
10. VABench：通过视觉演示、主动感知和度量控制测量具身空间智能
- 英文标题：VABench: Measuring Embodied Spatial Intelligence through Visual Demonstrations, Active Perception, and Metric Control
- arXiv：2609.19554v1
💡 一句话解读：现有AI能说出物体在哪，却无法在看不全时主动找线索并动手操作。本研究搭建VA-Bench测试平台，让通用多模态大模型仅凭视觉演示，自己选视角、发指令、根据反馈修正，完成抓取等任务。结果显示最强模型任务成功率仅约54%，说明具身空间智能仍远未解决。
空间智能不只是描述物体位置。在观测不完整的情况下，模型必须识别并获取缺失的证据，在统一的空间坐标系中加以解读，并据此行动。我们提出VA-Bench，用于评估完整的“观察—推理—行动—修正”闭环。通用多模态大语言模型（MLLMs）仅从RGB演示中学习程序性上下文，主动选择相机视角，发出度量化的笛卡尔指令，并根据执行反馈进行修正。模型不获得任何特权物体位姿、专家轨迹或学习到的动作头。一个固定的、与模型无关的控制器只执行模型指定的目标。VA-Bench包含14个基础任务族（11个单臂和3个双臂）、7个留出的几何/布局变体，以及一条长时程五物体组合赛道。我们在每个基础任务相同的20个经物理验证的随机种子上，对12种主要模型条件进行三次独立运行评估，报告最终成功率、九项轨迹级行为诊断指标和子任务进度。第一，表现最好的模型在标注运行中目标定位得分100.0%，空间关系得分78.9%，但其三次运行的宏平均任务成功率仅为53.93±3.17%。第二，主动相机控制相比被动多视角观察显著提升任务成功率；在一项匹配对比中，成功率从27.86%升至57.50%。第三，留出的几何迁移可使任务成功率下降超过30个百分点。尽管有大量部分进展，没有任何模型能完成严格的长时间程回合。因此，VA-Bench检验的是通用MLLMs能否将视觉演示和主动获取的证据转化为成功的具身行动。
📄 English Abstract
Spatial intelligence requires more than describing object locations. Under incomplete observation, models must identify and acquire missing evidence, interpret it in a common spatial frame, and act on it. We introduce VA-Bench to evaluate the complete observe-reason-act-revise loop. General-purpose MLLMs learn procedural context from RGB-only demonstrations, actively select camera viewpoints, issue metric Cartesian commands, and revise them from execution feedback. Models receive no privileged object poses, oracle trajectories, or learned action heads. A fixed model-agnostic controller executes only model-specified targets. VA-Bench contains 14 base task families (11 single-arm and three dual-arm), seven held-out geometry/layout variants, and a long-horizon five-object composition track. We evaluate 12 primary model conditions in three independent runs over the same 20 physically verified seeds per base task, reporting terminal success, nine trajectory-level behavioral diagnostics, and subtask progress. First, the best-performing model scores 100.0% on target localization and 78.9% on spatial relations in the annotated run. Its three-run macro-average task success is only 53.93+/-3.17%. Second, active camera control significantly improves task success over passive multi-view observation. In one matched comparison, success rises from 27.86% to 57.50%. Third, held-out geometric transfer can reduce task success by over 30 percentage points. No model completes a strict long-horizon episode, despite substantial partial progress. VA-Bench thus tests whether general-purpose MLLMs can turn visual demonstrations and actively acquired evidence into successful embodied action.
11. Video DeltaNet：面向直播视频生成的视频原生混合注意力
- 英文标题：Video DeltaNet: A Video-Native Hybrid Attention for Livestream Video Generation
- arXiv：2609.20744v1
💡 一句话解读：视频生成模型在去噪时要反复处理超长的时空数据，注意力计算成了最大瓶颈。本文提出 Video DeltaNet，把局部精细注意力与双向线性记忆结合，并设计逐帧更新的视频 Delta 注意力与分阶段对齐训练，在保持画质的同时大幅提速。
视频扩散模型在去噪过程中反复处理长时空 token 序列，使注意力成为主要计算瓶颈。线性注意力是一种颇具吸引力的替代方案，并已被近期大语言模型广泛采用，但直接将其用于视频模型往往无法保留高质量生成所需的细粒度交互。我们提出 Video DeltaNet（VDN），将局部 Softmax 注意力与用于长程视频上下文的双向线性记忆相结合。其线性分支引入 Video Delta Attention（VDA），通过联合纳入每帧的空间 token，实现每帧一次的记忆更新。独立的输出投影与可学习门控用于校准两个分支，而分阶段的教师对齐方案则将这一新路径逐步引入预训练模型。我们在 MiniMax H3 上实例化 VDN，将混合注意力用于视频到视频的交互，同时对涉及文本或音频的交互保留 Softmax。配合八步蒸馏与优化的 SGLang 服务栈，VDN-H3 在八块 NVIDIA B200 GPU 上于 6.70 秒内完成一段 14.3 秒、768p 视频的 DiT 去噪，相比相同 GPU 数量下的 50 步稠密 H3 基线实现了 14.5 倍加速。
📄 English Abstract
Video diffusion models repeatedly process long spatiotemporal token sequences during denoising, making attention a major computational bottleneck. Linear attention offers an appealing alternative and has been widely adopted in recent large language models, but directly applying it to video models often fails to preserve the fine-grained interactions required for high-quality generation. We present Video DeltaNet (VDN), which combines local Softmax attention with bidirectional linear memory for long-range video context. Its linear branch introduces Video Delta Attention (VDA), which updates memory once per frame by jointly incorporating its spatial tokens. Separate output projections and learnable gates calibrate the two branches, while a staged teacher-alignment recipe progressively introduces the new pathway into pretrained models. We instantiate VDN on MiniMax H3, applying the hybrid to video-to-video interactions while retaining Softmax for interactions involving text or audio. With eight-step distillation and an optimized SGLang serving stack, VDN-H3 completes DiT denoising for a 14.3-second, 768p video in 6.70 seconds on eight NVIDIA B200 GPUs, corresponding to a 14.5x speedup over the 50-step dense H3 baseline on the same GPU count.
12. FAMOS：基于稀疏观测的前馈式三维铰接建模
- 英文标题：FAMOS: Feed-Forward 3D Articulation Modeling from Sparse Observations
- arXiv：2609.20817v1
💡 一句话解读：从少量、零散的单目视角重建会动的物体（比如带抽屉的柜子）很难，因为每个视角只能看到局部形状和运动。FAMOS 用前馈模型同时看多份不完整点云，直接预测哪些部件能动、关节怎么转，还配了数据生成器补足训练数据。
从稀疏单目视角对铰接物体（articulated objects）建模具有挑战性，因为每个观测只能揭示部分几何与运动证据。大多数前馈方法仅从单一观测推断铰接关系，因而严重依赖学习到的类别级形状先验。我们提出 FAMOS，一种前馈模型，可从稀疏、无序的部分点云集合中预测可动部件分割与关节参数。该模型对多个观测进行联合推理，并天然支持可变数量的输入（包括单一视角）。为跨观测聚合铰接线索，我们引入多状态铰接 Transformer（Multi-state Articulation Transformer），交替使用状态级与全局注意力。我们进一步提出观测铰接跨度（observed articulation span）目标，用于监督每个部件在输入观测中所呈现的运动范围，促使模型充分利用整个观测集合。为克服现有数据集规模与多样性不足的问题，我们引入一个程序化数据生成器，在训练过程中合成自带标注的资产。在 PartNet-Mobility、ACD 与 ArtiCraft-10K 上的实验表明，该方法相较前馈与基于优化的基线均取得了一致提升。
📄 English Abstract
Modeling articulated objects from sparse monocular views is challenging because each observation reveals only partial geometry and motion evidence. Most feed-forward methods infer articulation from a single observation and therefore rely heavily on learned category-level shape priors. We present FAMOS, a feed-forward model that predicts movable-part segmentation and joint parameters from a sparse, unordered set of partial point clouds. Our model jointly reasons over multiple observations and naturally supports a variable number of inputs, including a single view. To aggregate articulation cues across observations, we introduce a Multi-state Articulation Transformer with alternating state-wise and global attention. We further propose an observed articulation span objective that supervises the motion range each part exhibits across the input observations, encouraging the model to leverage the full observation set. To overcome the limited scale and diversity of existing datasets, we introduce a procedural data generator that synthesizes self-annotated assets during training. Experiments on PartNet-Mobility, ACD, and ArtiCraft-10K demonstrate consistent improvements over both feed-forward and optimization-based baselines. Project page: https://kevinqu7.github.io/famos
13. UFO：面向多模态图像生成全条件对齐的评估链
- 英文标题：UFO: Chain-of-Evaluation for Omni-Condition Alignment in Multi-Modal Image Generation
- arXiv：2609.12397v2
💡 一句话解读：多模态图像生成（比如按文字和参考图定制图片）发展很快，但评价生成结果好不好还跟不上。现有方法把文字、图像等条件分开打分，忽略了它们要同时满足，所以和人的判断不一致。本文提出UFO，把“所有条件是否同时对齐”拆成一条细粒度、相互独立的评估链，再按类型分别验证，从而更接近人的评价。
多模态图像生成，尤其是主体驱动的定制化，近年来受到越来越多关注。尽管生成模型发展迅速，其评估方法却明显滞后。现有方法无论是基于嵌入（embedding）的还是基于多模态大语言模型（MLLM）的，都孤立地评估与每个模态条件的对齐，这与多模态图像生成需要同时满足多个条件对齐的目标相矛盾，导致与人类判断的一致性较差。为解决这一挑战，我们提出UFO，这是首个用于全条件对齐同时评估的统一框架。具体而言，UFO引入了一种新颖的原子化评估链（Atomized Chain-of-Evaluation）范式，即首先将全条件对齐分解为一系列细粒度、解耦的原子评估单元（Atomic Evaluation Unit, AEU），将其归入不同的模态相关性类别，然后针对不同AEU类型采用通用或专用的功能调用进行准确验证。实验结果表明，UFO与人类评估偏好达到了最高相关性，平均提升15.25%。此外，我们还提出了UFO-Bench，这是一个专门设计的基准，用于在文本与视觉条件多样交互下全面评估现有定制化模型的性能。
📄 English Abstract
Multi-modal image generation, particularly subject-driven customization, has garnered growing attention in recent years. Despite the rapid advancement of generative models, their evaluation remains largely lagging. Existing methods, whether embedding-based or Multi-modal Large Language Model (MLLM)-based, evaluate alignment with each modal condition in isolation, which contradicts the simultaneous condition alignment objective of multi-modal image generation, leading to poor consistency with human judgments. To address this challenge, we propose UFO, the first unified framework for omni-condition alignment simultaneous evaluation. Specifically, UFO introduces a novel Atomized Chain-of-Evaluation paradigm, i.e., it first decomposes omni-condition alignment into a sequential chain of fine-grained, disentangled Atomic Evaluation Units (AEUs), categorizes them into distinct modality-relevance classes, and then employs general or dedicated functional calls for accurate verification of different AEU types. Experimental results demonstrate that UFO achieves the highest correlation with human evaluation preferences, delivering an average improvement of 15.25%. Furthermore, we present UFO-Bench, a dedicated benchmark designed to holistically evaluate the performance of existing customization models under the diverse mutual interactions of textual and visual conditions.
14. MiniMax-H3 能否推理物理世界？一项全模态生成模型评估
- 英文标题：Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Model
- arXiv：2609.18323v1
💡 一句话解读：这篇研究想弄清一个问题：能同时处理文字、图像、视频和声音的全能生成模型，是否真的理解了物理世界？作者设计了一套测试，故意让每种输入只提供部分线索（比如只给声音和图片），看模型能否把不同线索拼起来推断出发生了什么、接下来会怎样。结果模型整体成功率约42%，说明多模态信息融合仍是难点。
近期的全模态生成模型（Omni-Modal Generative Models，简称 Omni-Models）推动了内容生成向文本、图像、视频与音频统一建模的方向发展。MiniMax-H3 是这一转变的代表，它在共享潜在框架中结合了多模态上下文理解与音视频联合生成。其统一架构引出一个根本性问题：多模态对齐能否提升模型的世界推理能力，全模态输入又能带来哪些新的评估范式？为此，本工作提出了一个围绕物理世界推理四个互补维度组织的综合评估框架。与现有视频生成和世界模型评估框架不同——后者常受限于输入模态有限、且提示与目标视频内容高度匹配——我们的评估专门设计以利用全模态模型的多模态输入。我们构建了一组多样化的新任务，要求模型整合跨模态的互补信息。具体而言，我们考虑了四种场景：隐式提示搭配多帧、音频-图像、前缀视频以及音频-视频输入。每种单一模态仅提供关于潜在事件的部分证据，要求模型联合推理互补的语义线索，以推断潜在事件状态与未来动态。在 517 个评估实例中，MiniMax-H3 的总体成功率为 41.97%。基于视频的决策推理成功率最高，为 56.00%，而基于音频的消歧推理最弱，仅为 27.40%。这些结果表明，有效的多模态整合仍是充分发挥多样输入模态优势的关键。项目地址：https://github.com/gulucaptain/MiniMax-H3-Reason。
📄 English Abstract
Recent Omni-Modal Generative Models (Omni-Models) have advanced content generation toward unified modeling of text, images, video, and audio. MiniMax-H3 exemplifies this transition by combining multimodal context understanding with joint audio-visual generation in a shared latent framework. Its unified architecture raises a fundamental question: Can multimodal alignment improve the model’s world reasoning, and what new evaluation paradigms do omni-modal inputs enable? To investigate this question, this work introduces a comprehensive evaluation framework organized around four complementary dimensions of physical world reasoning. Unlike existing evaluation frameworks for video generation and world models, which are often constrained by limited input modalities and evaluation settings where prompts closely match the target video content, our evaluation is specifically designed to exploit the multimodal inputs of Omni-Model. We construct a diverse set of novel tasks that require models to integrate complementary information across modalities. Specifically, we consider four scenarios, including implicit prompts paired with multiple frames, audio-image, prefix-videos, and audio-video inputs. Every single modality provides only partial evidence about the underlying event, requiring the model to jointly reason over the complementary semantic cues to infer latent event states and future dynamics. Across 517 evaluation instances, MiniMax-H3 achieves an overall success rate of 41.97%. Video-based Decision Reasoning yields the highest success rate at 56.00%, while Audio-based Disambiguation Reasoning is the weakest, reaching only 27.40%. These results indicate that effective multimodal integration remains key to fully exploiting the benefits of diverse input modalities. The project is available at https://github.com/gulucaptain/MiniMax-H3-Reason.
15. When2Think：面向高效混合推理模型的难度感知长度控制学习
- 英文标题：When2Think: Learning Difficulty-Aware Length Control for Efficient Hybrid Reasoning Models
- arXiv：2609.19671v1
💡 一句话解读：大推理模型常犯“简单题想太多、难题想太少”的毛病，浪费算力还掉准确率。这篇论文提出 When2Think，让模型根据题目难度自动决定思考深度：简单题直接答，难题才展开推理，从而在省算力的同时提升准确率。
大推理模型（LRMs）在复杂任务上表现优异，却存在系统性低效：常对简单问题过度思考，对困难问题思考不足。现有基于统一长度惩罚或刚性路由的方法会带来效率税，即以牺牲难题准确率换取简单实例上的计算缩减。我们将高效推理建模为实例自适应计算分配问题，并提出 When2Think——一个面向混合推理的后训练框架，根据问题难度动态分配计算量。该方法引入实例级难度感知控制（IDAC），一种奖励塑形机制，利用预计算的参考统计量（准确率与 token 用量）来调节推理深度。结合基于验证器的奖励与批次标准化优势，IDAC 无需学习奖励模型或在线参考模型查询，即可实现稳定的无评论家优化。When2Think 鼓励对简单实例直接作答，同时在困难实例上保留扩展推理，从而学会何时使用系统 1（NoThink）与系统 2（Think）。数学基准实验表明其改善了准确率-效率权衡：在 AIME24 上，相比基础模型，Pass@3 提升 10.0% 且 token 用量减少 27.9%；在 AIME25 上，When2Think 达到 40.0% Pass@3，优于仅压缩和仅路由的基线方法。
📄 English Abstract
Large Reasoning Models (LRMs) achieve strong performance on complex tasks but exhibit systematic inefficiency: they often overthink easy problems and underthink hard ones. Existing approaches based on uniform length penalties or rigid routing incur an efficiency tax, trading reduced computation on easy instances for accuracy loss on hard instances. We formulate efficient reasoning as an instance-adaptive computation allocation problem and propose When2Think, a post-training framework for hybrid reasoning that dynamically allocates computation based on problem difficulty. Our method introduces Instance-level Difficulty-Aware Control (IDAC), a reward-shaping mechanism that leverages pre-computed reference statistics (accuracy and token usage) to regulate reasoning depth. Combined with verifier-based rewards and batch-wise standardized advantages, IDAC enables stable critic-free optimization without learned reward models or online reference-model queries. When2Think encourages direct answering on easy instances while preserving extended reasoning on hard instances, thereby learning when to use System 1 (NoThink) versus System 2 (Think). Experiments on mathematical benchmarks demonstrate improved accuracy-efficiency trade-offs: on AIME24, Pass@3 increases by 10.0% while token usage is reduced by 27.9% relative to the base model, and on AIME25, When2Think achieves 40.0% Pass@3, outperforming compression and routing-only baselines.