Daily Papers · 2026-09-22
📄 论文日报 | 来源：Hugging Face Daily Papers
🗓 Episode：2026.09.21 | 代码合成有据技能；源码扩展编程RL环境
1. 面向智能体智能的从大规模代码中合成有依据技能
- 英文标题：Grounded Skill Synthesis from Code at Scale for Agentic Intelligence
- arXiv：2609.05571
💡 一句话解读：现有方法让AI智能体学技能要么依赖与环境互动，要么从文档提取但缺乏可执行验证。本研究提出Code2Skill，自动从海量GitHub代码中提取原子操作、工作流和模式，并通过重建验证，构建了百万级技能库，让智能体无需先验经验就能获得可迁移技能。
可复用技能为智能体（agent）提供可迁移的过程性知识，因此可扩展的技能获取对于让智能体超越先前经验至关重要。现有方法面临两个局限：基于轨迹（trajectory）的合成需要与特定环境交互，而从文档中提取的技能可能缺乏可执行证据和验证。源代码提供了一条互补路径：它不需要智能体具备先验经验，却能提供可执行证据来为抽象概念提供依据。我们提出Code2Skill，这是一个全自动流水线，将选定的代码单元转化为锚定实现细节的原子操作、复合工作流和重复模式记录，然后通过源主体盲重建（source-body-blind reconstruction）和源感知比较（source-aware comparison）来验证每条记录。将其应用于19,769个流行且活跃维护的GitHub仓库，Code2Skill生成了CodeSkillBank，这是一个包含1,006,822条被接受记录的有依据技能库，带有工作流、边界、来源和源证据元数据。在覆盖九种模型设置和八个基准的72项协议匹配评估中，使用检索到的CodeSkillBank技能增强的模型平均比匹配基线提升11.7%，并在57个案例中优于基线。在统一的下游接口下，Code2Skill在所有七个共享基准上也优于基于轨迹衍生的技能库，表明从仓库衍生的技能可以在智能体积累足够交互经验之前提供有用的过程性知识。从经过测试的AI生成代码中合成的技能通过率达到93.50%，而人类编写代码为93.00%，这初步表明该流水线可以随着AI生成软件量的增长而扩展。总体而言，Code2Skill将嵌入在代码仓库中的过程性知识转化为有依据、可验证且可迁移的智能体技能。
📄 English Abstract
Reusable skills give agents transferable procedural knowledge, making scalable acquisition essential for extending agents beyond prior experience. Existing methods face two limitations: trajectory-based synthesis requires interactions with specific environments, while document-derived skills may lack executable evidence and verification. Source code offers a complementary path: it requires no prior agent experience yet provides executable evidence for grounding abstractions. We present Code2Skill, a fully automated pipeline that transforms selected code units into implementation-anchored records of atomic operations, composite workflows, and recurring patterns, then verifies each record through source-body-blind reconstruction and source-aware comparison. Applied to 19,769 popular, actively maintained GitHub repositories, Code2Skill produces CodeSkillBank, a grounded bank of 1,006,822 accepted records with workflow, boundary, provenance, and source-evidence metadata. Across 72 protocol-matched evaluations covering nine model settings and eight benchmarks, models augmented with retrieved CodeSkillBank skills improve by 11.7% on average over matched baselines and outperform them in 57 cases. Under a unified downstream interface, Code2Skill also outperforms trajectory-derived skill banks on all seven shared benchmarks, showing that repository-derived skills can provide useful procedural knowledge before agents accumulate sufficient interaction experience. Skills synthesized from tested AI-generated code achieve a 93.50% pass rate, compared with 93.00% for human-written code, providing initial evidence that the pipeline can expand with the growing volume of AI-generated software. Overall, Code2Skill transforms procedural knowledge embedded in repositories into grounded, verifiable, and transferable agent skills.
2. CodeMidas：从代码本身扩展智能体编程强化学习环境
- 英文标题：CodeMidas: Scaling Agentic Coding RL Environments from Code Itself
- arXiv：2609.22068
💡 一句话解读：训练会写代码的AI需要大量带标准答案的练习题，但现有方法只能从代码仓库的讨论和提交记录里挖题，来源有限。本研究提出CodeMidas：只用源代码本身，让AI自动探索代码功能、出题、写测试并筛选，造出五千多道练习题，训练后AI在多个编程测试上表现明显提升。
通过强化学习（RL）训练有能力的编程智能体，需要多样化且配有可靠验证器的任务。开源代码库为此类任务提供了丰富来源，但现有方法通常依赖 issue、commit 等开发过程产物，限制了可提取任务的范围。为更好地扩展 RL 环境，我们提出 CodeMidas——一个智能体流水线，仅以源代码作为任务特定输入，将现有代码库中已实现的功能转化为可执行的 RL 环境。CodeMidas 在环境构建的每个阶段都投入智能体算力：智能体探索已实现功能以形成行为规范，基于原始代码的执行构建测试，并通过执行检查和反复求解采样来验证和筛选候选任务。最终数据集包含来自 3,185 个开源代码库、覆盖 23 种编程语言和 15 个技术领域的 5,545 个训练任务。用 GRPO 在这些任务上训练 MiMo-V2.5，在五个不同基准上均取得提升，涵盖问题修复（DeepSWE +11.7%）、整程序构建（ProgramBench +17%）和终端操作（Terminal-Bench v2.1 +8.5%）。消融实验表明，增加高质量训练任务数量可提升性能。轨迹分析显示，经 RL 训练的智能体表现出更好的行为，如增加代码库探索和更多样的自我验证。这些结果确立了源代码作为构建 RL 环境的可扩展基础，能够提升编程智能体在各类软件任务上的表现。
📄 English Abstract
Training capable coding agents via reinforcement learning (RL) requires diverse tasks with reliable verifiers. Open-source codebases offer a rich source of such tasks, while existing methods typically rely on development artifacts such as issues and commits, limiting the range of tasks that can be extracted. To better scale RL environments, we present CodeMidas, an agentic pipeline that turns implemented functionality in existing codebases into executable RL environments using source code as its only task-specific input. CodeMidas allocates agentic compute to every stage of environment construction: agents explore implemented functionality to formulate behavioral specifications, construct tests grounded in execution of the original code, and validate and filter candidate tasks through execution checks and repeated solution rollouts. The resulting dataset has 5,545 training tasks from 3,185 open-source codebases spanning 23 programming languages and 15 technical domains. Training MiMo-V2.5 on these tasks with GRPO improves performance on all five diverse benchmarks, covering issue repair (DeepSWE + 11.7%), whole-program construction (ProgramBench +17%), and terminal work (Terminal-Bench v2.1 +8.5%). Ablations show that increasing the number of high-quality training tasks improves performance. Trajectory analysis shows the RL-trained agent demonstrates better behaviors like increasing codebase exploration and more diverse self-verification. These results establish source code as a scalable foundation for constructing RL environments that improve coding agents across diverse software tasks.
3. EvoOntology：面向数据智能体的自进化本体层
- 英文标题：EvoOntology: A Self-Evolving Ontology Layer for Data Agents
- arXiv：2609.15779
💡 一句话解读：数据智能体处理表格、文件、数据库等杂乱数据时，常因看不懂数据含义而犯错。现有做法要么让智能体自己瞎摸索，要么靠人工写说明。本文提出 EvoOntology，把数据含义做成一个可随时查询的“活字典”，并让智能体自动搭建、持续自我修正，从而更准确地使用各种数据。
数据智能体（data agents）旨在对表格、文件和数据库等异构数据执行自然语言指令。然而，数据智能体面临一个棘手的“智能体—数据鸿沟”：异构数据位于智能体之外，而智能体只能通过通用工具访问它（例如列名和文件路径）。现有方法要么让智能体直接探索原始数据源，要么将人工构建的语义层注入提示词中，但两者都难以扩展到大规模异构数据源，也无法适应不同的智能体行为。本文提出 EvoOntology，一种面向数据智能体的自进化本体层（self-evolving ontology layer）。EvoOntology 将本体封装为一个 MCP 服务器，包含模式层、内容层和工具层，使智能体能够在运行时主动查询并与本体交互。为此，我们引入一个构建器智能体（builder agent）用于自主构建本体，以及一个自进化循环（self-evolution loop），通过归因引导的类型化编辑持续精炼本体，且这些编辑只有在经过骨干条件配对评估（backbone-conditional paired evaluation）后才被接受。在三个广泛采用的数据智能体基准和四个 LLM 骨干上的实验表明，EvoOntology 持续优于强基线及现有语义层方法，有效弥合了智能体—数据鸿沟，并实现了与异构数据更有效的交互。代码：https://github.com/ruc-datalab/EvoOntology
📄 English Abstract
Data agents aim to fulfill natural-language instructions over heterogeneous data, including tables, files, and databases. However, data agents face a challenging agent-data gap: heterogeneous data resides outside the agent, while the agent can access it (e.g., column names and file paths) only through generic tools. Existing approaches either let agents directly explore raw data sources or inject manually constructed semantic layers into prompts. However, neither scales well to large heterogeneous data sources nor adapts to different agent behaviors. In this paper, we introduce EvoOntology, a self-evolving ontology layer for data agents. EvoOntology encapsulates the ontology as an MCP server comprising a schema layer, a content layer, and a tool layer, enabling agents to actively query and interact with the ontology at runtime. To this end, we introduce a builder agent for autonomous ontology construction and a self-evolution loop that continuously refines the ontology through attribution-guided typed edits that are accepted only after a backbone-conditional paired evaluation. Experiments on three well-adopted data-agent benchmarks with four LLM backbones demonstrate that EvoOntology consistently outperforms strong baselines and existing semantic-layer approaches, effectively bridging the agent-data gap and enabling more effective interaction with heterogeneous data. Code: https://github.com/ruc-datalab/EvoOntology
4. RecreationWorld：面向混合计算机使用智能体的可扩展且可验证环境
- 英文标题：RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents
- arXiv：2609.22000
💡 一句话解读：现有电脑操作智能体要么只会点界面，要么只会写代码，而真实工作往往需要两者交替进行。作者提出 RecreationWorld：给智能体一个正在运行的参考软件，让它自己摸索行为、写代码复刻出来，并在五大平台上提供可复现环境和自动打分，还发布了 250 项任务的评测集 RecreationBench。
计算机使用智能体（CUA）此前沿两条独立路线发展：图形交互，以及通过代码和命令行进行软件开发。真实的数字工作却需要两者交替进行，而非端到端地简单叠加。我们研究混合型 CUA，它们能自主决定何时探索界面、实现软件，并运行和视觉验证其产物。我们提出 RecreationWorld，一个围绕“复刻”构建的五平台框架：给定一个正在运行的参考程序，智能体必须发现其行为并在没有规定工作流的情况下构建出忠实实现。RecreationWorld 在 Ubuntu、macOS、Windows、Android 和 Web 上提供可复现环境，并配有统一测试装置，支持原生 GUI 控制与编码工具。运行中的参考程序充当隐藏行为测试的判定基准（oracle），提供基于执行的奖励。我们利用高质量开源应用扩展轨迹生成规模。在这些轨迹上训练的模型在五个分布外的编码与混合计算机使用基准上均有提升，并更频繁地验证其渲染输出，表明能力可迁移到复刻任务之外。为进行留出评估，我们提出 RecreationBench，包含跨领域、跨平台的 250 个多样化任务。基于参考程序的程序化与视觉断言覆盖多个交互深度上的动作条件结果；每条断言都在参考程序上验证并经人工审核，随后测试套件冻结用于自动评分。GPT-6 Astra 以 58.1% 的总成绩领先，但仅在 2.8% 的任务上通过全部程序化测试。智能体复现静态界面结构比复现交互和计算结果更可靠，而生成的应用程序相比参考程序仍更小、更单一。我们发布了该基准、环境与测试套件。
📄 English Abstract
Computer-use agents (CUAs) have advanced along two separate lines: graphical interaction and software development through code and the command line. Real digital work requires both, interleaved rather than stacked end to end. We study hybrid CUAs that autonomously decide when to explore an interface, implement software, and run and visually verify their artifacts. We introduce RecreationWorld, a five-platform framework built around recreation: given a running reference, an agent must discover its behavior and build a faithful implementation with no prescribed workflow. RecreationWorld provides reproducible environments on Ubuntu, macOS, Windows, Android, and Web, plus a unified harness with native GUI control and coding tools. The running reference serves as an oracle for hidden behavioral tests, providing execution-grounded rewards. We scale trajectory generation with high-quality open-source applications. Models trained on these trajectories improve across five out-of-distribution coding and hybrid computer-use benchmarks and more frequently verify their rendered outputs, providing evidence of transfer beyond recreation. For held-out evaluation, we introduce RecreationBench, comprising 250 diverse tasks across domains and platforms. Reference-grounded programmatic and visual assertions cover action-conditioned outcomes at multiple interaction depths; each is validated on the reference and by human reviewers before the suite is frozen for automatic scoring. GPT-6 Astra leads at 58.1% overall, but passes all programmatic tests on just 2.8% of tasks. Agents reproduce static interface structure more reliably than interactions and computed outputs, while generated applications remain smaller and more monolithic than their references. We release the benchmark, environments, and test suites.
5. IntBMoE：将块级条件融入专家组合以实现全参与混合专家
- 英文标题：IntBMoE: Integrating Block-Level Conditioning into Expert Composition for Full-Participation Mixture-of-Experts
- arXiv：2609.21346
💡 一句话解读：现有混合专家模型无法同时兼顾三件事：每个词用多少专家（参与度）、实际算多少专家（算力）、要存多少套参数（显存）。稀疏路由省算力但参与度低，稠密混合参与度高但算力随专家数暴涨。本文提出 IntBMoE：用一个小码本生成若干“块”，每层用轻量网络把全部专家融合成一个组合专家，让每个词都用到全部专家（参与度满），但只计算少数几个块（算力省），且块的数量由码本固定（存储可控）。
混合专家（MoE）能扩展模型容量，但现有设计无法独立设定三个量。对单个 token 而言，参与度（participation）是有多少专家为其输出贡献知识，执行量（execution）是实际计算了多少专家（计算成本），物化量（materialization）是需要构建和存储多少套专家规模的参数（内存成本）。稀疏路由使执行量和物化量保持较低，但牺牲了参与度：每个 token 只有少数专家贡献。稠密输出混合恢复了完全参与，但其执行量随专家数量增长。参数合并将执行量保持在一个专家，但其物化量随路由决策数量增长。我们提出 IntBMoE，一种块级条件混合专家，通过将稠密专家组合与稀疏块执行配对，解耦上述三个量。其块来自一个小型可学习码本，每个条目对应一个块。在每个内部层，一个轻量级超网络将该层专家池中的所有专家基合并为一个组合专家。参与度是完全的，因为每个组合专家都利用了整个专家池；执行保持稀疏，因为路由器只将每个 token 发送到少数几个块；物化量是有界的，因为块的数量由码本而非输入决定。双路径残差门控（DPRG）进一步通过乘法门控耦合两条独立组合的路径。图像分类实验表明，该方法相较代表性的稀疏和稠密 MoE 基线均有稳定提升。在语言建模和序列推荐上的额外实验验证了其超越视觉领域的泛化能力。IntBMoE 已全面部署于高德的生成式推荐系统，在 60ms 延迟预算下服务数亿用户，在线 A/B 测试中取得 2.4% 的相对 UVCTR 提升。我们的代码见 https://github.com/AMAP-ML/DreamX-Rec/。
📄 English Abstract
Mixture-of-Experts (MoE) scales capacity, but existing designs cannot set three quantities independently. For a single token, participation is how many experts contribute knowledge to its output, execution is how many are actually computed (compute cost), and materialization is how many expert-sized parameter sets must be built and stored (memory cost). Sparse routing keeps execution and materialization low, but shrinks participation: for each token, only a few experts contribute. Dense output-mixing restores full participation, but its execution grows with the number of experts. Parameter-merging keeps execution at one expert, but its materialization grows with the number of routing decisions. We propose IntBMoE, a block-conditioned MoE that decouples all three by pairing dense expert composition with sparse block execution. Its blocks come from a small learned codebook, one per entry. At each internal layer, a lightweight hypernetwork merges all expert bases in that layer's pool into one composed expert. Participation is full, because every composed expert draws on the entire pool. Execution stays sparse, because a router sends each token to only a few blocks. Materialization is bounded, because the codebook, not the input, fixes how many blocks exist. Dual-Path Residual Gating (DPRG) further couples two independently composed paths through multiplicative gating. Experiments on image classification show consistent gains over representative sparse and dense MoE baselines. Additional experiments on language modeling and sequential recommendation validate its generalization beyond vision. IntBMoE is fully deployed in AMap's generative recommendation system, serving hundreds of millions of users under a 60ms latency budget, with a 2.4% relative UVCTR gain in online A/B testing. Our code is available at https://github.com/AMAP-ML/DreamX-Rec/.
6. OmniVChat：面向原生音视频对话的合成、基准测试与训练
- 英文标题：OmniVChat: Synthesizing, Benchmarking, and Training for Native Audio-Visual Dialogue
- arXiv：2609.21465
💡 一句话解读：这篇论文想解决的是：让AI能像真人一样，直接看着你、听着你说话来聊天，而不是先把你说的转成文字再回答。难点在于缺少真人用自己设备录制的对话数据，而且回答好不好很难用关键词判断。作者的办法是用多个AI代理合成大量音视频对话，用来训练和评测模型，并设计了一套奖励机制让回答更准、更快、更自然。
我们将 OmniVChat（全模态视频对话）定义为用户与全模态模型之间的原生音视频对话任务。在 OmniVChat 中，全模态模型直接同时接收用户的音频和视频并返回文本。用户的查询嵌入在音频和视频中，无需单独的文本问题、外部描述或语音识别。直接的音视频输入在保留感知线索的同时，降低了外部延迟和计算量。然而，OmniVChat 的研究面临两个限制：数据可用性和评估。人们使用自己设备录制的视频十分稀缺。此外，一个好的回复往往需要考虑用户周围环境、面部表情和附近物体，且这类回复可以有多种表达方式，使得关键词匹配难以可靠地评估回复质量。近期智能体系统和视频生成的进展使得“为理解而生成”变得可行，即利用合成对话进行训练和评估。因此，我们提出了 OmniVChat-Studio，一个用于合成单轮和多轮音视频对话的多智能体数据引擎。我们利用合成对话构建了 OmniVChat-Bench，一个评估基准，从五个能力类别评估全模态模型的基本对话能力。我们还提出了 OmniVChat-RL，一种强化学习奖励设计，在 OmniVChat 中联合优化回复的正确性、效率和风格。在合成对话上使用 OmniVChat-RL 训练 Qwen3-Omni-Instruct，提升了其在 OmniVChat-Bench 和人工录制的 OmniVChat-Bench-Human 上的表现。这些提升验证了奖励设计的有效性，并表明其可迁移至训练和评估中的真实世界对话。
📄 English Abstract
We define OmniVChat (Omni Video Chat) as the task of native audio-visual dialogue between a user and an omni model. In OmniVChat, omni models directly and simultaneously receive audio and video from a user and return text. The user's query is embedded in the audio and video, without a separate text question, external captioning, or speech recognition. Direct audio-visual input reduces external latency and computation while preserving perceptual cues. However, research on OmniVChat faces two constraints: data availability and evaluation. Recordings of people using their own devices are scarce. Furthermore, a good reply often needs to account for the user's surroundings, facial expressions, and nearby objects, and such responses can be expressed in many different ways, making keyword matching unreliable for evaluating reply quality. Recent progress in agent systems and video generation makes generation for comprehension viable, which means using synthesized dialogues for training and evaluation. Therefore, we present OmniVChat-Studio, a multi-agent data engine for synthesizing single- and multi-turn audio-visual dialogues. We use synthesized dialogues to build OmniVChat-Bench, an evaluation benchmark that evaluates omni models' basic dialogue abilities across five ability categories. We also present OmniVChat-RL, a reinforcement learning reward design that jointly targets reply correctness, efficiency, and style in OmniVChat. Training Qwen3-Omni-Instruct with OmniVChat-RL on synthesized dialogues improves its performance on both OmniVChat-Bench and the human-recorded OmniVChat-Bench-Human. These gains validate the reward design and show transfer to real-world dialogues in training and evaluation.
7. Paint-Anything：面向图像生成与编辑的统一任意颜色控制
- 英文标题：Paint-Anything: Unified Any-Color Control for Image Generation and Editing
- arXiv：2609.20816
💡 一句话解读：专业设计需要精确到任意十六进制色值的颜色控制，但现有方法往往依赖专门的色彩表示或特殊推理流程。本文提出 Paint-Anything，用统一的十六进制颜色提示接口，通过物体级颜色监督同时支持生成与编辑，并构建了 Paint-500K 数据和 ACBench 评测基准，显著提升了颜色还原准确度。
专业设计需要任意颜色控制：即能够为图像生成与编辑指定物体目标颜色的任意 24 位十六进制值。已有工作探索了颜色生成、编辑与上色，但往往依赖专门的色彩表示或特殊推理流程。大语言模型的进展提供了一个更简单的起点：即使是紧凑模型也能将十六进制值与颜色语义关联起来。我们提出 Paint-Anything，通过物体级颜色监督，学习一个用于生成和编辑的共享十六进制提示（hex-prompt）接口。我们开发了一条数据流水线，通过物体定位、感知颜色标注和编辑对合成，从真实图像构建出 Paint-500K。由于阴影使真实图像标签只能近似颜色，我们用像素与其配对十六进制值完全一致的纯色锚点来补充这种监督。这些锚点仅在高噪声时间步使用，低噪声训练则留给自然图像。我们进一步提出 Any Color Benchmark（ACBench），包含 ACBench-T2I 和 ACBench-Edit，用于衡量两个任务中的物体级十六进制颜色保真度。在 FLUX.2-4B 上，Paint-Anything 相比基础模型将 ACBench-T2I 和 ACBench-Edit 分数分别提升 85.3% 和 28.3%，消融实验也支持该训练方案。在对比方法中，它还取得了最高的平均 CompColor 分数。
📄 English Abstract
Professional design requires any-color control: the ability to specify an object's target color with any 24-bit hex value for image generation and editing. Prior work has explored color generation, editing, and colorization, but often relies on dedicated color representations or specialized inference procedures. Advances in large language models offer a simpler starting point: even compact models can associate hex values with color semantics. We present Paint-Anything, which learns a shared hex-prompt interface for generation and editing through object-level color supervision. We develop a data pipeline that constructs Paint-500K from real images through object grounding, perceptual color labeling, and editing-pair synthesis. Since shadows make real-image labels only approximate colors, we complement this supervision with pure-color anchors whose pixels exactly match their paired hex values. These anchors are used only at high-noise timesteps, leaving low-noise training to natural images. We further introduce Any Color Benchmark (ACBench), comprising ACBench-T2I and ACBench-Edit, to measure object-level hex color fidelity across both tasks. On FLUX.2-4B, Paint-Anything improves ACBench-T2I and ACBench-Edit scores by 85.3% and 28.3%, respectively, relative to the base model, with ablations supporting the training recipe. It also achieves the highest average CompColor score among the compared methods.
8. OmniVBench：面向全能参考到视频生成的基准与大规模数据集
- 英文标题：OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generation
- arXiv：2609.22069
💡 一句话解读：现有“参考图生视频”模型的测试标准太简单，只检查整体像不像，不管参考里的内容、动作、风格等是否被正确拆开、各归各位。为此，作者建了一个更全面的测试集 OmniVBench（含7类任务、18项细分任务）和一套逐项打勾的评分方法，还发布了34万条训练数据 Omni-R2V Dataset，帮大家训练和评估这类模型。
参考到视频（R2V）生成正朝着越来越通用、多功能的参考控制方向发展，催生了新兴的“全能 R2V 生成”范式。然而，现有基准无法覆盖这些新兴能力：其测试用例涉及的参考类型与组合有限，评估协议也主要考察整体参考一致性，忽略了参考因素是否被正确保留、解耦和路由。与此同时，构建全能 R2V 训练数据的高昂成本使得合适的训练资源十分稀缺。为填补这些空白，我们提出了 OmniVBench 和 Omni-R2V Dataset，用于评估和训练全能 R2V 模型。OmniVBench 将 R2V 评估扩展到更广泛的参考类型、细粒度控制任务和更丰富的参考组合，涵盖 7 个任务族和 18 项细粒度任务，涉及内容、运动、风格、结构、叙事及多参考设置。我们引入了基于因素的评估方法，包含 12,172 条针对具体案例的清单项，用以评估预期的参考因素是否被忠实保留、正确解耦并绑定到其目标，以及是否按指令正确实现。我们进一步推出 Omni-R2V Dataset，将面向多样化 R2V 任务的工业级训练资源带给更广泛的研究社区。该数据集主要基于大规模专业视频素材语料库，包含 34 万条经过处理的训练样本，涵盖多样化的参考类型和多参考组合。我们开发了针对具体任务的参考-目标配对构建流程，为全能 R2V 数据构建提供了实用且可扩展的方案。对先进开源和闭源 R2V 模型的大量评估显示，在 OmniVBench 上，各任务族和评估维度均存在明显性能差距，凸显了当前 R2V 模型的剩余局限。
📄 English Abstract
Reference-to-video (R2V) generation is evolving toward increasingly general and versatile reference control, giving rise to the emerging paradigm of omni R2V generation. However, existing benchmarks fall short of these emerging capabilities: their test cases cover limited reference types and compositions, and their evaluation protocols largely assess holistic reference consistency, overlooking whether reference factors are properly preserved, disentangled, and routed. Meanwhile, the high cost of constructing omni R2V training data makes suitable training resources scarce. To address these gaps, we introduce OmniVBench and the Omni-R2V Dataset for evaluating and training omni R2V models. OmniVBench expands R2V evaluation across broader reference types, fine-grained control tasks, and richer reference compositions, covering 7 task families and 18 fine-grained tasks spanning content, motion, style, structure, narrative, and multi-reference settings. We introduce factor-grounded evaluation with 12,172 case-specific checklist items, assessing whether intended reference factors are faithfully preserved, correctly disentangled and bound to their targets, and properly realized according to the instruction. We further introduce the Omni-R2V Dataset, bringing industrial-grade training resources for diverse R2V tasks to the broader research community. Drawing primarily on a large-scale corpus of professional video footage, it comprises 340K processed training samples spanning diverse reference types and multi-reference compositions. We develop task-specific pipelines for reference-target pair construction, offering a practical and scalable recipe for omni R2V data construction. Extensive evaluation of advanced open- and closed-source R2V models reveals clear performance gaps across task families and evaluation dimensions on OmniVBench, highlighting remaining limitations of current R2V models.
9. GraphSkillEvo：图结构智能体技能的进化优化
- 英文标题：GraphSkillEvo: Evolutionary Optimization of Graph-Structured Agent Skills
- arXiv：2609.21749
💡 一句话解读：大模型智能体执行任务时，常靠一份“技能说明”来指导操作，但现有说明都是大段自然语言，既啰嗦又没讲清步骤顺序，导致模型难执行、优化也难。本文把技能改写成“流程图”式的图结构：每个节点是一步操作及说明，箭头表示步骤间的跳转，并用进化算法（变异、交叉）不断筛选组合出更好的技能。
技能可以通过提供任务特定的流程指导来提升大语言模型（LLM）智能体的表现，而技能优化则通过迭代改进进一步增强其效果。然而，现有技能优化方法通常将技能表示为非结构化的自然语言指令，由此带来两个关键挑战：1）非结构化技能往往缺乏明确的工作流层面指导且包含大量冗余，使LLM难以执行；2）无约束自然语言技能的巨大搜索空间使技能优化效果不佳。为解决这些问题，我们提出将技能表示为图结构的自然语言产物：其中每个节点代表一个执行步骤及其操作指导，有向边则编码步骤之间依赖上下文的转移关系。与非结构化技能相比，图结构技能能够提供清晰的工作流层面指导，同时也有利于技能优化。基于这种结构化表示，我们提出GraphSkillEvo——一个面向图结构技能的、基于种群的进化优化框架，包含变异与交叉算子。通过维护多个候选技能并组合有效组件，GraphSkillEvo能够比纯基于LLM的迭代自我改进更广泛、更全面地探索结构化技能空间。在五个智能体基准上的大量实验表明，GraphSkillEvo持续优于强技能优化基线SkillOpt，在GPT-5.4-nano上平均准确率提升4.01%，在GPT-5.4上提升1.76%。我们的代码已发布于 https://github.com/ruisun7/GraphSkillEvo。
📄 English Abstract
Skills can improve the performance of Large Language Model (LLM) agents by providing task-specific procedural guidance, while skill optimization further improves their effectiveness through iterative refinement. However, existing skill optimization methods typically represent skills as unstructured natural-language instructions, creating two key challenges: 1) Unstructured skills often lack explicit workflow-level guidance and contain substantial redundancy, making them difficult for LLMs to execute; 2) the vast search space of unconstrained natural-language skills makes skill optimization ineffective. To address these challenges, we propose representing skills as graph-structured natural-language artifacts. In graph-structured skills, each node represents an execution step together with its operational guidance, while directed edges encode context-dependent transitions between steps. Compared to unstructured skills, graph-structured skills can provide clear workflow-level guidance. Moreover, the proposed graph-structured skill can also facilitate skill optimization. Building on this structured representation, we introduce GraphSkillEvo, a population-based evolutionary optimization framework with mutation and crossover operators for graph-structured skills. By maintaining multiple candidate skills and combining effective components, GraphSkillEvo enables broader and more comprehensive exploration of the structured skill space than purely LLM-based iterative self-refinement. Extensive experiments across five agent benchmarks demonstrate that GraphSkillEvo consistently outperforms the strong skill optimization baseline SkillOpt, improving average accuracy by 4.01% on GPT-5.4-nano and 1.76% on GPT-5.4. Our code is available at https://github.com/ruisun7/GraphSkillEvo.
10. MintAct：面向数字环境的统一视觉智能体
- 英文标题：MintAct: A Unified Visual Agent for Digital Environments
- arXiv：2609.22083
💡 一句话解读：这篇研究想解决的是：手机、电脑、网页上的操作助手各管一摊，互不通用。MintAct 用一套统一的视觉语言模型，把界面识别、跨设备多步操作和调用工具都包下来，并搭建可并行的训练环境和异步强化学习框架，让模型在多个测试上达到同类最佳水平。
我们提出 MintAct，一个统一的视觉语言模型系列，将 UI 定位（UI grounding）、跨移动端、桌面端和网页的多步导航，以及视觉工具使用整合在一起，并在 2B、4B 和 8B 三种规模上训练。通过精心设计环境、数据和训练方案，MintAct 模型在这些能力上均达到各领域专用模型（per-domain specialists）的水平。为此，我们开发了可扩展的环境与强化学习（RL）基础设施：环境侧可跨异构领域后端托管数百个并发实例，同时服务于轨迹数据收集和在线 RL；训练侧采用异步框架，显式控制跨领域训练分布，并在嘈杂环境反馈和离策略漂移（off-policy drift）下保持稳定。实验结果表明，在相近模型规模下，MintAct 在广泛基准测试中取得最先进性能（OSWorld-Verified 上为 48.9）。
📄 English Abstract
We present MintAct, a family of vision-language models that unifies UI grounding, multi-step navigation across mobile, desktop, and web, and visual tool use, trained at 2B, 4B, and 8B scales. Through careful design of our environments, data, and training recipes, MintAct models match the performance of per-domain specialists across all of these capabilities. To enable this, we develop a scalable environment and reinforcement learning (RL) infrastructure. On the environment side, we host hundreds of concurrent instances across heterogeneous per-domain backends, serving both trajectory data collection and online RL. To enable efficient and scalable RL training, an asynchronous framework keeps explicit control over the cross-domain training distribution and remains stable under noisy environment feedback and off-policy drift. Experimental results show that MintAct achieves state-of-the-art performance (48.9 on OSWorld-Verified) across a wide range of benchmarks at comparable model sizes.
11. Designer-RSI：从用户流量中演化程序性记忆以支持智能体图形设计
- 英文标题：Designer-RSI: Evolving Procedural Memory from User Traffic for Agentic Graphic Design
- arXiv：2609.22086
💡 一句话解读：专业图形设计任务步骤多、结果好坏没有可靠的自动评判标准，AI 很难自己越做越好。研究者让一个不更新参数的大模型操作 230 多个设计工具，同时把成功经验写成可复用的“技能笔记”存起来，不断补充新技能、修正旧技能，并只保留不破坏已有成功的改动。经过五轮真实用户任务训练，技能从 76 条增至 139 条，设计成功率从 72.7% 提升到 99.3%。
专业图形设计是一项长周期智能体任务，其结构化的可编辑成果源自许多相互依赖的操作，但结果缺乏可靠的程序化评判标准（oracle）。我们提出一种持续适应框架：一个冻结的前沿模型通过 230 多个工具操作专业设计软件，同时一个由自然语言技能构成的外部程序性记忆（procedural memory）从经验中积累并精炼可复用的设计流程。该记忆通过为反复出现且尚未覆盖的子任务获取新流程来“拓宽”，并通过依据自身成功与失败执行来修订已有流程而“加深”，同时一个匹配回放门控（matched replay gate）只接纳那些能修复失败且不使已观察到的成功退化的改动。在 1,406 份真实用户简报和 1,869 条自动评分的轨迹上进行五轮实验，无需更新权重、无需人工标注，技能库从 76 条源自文档的技能增长到 139 条，并将 Claude-Sonnet-4 上的 GenEval2 执行成功率从 72.7% 提升至 99.3%（生成质量提升 11.99 分），在四个专业设计基准上对 Claude-Sonnet-4 和 Claude-Opus-4.6 分别取得 61.8% 和 67.6% 的对无技能智能体的胜率。我们进一步表明这两种机制结合使用时有效：在用户流量基准的 200 份留出简报上，单独“拓宽”或“加深”对无技能智能体的胜率分别为 49.4% / 48.6%，而二者结合达到 58.5%（p = 0.025）。程序性记忆为智能体在嘈杂、不可验证反馈下的持续适应提供了一条实用路径。
📄 English Abstract
Professional graphic design is a long-horizon agentic task in which structured, editable artifacts emerge from many interdependent actions, yet outcomes admit no reliable programmatic oracle. We introduce a continual adaptation framework in which a frozen frontier model operates professional design software through more than 230 tools, while an external procedural memory of natural-language skills accumulates and refines reusable design procedures from experience. The memory widens by acquiring procedures for recurring uncovered subtasks and deepens by revising existing procedures against their own successful and failed executions, while a matched replay gate admits only changes that repair failures without regressing observed successes. Five rounds over 1,406 real user briefs and 1,869 automatically graded trajectories, with no weight updates and no human labels, grow the bank from 76 documentation-derived skills to 139 and raise GenEval2 execution success on Claude-Sonnet-4 from 72.7% to 99.3% (+11.99 points in generation quality), with 61.8% and 67.6% win rates against the no-skill agent across four specialized design benchmarks on Claude-Sonnet-4 and Claude-Opus-4.6. We further show the two mechanisms are effective in combination: on 200 held-out briefs from user-traffic benchmark, widening or deepening alone reaches a 49.4% / 48.6% win rate over the no-skill agent, while their combination reaches 58.5% (p = 0.025). Procedural memory offers a practical route to continual adaptation of agents under noisy, unverifiable feedback.
12. 当AI评审训练AI评审者：科学判断坍缩与缓解
- 英文标题：When AI Reviews Train AI Reviewers: Scientific-Judgment Collapse and Mitigation
- arXiv：2609.20942
💡 一句话解读：这篇研究关注AI评审员互相学习会导致评审意见越来越单调的问题。研究者让AI先学习人类评审，再用AI自己生成的评审去训练下一代AI，发现评分和意见的多样性都下降了。他们提出一种叫TrustReviewer的方法，通过精选训练数据和调整AI内部状态来保持评审的多样性。
大语言模型（LLMs）越来越多地参与科学评估，既作为自动评审员，也作为人类评审员的助手。随着模型生成的评审进入公共数据和未来的训练语料库，AI同行评审可能变得递归：后来的评审员从早期模型产生的判断中学习。我们在受控环境中研究了这一反馈回路的一步。从Llama 3.1 8B开始，我们首先在2018-2023年ICLR官方评审上微调了一个评审员，然后使用官方和模型生成评审的系统性混合数据在ICLR 2024数据上训练了四个后继模型。我们的研究表明，引入合成评审会压缩评分分布，并降低同一论文和语料库层面的语义多样性。我们将这种模式称为“科学判断坍缩”。为了缓解这种失败模式，我们引入了TrustReviewer，一个基于开源LLM的系统，用于生成AI和机器学习论文的同行评审。TrustReviewer在两个互补阶段进行干预。对于训练时的预防，我们在一个精心设计的语料库上以单阶段方式训练核心评审员，以减少低质量和语义退化的监督。对于测试时的校正，配对激活引导旨在进一步缓解残余的坍缩判断倾向，而无需进一步训练或额外的专家标注。总之，这些结果描述了递归评审员训练的具体风险，并为在AI辅助科学评估中保持判断多样性和改进推荐对齐提供了实际干预措施。
📄 English Abstract
Large language models (LLMs) increasingly participate in scientific evaluation, both as automated reviewers and as assistants to human reviewers. As model-generated reviews enter public data and future training corpora, AI peer review can become recursive: later reviewers learn from judgments produced by earlier models. We study one step of this feedback loop in a controlled setting. Starting from Llama 3.1 8B, we first fine-tune a reviewer on official ICLR reviews from 2018–2023 and then train four successor models on ICLR 2024 data with systematically varied mixtures of official and model-generated reviews. Our study shows that introducing synthetic reviews compresses rating distributions and reduces both same-paper and corpus-level semantic diversity. We call this pattern $\textbf{scientific-judgment collapse}$. To mitigate this failure mode, we introduce $\textbf{TrustReviewer}$, an open-source LLM-based system for generating peer reviews of AI and machine learning papers. TrustReviewer intervenes at two complementary stages. For training-time prevention, we train the core reviewer in a single stage on a curated corpus designed to reduce low-quality and semantically degenerate supervision. For test-time correction, paired activation steering aims to further mitigate residual tendencies toward collapsed judgments without further training or additional expert annotation. Together, these results characterize a concrete risk of recursive reviewer training and provide practical interventions for preserving judgment diversity and improving recommendation alignment in AI-assisted scientific evaluation.
13. 面向在线策略蒸馏的教师—学生差异校准
- 英文标题：Calibrating Teacher–Student Discrepancy for On-Policy Distillation
- arXiv：2609.21619
💡 一句话解读：在线蒸馏中，学生模型会模仿教师模型，但教师自身的偏差也会被学生学去。本文提出Cal-OPD方法，通过正负特权干预估计教师的自身偏差区域，只保留超出该区域的师生差异作为学习信号，从而让学生学得更准。
在线策略蒸馏（OPD）通过学习更强教师模型与在线学生模型之间的词元级差异来提升推理模型。然而，这种差异并不纯粹反映师生之间的能力差距：它还包含教师自身产生的偏差，这些偏差因此被混入观察到的师生差异中，并被标准OPD在训练中不加区分地学习。特权OPD进一步加剧了这一问题，因为特权信息会引发更大的教师侧似然偏移，从而鼓励学生更多地学习教师自身的偏差。我们提出了校准在线策略蒸馏（Cal-OPD），它通过正向和负向特权干预来估计教师的自身偏差区域，并通过仅保留超出该区域的成分来校准原始的师生差异。在数学推理基准上的实验表明，仅保留原始师生差异的约52%–65%作为优化信号时，Cal-OPD在不同模型规模上均持续优于标准OPD及其变体。
📄 English Abstract
On-policy distillation (OPD) improves reasoning models by learning the token-level discrepancy between a stronger teacher and an on-policy student. However, this discrepancy does not purely reflect the capability gap between the teacher and the student: it also contains deviations arising from the teacher itself, which are consequently mixed into the observed teacher–student discrepancy and indiscriminately learned by standard OPD during training. This issue is further exacerbated by privileged OPD, where privileged information induces larger teacher-side likelihood shifts, thereby encouraging the student to learn more of the teacher's own deviation. We introduce \textbf{Calibrated On-Policy Distillation (Cal-OPD)}, which estimates the teacher's self-deviation region through positive and negative privileged interventions and calibrates the original teacher–student discrepancy by retaining only the component that lies beyond this region. Experiments on mathematical reasoning benchmarks show that, while retaining only about 52–65% of the original teacher–student discrepancy as the optimization signal, Cal-OPD consistently outperforms standard OPD and its variants across model scales.
14. FRAUDSkill：面向音频反欺诈检测的结构化冻结权重技能优化
- 英文标题：FRAUDSkill: Structured Frozen-Weight Skill Optimization for Audio Anti-Fraud Detection
- arXiv：2609.18766
💡 一句话解读：现有音频反欺诈模型要按固定流程（识别场景、判断是否诈骗、再细分诈骗类型）输出结果，但微调或写提示词的方式把规则写死在模型里，诈骗手法和标注政策一变就难改。本文提出 FRAUDSkill：不动原模型，只在外层优化一套技能程序、分路策略和决策规则，并配合输出控制与多路验证推理，使预测符合规范。
大型音频语言模型通过直接处理语音并推理诈骗相关证据，在反欺诈检测中展现出潜力。然而其部署要求预测遵循预定义的标签空间，以及由服务场景识别、欺诈检测和条件式欺诈类型分类组成的结构化决策协议。现有的微调与基于提示的方法通常将任务知识、约束和决策规则编码进模型参数或人工维护的提示中，难以适应诈骗模式与标注策略的演变。为此，我们提出 FRAUDSkill，一种结构化冻结权重适配框架，它保持底层音频语言模型不变，转而优化外部的技能程序层、分路专用策略和决策规则。我们进一步将结构化输出控制与验证引导的多路径推理相结合，以确保预测符合协议。在 TeleAntiFraud 基准上，FRAUDSkill 取得 73.50% 的 Macro-F1，比共享冻结模型基线高出 31.96%，同时将无效输出降至 1.94%。大量实验表明，外部技能优化为结构化音频反欺诈检测提供了一种有效且可适配的方案，且无需修改底层模型。源代码见 https://anonymous.4open.science/r/FRAUDSKILL-114514。
📄 English Abstract
Large audio-language models have shown promise for anti-fraud detection by directly processing speech and reasoning over fraud-related evidence. Their deployment, however, requires predictions to follow a predefined label space and a structured decision protocol consisting of service-scenario identification, fraud detection, and conditional fraud-type classification. Existing fine-tuning and prompt-based approaches typically encode task knowledge, constraints, and decision rules into model parameters or manually maintained prompts, making them difficult to adapt as fraud patterns and labeling policies evolve. To this end, we propose FRAUDSkill, a structured frozen-weight adaptation framework that leaves the underlying audio-language model unchanged while optimizing an external layer of skill programs, route-specific policies, and decision rules. We further combine structured output control with validation-guided multi-path inference to ensure protocol-compliant predictions. On the TeleAntiFraud benchmark, FRAUDSkill achieves 73.50% Macro-F1, outperforming the shared frozen-model baseline by 31.96% while reducing invalid outputs to 1.94%. Extensive experiments demonstrate that external skill optimization provides an effective and adaptable solution for structured audio anti-fraud detection without modifying the underlying model. The source code is available at https://anonymous.4open.science/r/FRAUDSKILL-114514.
15. TeleAntiFraud 2.0：一个可刷新、以画像为依据且基于音频的电信诈骗检测基准
- 英文标题：TeleAntiFraud 2.0: A Refreshable, Profile-Grounded, and Audio-Based Benchmark for Telecom Fraud Detection
- arXiv：2609.18748
💡 一句话解读：电信诈骗话术更新快、又爱装成正常客服，现有测试集要么过时、要么拿话题完全不同的例子当“非诈骗”，测不出真实水平。本文提出 TeleAntiFraud 2.0：用混合树生成流程把诈骗案例改写成贴近真实场景的对话，并每月冻结一批900通电话（600诈骗+300易混正常），让模型在“真假难辨”的条件下接受评测。
电信诈骗话术快速演化，且常被设计成类似日常服务通话，这给基于音频的电信诈骗评测提出两项关键要求：第一，基准必须纳入新出现的诈骗模式，同时不覆盖此前已建立的测试集；第二，必须区分诈骗与合法且领域相近的通话，而不能依赖话题上完全分离的负样本。我们提出 TeleAntiFraud 2.0，它由我们的混合树反诈骗生成流程（Mixed-Tree Anti-Fraud Generation Pipeline）构建，并在每月冻结的评测协议下评估。该流程将在线诈骗案例摘要转化为以画像为依据的场景，通过混合树生成进行扩展，在共享语境下实现诈骗与非诈骗对话路径，将验证过的对话渲染为角色匹配的语音，并为每个月度评测集冻结生成的音频、标签、提示、清单与来源记录。每个冻结集包含900通中文电话，其中600通为诈骗、300通为领域相近的非诈骗案例。受控文本实验显示，三个分类器在针对无关或普通负样本评测时达到完美的宏平均F1（Macro-F1），但在领域相近的同类负样本下降至0.65–0.68。全量音频以及自动语音识别加大语言模型（ASR+LLM）评测进一步揭示了类别先验捷径、预测坍缩和快照敏感性问题。综上，这些发现确立了领域相近构建与坍缩感知报告，是在现实易混淆条件下评测基于音频的电信诈骗模型的核心要求。随附的研究工件包括构建代码、评测脚本、清单和文档。我们的数据集与代码见 https://anonymous.4open.science/r/TeleAntiFraud-2_0-EEB2/。
📄 English Abstract
Telecom fraud scripts evolve rapidly and are often designed to resemble routine service conversations, creating two key requirements for audio-based telecom-fraud evaluation. First, benchmarks must incorporate newly observed scam patterns without overwriting previously established test sets. Second, they must distinguish fraud from lawful, near-domain calls rather than relying on topic-separated negative examples. We present TeleAntiFraud 2.0, constructed with our Mixed-Tree Anti-Fraud Generation Pipeline and evaluated under a monthly frozen evaluation protocol. The pipeline transforms online fraud-case abstracts into profile-grounded scenarios, expands them through mixed-tree generation, realizes fraud and non-fraud dialogue paths under shared contexts, renders validated dialogues as role-matched speech, and freezes the resulting audio, labels, prompts, manifests, and provenance records for each monthly evaluation set. Each frozen set contains 900 Chinese calls, comprising 600 fraud and 300 near-domain non-fraud cases. Controlled text experiments show that three classifiers achieve perfect macro-averaged F1 (Macro-F1) when evaluated against unrelated or ordinary negatives, but drop to 0.65-0.68 with near-domain sibling negatives. Full-set audio and automatic-speech-recognition plus large-language-model (ASR+LLM) evaluations further reveal class-prior shortcuts, prediction collapse, and snapshot sensitivity. Together, these findings establish near-domain construction and collapse-aware reporting as core requirements for evaluating audio-based telecom-fraud models under realistic confusable conditions. The accompanying research artifact includes the construction code, evaluation scripts, manifests, and documentation. Our dataset and code are available at https://anonymous.4open.science/r/TeleAntiFraud-2_0-EEB2/.