Daily Papers · 2026-09-13
📄 论文日报 | 来源：Hugging Face Daily Papers
🗓 Episode：【周末特辑】9月第2周最火AI论文 | 世界模型降本训练智能体；路由调度驱动递归自改进
1. 通过世界模型扩展自动研究智能体
- 英文标题：Scaling Automatic Research Agents via World Models
- arXiv：2608.12564
💡 一句话解读：自动做科研的AI智能体训练时，真正跑实验（环境执行）比生成方案慢得多，成了瓶颈。作者提出用“世界模型”代替真实环境执行，并加了两招纠偏和降噪，训练提速3-4倍，小模型效果反超大模型。
自动化实证研究是人工智能的一个长期方向。近期的自动研究（AutoResearch）智能体使这一目标触手可及，因为现代大语言模型（LLM）展现出独立实现解决方案并从执行结果中学习的能力。在这些进展背后，后训练（尤其是强化学习，RL）起着核心作用。本文指出，在为这类智能体扩展强化学习时存在一个根本性矛盾：每条自动研究轨迹的两个组成部分（智能体生成与环境执行）的扩展方式截然不同，因为所有生成通过批处理共享计算资源，而每次执行都占用其专属沙箱和真实机器时间。因此，随着轨迹增长，环境执行主导了训练成本并成为瓶颈。为解决这一矛盾，我们提出世界模型强化学习（World Model RL, WMRL），用世界模型替代环境执行以消除该瓶颈。此外，世界模型可能并不完美，其奖励会受到偏差和噪声的干扰。因此，我们进一步为WMRL配备两种缓解措施——在线去偏（Online Debiasing）和逆方差去噪（Inverse-Variance Denoising），分别抵消偏差和抑制噪声。理论上，我们证明WMRL的这两种缓解措施都能严格改进收敛保证。实验上，WMRL在不同智能体规模的各种任务上加速训练3-4倍，同时超越标准强化学习基线的性能。此外，我们后训练得到的4B和9B智能体在留出基准上优于规模大得多的48B和120B开放权重智能体。除自动研究外，WMRL还可迁移至后训练具身VLA策略，这证明了我们方法的通用性。
📄 English Abstract
Automating empirical research is a long-standing direction of AI. Recent automatic research (AutoResearch) agents bring this goal within reach, as modern LLMs show the capability to independently implement solutions and learn from the execution outcomes. Behind these gains, post-training (especially RL) plays a central role. In this paper, we identify a fundamental tension when scaling RL for these agents: the two components of every AutoResearch trajectory (agent generation and environment execution) scale in very different manners, since all generation shares compute through batching, while each execution occupies its exclusive sandbox and real machine time. As a result, the environment execution dominates the training cost and becomes the bottleneck as trajectories grow. To resolve this tension, we propose World Model RL (WMRL), which replaces environment execution with a world model to remove this bottleneck. Additionally, the world model can be imperfect, as its rewards are corrupted by bias and noise. Therefore, we further equip WMRL with two mitigations, Online Debiasing and Inverse-Variance Denoising, which offset the bias and suppress the noise respectively. Theoretically, we prove that both mitigations of WMRL strictly improve the convergence guarantee. Empirically, WMRL accelerates training by 3-4x on various tasks at different agent scales, while exceeding the performance of standard RL baselines. Moreover, our post-trained 4B and 9B agents outperform much larger open-weight agents of 48B and 120B on held-out benchmarks. Beyond AutoResearch, WMRL also transfers to post-training embodied VLA policies, which demonstrates the generalizability of our method.
2. NeoHorse-1：通过带路由调度框架的智能体后训练实现递归自我改进
- 英文标题：NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness
- arXiv：2609.08183
💡 一句话解读：这篇论文想解决的是：AI 系统如何通过观察自身表现来自动改进下一轮学习，即递归自我改进。它提出 NeoHorse-1，把不同大小的模型组成池子并智能调度，记录每次用户请求的能力需求、选用的模型层级和交互结果，再把这些记录变成训练数据，经过筛选和分阶段课程训练，最后用评估反馈决定下一轮训练用什么数据，形成闭环。
递归自我改进（RSI）需要一种具体机制，使 AI 系统能够观察自身能力并将这些证据转化为下一轮学习。我们提出 NeoHorse-1，这是一个为探索该路径而开发的智能体原生模型系列，通过智能体后训练实现。该系统将异构模型池与智能路由相结合，记录每个用户轮次的预测能力需求、所选服务层级及后续交互。这些记录被转化为训练样本，保留交错推理、工具调用和运行环境上下文，并通过结构验证、六维语义评估和子场景级标注进行筛选。路由信号将监督微调组织为三阶段课程，并扩展到路由引导的在线策略蒸馏，其中教师在同一进度下监督学生生成的回复。能力引导分配随后将评估反馈转化为下一轮训练数据混合，从而闭合“评估—选择—更新”循环，使系统学会做什么决定了它接下来从什么中学习。在涵盖基于运行环境的智能体、工具使用、编程和指令遵循的十一个基准上，后训练将 4B 模型的宏平均从 58.94 提升至 64.87，将 9B 模型从 65.60 提升至 69.04，显著缩小了后训练 4B 模型与 9B 基础模型之间的总体差距。NeoHorse-1 为这种反馈驱动过程提供了初步原型，并指明了跨连续迭代实现运行环境介导的 RSI 的路径。
📄 English Abstract
Recursive self-improvement (RSI) requires a concrete mechanism through which an AI system observes its capabilities and converts that evidence into the next round of learning. We present NeoHorse-1, a family of agent-native models developed to explore this path through agentic post-training. Our system combines a heterogeneous model pool with intelligent routing, recording the predicted capability demand, selected service tier, and subsequent interaction for each user turn. These records are converted into training examples that preserve interleaved reasoning, tool calls, and harness context, and are admitted through structural validation, six-dimensional semantic evaluation, and subscene-level labeling. Routing signals organize supervised fine-tuning into a three-stage curriculum and extend to routing-guided on-policy distillation, where a teacher supervises student-generated responses under the same progression. Capability-guided allocation then converts evaluation feedback into the next training mixture, closing an evaluation-selection-update loop in which what the system learns to do shapes what it learns from next. Across eleven benchmarks covering harness-based agents, tool use, coding, and instruction following, post-training raises the macro-average from 58.94 to 64.87 at 4B and from 65.60 to 69.04 at 9B, substantially narrowing the aggregate gap between the post-trained 4B model and the 9B base model. NeoHorse-1 provides an initial prototype of this feedback-driven process and a path toward harness-mediated RSI across successive iterations.
3. AuK技术报告：用于语音生成与编辑的开源基础模型
- 英文标题：AuK Technical Report: An Open-Source Foundational Model for Speech Generation and Editing
- arXiv：2609.08936
💡 一句话解读：这篇论文要解决的是：语音生成和编辑通常需要不同模型，使用起来很麻烦。作者提出了AuK，一个开源基础模型，用自然语言指令和音频上下文统一了语音生成与编辑，并训练了约30亿条指令-音频数据，还通过蒸馏加速推理，最终模型AuK-Flash只需4步就能快速生成。
我们介绍了AuK，一个开源基础模型，通过自然语言指令和音频上下文的通用接口统一了语音生成与编辑。为支持这一广泛能力，我们构建了约30.3亿条指令-音频实例和195万小时的有效监督数据，涵盖五个任务族：语音生成、内容编辑、增强与分离、副语言编辑和声学编辑。AuK结合了用于语义条件控制的多模态大语言模型、在语音、通用音频和音乐上联合训练的VAE（变分自编码器）用于声学条件控制，以及一个混合整流流Transformer，该Transformer执行双流MMDiT块，随后是统一的单流DiT块进行生成。训练从仅生成预热开始，然后进行联合生成-编辑预训练。接着我们应用互补的后训练策略：针对开放式编辑的人类反馈偏好优化，以及针对语音生成的基于奖励的强化学习。为降低推理成本，我们进一步通过一致性初始化和任务路由的解耦DMD对模型进行蒸馏。最终得到的AuK-Flash在无分类器引导下执行4步推理，在匹配条件下相比完整模型实现了4.5倍的挂钟时间加速。实验表明，在零样本和指令控制的语音生成以及通用指令引导编辑上取得了领先性能，同时在信号级恢复任务上保持竞争力。我们发布了源代码和模型权重，以支持可复现性和进一步研究。
📄 English Abstract
We introduce AuK, an open-source foundational model that unifies speech generation and editing through a common interface of natural-language instructions and audio context. To support this broad capability set, we construct approximately 3.03 billion instruction–audio instances and 1.95 million hours of effective supervision across five task families: speech generation, content editing, enhancement and separation, paralinguistic editing, and acoustic editing. AuK combines a multimodal large language model for semantic conditioning, an VAE jointly trained on speech, general audio, and music for acoustic conditioning, and a hybrid rectified-flow Transformer that performs dual-stream MMDiT blocks followed by unified single-stream DiT blocks for generation. Training begins with generation-only warm-up and proceeds to joint generation–editing pre-training. We then apply complementary post-training strategies: human-feedback preference optimization for open-ended editing and reward-based reinforcement learning for speech generation. To reduce inference cost, we further distill the model with consistency initialization and task-routed Decoupled DMD. The resulting AuK-Flash performs 4-step inference without classifier-free guidance and achieves a 4.5 wall-clock speedup over the full model under matched conditions. Experiments demonstrate leading performance on zero-shot and instruction-controlled speech generation and general instruction-guided editing, while remaining competitive on signal-level restoration tasks. We release both the source code and model weights to support reproducibility and further research.
4. NCP-ArchPreview 技术报告：通过下一概念预测迈向潜在空间语言模型
- 英文标题：NCP-ArchPreview Technical Report: Moving towards Latent Space Language Models through Next Concept Prediction
- arXiv：2609.10715
💡 一句话解读：现有语言模型逐词预测，效率不高。本文提出NCP-ArchPreview，在逐词预测之外，增加“预测未来一段话的核心概念”这一更难的任务，用概念指导生成。模型规模达89亿参数，仅用约一半训练量就达到同类模型效果，下游任务平均提升2.45分。
我们提出NCP-ArchPreview，一种潜在空间语言模型，将自回归预训练推向了标准下一词元预测（NTP）之外。除NTP外，该模型还通过下一概念预测（NCP）学习预测跨越多个词元的离散概念，在保留标准词元级自回归生成的同时，引入了一个显式且更具挑战性的概念级目标。NCP-ArchPreview通过直接从其隐藏状态构建乘积量化概念词表来建立潜在空间，并随后通过专门的Concept Module学习预测未来概念。这些预测的概念随后被反馈到词元级以指导后续生成，NTP和NCP以端到端方式联合训练。我们将该架构扩展到89亿参数，并在Dolma-3数据集的5.73万亿词元上进行训练，这是迄今为止潜在空间语言模型的最大规模演示。值得注意的是，NCP-ArchPreview仅消耗总训练词元的51.3%，就达到了OLMo-3-7B的最终预训练损失。在完整预训练后，其在下游宏平均上比OLMo-3-7B高出2.45分，其中在GSM8K上显著提升了5.99分。控制实验分离出了来自潜在架构和NCP目标的清晰性能提升进展。此外，仅使用标准计算的85%，NCP-ArchPreview就接近了严格参数对齐的89亿基线模型的训练损失。学习到的潜在空间在预训练阶段之后仍极具价值：仅更新1700万参数的VQ模块就能产生一种新颖、轻量的领域适应接口，而将概念表示简单注入DFlash2草稿模型后，在可忽略的开销下将平均接受长度提高了4.17%。
📄 English Abstract
We introduce NCP-ArchPreview, a latent-space language model that pushes autoregressive pretraining beyond standard next-token prediction (NTP). Alongside NTP, the model learns through Next Concept Prediction (NCP) to predict discrete concepts that span multiple tokens, introducing an explicit and more challenging concept-level objective while preserving standard token-level autoregressive generation. NCP-ArchPreview builds a latent space by constructing a product-quantized concept vocabulary directly from its hidden states, and subsequently learns to predict future concepts via a dedicated Concept Module. These predicted concepts are then fed back to the token level to guide subsequent generation, with NTP and NCP trained jointly end-to-end. We scale this architecture to 8.9B parameters and train it on 5.73T tokens from the Dolma-3 dataset, marking the largest demonstration of a latent-space language model to date. Remarkably, by consuming only 51.3% of the total training tokens, NCP-ArchPreview achieves the final pretraining loss of OLMo-3-7B. Following full pretraining, it outperforms OLMo-3-7B by 2.45 points on the downstream macro-average, including a notable 5.99-point gain on GSM8K. Controlled experiments isolate a clear progression of performance gains stemming from both the latent architecture and the NCP objective. Furthermore, utilizing only 85% of the standard computation, NCP-ArchPreview approaches the training loss of a strictly parameter-aligned 8.9B baseline. The learned latent space remains highly valuable after the pretraining stage: updating just the 17M-parameter VQ module yields a novel, lightweight interface for domain adaptation, while a simple injection of concept representations into a DFlash2 drafter improves the mean accepted length by 4.17% with negligible overhead.
5. Dr. Claw：面向 Vibe Research 的 AI 科学家工作空间
- 英文标题：Dr. Claw: An AI Scientist Workspace for Vibe Research
- arXiv：2609.00365
💡 一句话解读：做研究时，AI 助手、聊天工具、编辑器和终端各管一摊，过程零散、决策难以追溯。作者做了 Dr. Claw 这个开源工作空间，把现有编程 AI 包进一个可人工把关、可审计的流程里，用状态记录、技能库和多执行器协作把规划、执行、写作串成可回溯的闭环。
命令行编程智能体（如 Claude Code、Gemini CLI）已能读写文件并维持长会话，但端到端研究仍分散在聊天工具、IDE、终端和写作环境中，且使研究可审计的决策很少被保留。我们提出 Dr. Claw，一个开源工作空间，它将现有的编程智能体执行器包裹在可控且可审计的人在环（human-in-the-loop）工作流中，而非引入另一个自主智能体。持久化状态对象、可复用技能库和多执行器协调将人类决策与 AI 执行连接起来，把规划、执行和写作变成一个可追溯、可恢复的闭环。我们通过一个交互式三视图场景和一次故障恢复演示来展示 Dr. Claw，并将其与共享同一后端执行器的裸命令行智能体进行对比评估，从而将整个编排层（任务图、状态对象和技能库）与其所包裹的智能体进行对照。在执行器固定的情况下，Dr. Claw 在研究完整性上得分更高，同时持久化了一条可审计、可恢复的过程轨迹。演示访问：仓库见本 URL，以 AGPL-3.0 发布，含 GPL-3.0 上游组件。
📄 English Abstract
Command-line coding agents (e.g., Claude Code, Gemini CLI) can already read and write files and sustain long sessions, yet end-to-end research still fragments across chat tools, IDEs, terminals, and writing environments, and the decisions that make it auditable are rarely preserved. We present Dr. Claw, an open-source workspace that wraps existing coding-agent executors in a controllable and auditable human-in-the-loop workflow rather than introducing another autonomous agent. Persistent state objects, a reusable skill library, and multi-executor coordination link human decisions to AI execution, turning planning, execution, and writing into one traceable, recoverable loop. We demonstrate Dr. Claw through an interactive three-view scenario and a failure-recovery walkthrough, and evaluate it against a bare command-line agent sharing the same backend executor, so the comparison contrasts the whole orchestration layer (task graph, state objects, and skill library) with the agent it wraps. Holding the executor fixed, Dr. Claw scores higher on research completeness while persisting an auditable, recoverable process trail. Demo access: repository this https URL, released under AGPL-3.0 with GPL-3.0 upstream components.