Daily Papers · 2026-10-18

📄 论文日报 | 来源：Hugging Face Daily Papers
🗓 Episode：2026.10.16 | 全模态视频定位理解基准；强化学习门控蒸馏迁移法

1. OmniCompass：全模态视频定位与理解的综合性基准

💡 一句话解读：现有视频理解测试只覆盖部分能力，难以全面诊断模型水平。作者构建了 OmniCompass 基准，分视觉-语言和音视频两条评测线，每条线都考查“何时出现、在哪里、发生了什么”三类任务，并用人工标注数据测试了 50 个模型。

随着多模态大语言模型（MLLMs）的快速发展，评估其视频理解能力变得日益重要。现有视频理解基准仅覆盖部分评测维度，限制了对全模态视频理解的系统性诊断。我们提出 OmniCompass，一个面向全模态视频定位与理解的综合性基准。OmniCompass 定义了两条评测赛道：OmniCompass-VL 评估视觉-语言模型在需要视觉证据的任务上的表现，OmniCompass-AV 评估全模态模型在需要联合音视频证据的任务上的表现。每条赛道包含三个互补任务：时间定位（TG）、空间定位（SG）和问答（QA），分别考察被指目标何时出现、位于何处以及发生了什么。QA 在 OmniCompass-VL 中涵盖五个感知类别和三个推理类别，在 OmniCompass-AV 中涵盖三个感知类别和两个推理类别。OmniCompass-VL 包含 820 个视频和 2,460 个实例，覆盖多个领域以及短、中、长三种时长类型；一个叙事与音频丰富的子集构成 OmniCompass-AV，包含 240 个视频和 720 个实例。每个实例均在两种配对设置下评估：纯文本和文本+图像，分别通过文本和视觉-语言引导指定同一目标。所有实例均经人工标注并经过多轮质量验证。对 50 个 MLLM 的评估表明，开源模型仍落后于专有模型，尤其在音视频理解方面。即使对领先模型而言，精确的时间与空间定位仍然具有挑战性，且其预测在不同配对目标指定方式下仍表现出不一致性。

📄 English Abstract

With the rapid development of multimodal large language models (MLLMs), evaluating their video understanding capabilities has become increasingly important. Existing video understanding benchmarks cover only part of the evaluation dimensions, limiting systematic diagnosis of omni-modal video understanding. We introduce OmniCompass, a holistic benchmark for omni-modal video grounding and understanding. OmniCompass defines two evaluation tracks: OmniCompass-VL assesses vision-language models on tasks requiring visual evidence, while OmniCompass-AV assesses omni-modal models on tasks requiring joint audio-visual evidence. Each track includes three complementary tasks: temporal grounding (TG), spatial grounding (SG), and question answering (QA), assessing when a referenced target appears, where it is, and what happens. QA covers five perception and three reasoning categories in OmniCompass-VL, and three perception and two reasoning categories in OmniCompass-AV. OmniCompass-VL contains 820 videos and 2,460 instances across domains and three duration types: short, medium, and long. A narrative- and audio-rich subset forms OmniCompass-AV with 240 videos and 720 instances. Each instance is evaluated under two paired settings: text-only and text+image, specifying the same target through text and visual-language guidance. All instances are manually annotated with multi-round quality verification. Evaluations of 50 MLLMs show that open-source models still lag behind proprietary counterparts, particularly in audio-visual understanding. Precise temporal and spatial grounding remains challenging even for leading models, and their predictions still exhibit inconsistencies across the paired target specifications.

2. Waypoint-OPD：通过门控同策略蒸馏迁移强化学习引发的策略偏移

💡 一句话解读：强化学习训练后的教师模型会把“前后变化”教给学生，但只看最终变化可能掩盖训练中途方向反复的问题。本研究用训练中途的检查点比较早期和晚期变化方向是否一致，只在方向一致时传递信号，从而让学生学得更准。

策略偏移蒸馏（policy-shift distillation）将教师模型在强化学习（RL）前后对数概率的差异，作为对学生生成轨迹的监督信号。然而，这种仅基于终点的策略偏移无法揭示教师训练过程中局部动作偏好是否一致演化；相同的终点偏移可能由相互增强或部分抵消的早期与晚期偏移造成。我们研究早期与晚期教师策略偏移之间的一致性能否指导选择性迁移，并提出路径点引导的同策略蒸馏（Waypoint-guided On-Policy Distillation, Waypoint-OPD）：它利用同一 RL 运行中的中间教师检查点，在学生访问过的前缀处比较早期与晚期偏移在相对动作偏好上的差异，并根据二者方向一致性对终点偏移进行门控，当方向相反时予以抑制。与仅使用终点偏移的迁移相比，本方法仅需额外一次冻结教师评分。我们以 RL 训练的 Qwen3-4B 为教师，在数学与代码推理任务上以 Qwen3-8B、Qwen3-14B 和 Qwen3-30B-A3B 为学生模型进行评估；在全部六个领域—学生设置中，Waypoint-OPD 在所比较的蒸馏方法中取得最高的领域平均分。以 Qwen3-8B 为学生模型的消融实验表明，移除或打乱门控会降低数学推理性能。这些结果支持在学生训练中利用早期与晚期教师策略偏移的方向一致性来加权终点偏移。

📄 English Abstract

Policy-shift distillation reuses the difference between a teacher's log probabilities before and after reinforcement learning (RL) as supervision on student-generated trajectories. However, this endpoint policy shift cannot reveal whether local action preferences evolved consistently during teacher training. The same endpoint shift can result from early and late shifts that reinforce or partially cancel each other. We investigate whether agreement between early and late teacher policy shifts can guide selective transfer. We introduce Waypoint-guided On-Policy Distillation (Waypoint-OPD), which uses an intermediate teacher checkpoint from the same RL run to compare early and late shifts in relative action preferences at student-visited prefixes. We gate the endpoint shift according to the directional agreement between the early and late shifts, suppressing it when they point in opposing directions. Compared with endpoint-only transfer, our method requires only one additional frozen-teacher scoring pass. Using RL-trained Qwen3-4B teachers, we evaluate our method with Qwen3-8B, Qwen3-14B, and Qwen3-30B-A3B as student models in mathematical and code reasoning. In all six domain–student settings, Waypoint-OPD achieves the highest domain-average score among the compared distillation methods. Ablations with the Qwen3-8B student show that removing or shuffling gates reduces mathematical reasoning performance. These results support the use of directional agreement between early and late teacher policy shifts to weight the endpoint shift during student training.

3. 自进化智能体运行框架

💡 一句话解读：这篇研究要解决的是：在不改动AI模型本身参数的前提下，如何让AI智能体的外围软件（如提示词、记忆、工作流、代码组件）能通过反馈自动迭代进化。方法是提出一个“自进化框架”，把更新目标与提案、准入机制分开，并用持续学习理论分析其泛化性和可靠性。

“自进化框架”（Harness self-evolution）在保持模型权重不变的情况下，修改智能体的软件脚手架，包括提示词、记忆、工作流和可执行组件。我们形式化了一个反馈驱动的过程，其中被接受的修订会在多次运行间持续保留。该框架将更新目标与提案和准入机制分离，并区分了对解释状态的更改与对工具和代码、仓库级控制逻辑以及插件组合的修订。我们通过进化优化来刻画搜索过程，并通过持续PAC学习（continual PAC-learning）来检验其可靠性。在明确假设下，分析给出了期望泛化界、以历史回归预算为条件的保留界，以及使用新鲜独立验证数据的高概率准入规则。我们探讨了持久的框架修订如何影响知识共享和组织工作流，并对科学可靠性、物理安全和机构问责制产生影响。先前工作的案例说明了模型-框架协同设计、分布式执行和插件复用。可靠部署需要随着任务分布和评估器的变化进行持续验证，并对学习、运行时权限和后果性行动采取独立保障措施。我们提供了关于Harness RSI的相关工作、理论、算法、评估、应用及更广泛影响的66页全面讨论，以及持续更新的代码和补充材料，详见此URL。

📄 English Abstract

Harness self-evolution revises an agent's software scaffolding, including prompts, memory, workflows, and executable components, while keeping model weights fixed. We formalize a feedback-driven process in which accepted revisions persist across runs. The framework separates update targets from proposal and admission mechanisms. It distinguishes changes to interpreted state from revisions to tools and code, repository-level control logic, and plugin composition. We characterize search through evolutionary optimization and examine its reliability through continual PAC-learning. Under explicit assumptions, the analysis yields an expected generalization bound, a retention bound conditional on historical regression budgets, and a high-probability admission rule using fresh, independent validation data. We examine how persistent harness revisions affect knowledge sharing and organizational workflows, with implications for scientific reliability, physical safety, and institutional accountability. Cases from prior work illustrate model–harness co-design, distributed execution, and plugin reuse. Reliable deployment requires continual validation as task distributions and evaluators change, with independent safeguards for learning, runtime privileges, and consequential actions. We provide a comprehensive 66-page discussion of related work, theory, algorithms, evaluation, applications, and the broader impact of Harness RSI, together with continuously updated code and supplementary materials, at this https URL .

4. InfiniSplat v2：从任意视角分析与改进前馈 3DGS

💡 一句话解读：现有方法能从图片直接重建3D场景，但视角变化大时画质会明显变差。本研究先分析问题根源，再改进模型结构和训练方式：把几何预测与外观属性预测拆开、固定几何部分，训练时用真实相机位姿对齐高斯点，并用大角度视角监督，从而显著提升新视角渲染质量。

前馈高斯重建（feed-forward Gaussian reconstruction）直接从输入图像构建3D场景以用于新视角渲染，但如何从预测的几何中学习高斯属性、并在大视角变化下实现高质量渲染，仍具挑战性。我们分析了高斯属性学习中的局限，并提出 InfiniSplat v2，通过架构与训练方式的改进提升渲染质量。该方法用一个冻结的几何分支将几何预测与属性预测解耦，在训练时利用真实相机位姿对齐预测的高斯，并采用大角度目标视角监督。在六个数据集上，同一训练模型的前馈输出在姿态条件（pose-conditioned）与无姿态（pose-free）两种设置下，平均 LPIPS 和 DISTS 均在所评估方法中达到最优；在姿态条件设置下，平均 LPIPS 和 DISTS 相比最强的评估基线分别下降 25.8% 和 32.9%。我们进一步发现，仅用图像重建监督学习到的透明度（opacity）会抑制几何上不一致的点。

📄 English Abstract

Feed-forward Gaussian reconstruction builds 3D scenes directly from input images for novel view rendering, but learning Gaussian attributes from predicted geometry for high-quality rendering under large viewpoint changes remains challenging. We analyze limitations in Gaussian attribute learning and introduce InfiniSplat v2, which improves rendering quality through architectural and training changes. Our method decouples geometry and attribute prediction with a frozen geometry branch, aligns predicted Gaussians using ground-truth camera poses during training, and adopts large-angle target view supervision. Across six datasets, the feed-forward outputs of the same trained model achieve state-of-the-art mean LPIPS and DISTS among the evaluated methods in both pose-conditioned and pose-free settings. In the pose-conditioned setting, mean LPIPS and DISTS decrease by 25.8% and 32.9%, respectively, compared with the strongest evaluated baselines. We further find that opacity learned with only image reconstruction supervision suppresses geometrically inconsistent points. Project page: this https URL

5. 学习在何处分割：面向音频源分离的动态频带分割

💡 一句话解读：音频分离模型要把不同声音分开，但逐个频点处理太费算力。现有做法是把频谱切成固定频带、每带压缩成一个“词元”，可切法对所有音频都一样，容易把关键细节切没或浪费算力。本文让切分位置随每段音频内容动态调整，在词元预算不变下提升分离效果。

音频源分离模型既要保留精细的频谱结构，又要控制分离器的计算开销。频带分割（band-split）架构通过将频谱手动划分为预定义频带、并把每个频带编码为一个词元来降低开销，但所有混合信号都使用相同的边界。由于有信息量的频谱结构随源内容而变化，固定边界可能把有判别力的成分合并在一起，或在不需要高分辨率的地方浪费词元。我们提出动态频带分割（Dynamic Band Splitting, DBS），这是一个端到端框架，能在固定的频带词元预算下，为每个混合信号自适应调整边界位置和频带宽度。DBS 在频率块内预测以内容为条件的边界分数，用硬 Top-K 选出规定数量的边界，并施加依赖内容的池化以获得紧凑的频带词元；分离后，它将这些词元平滑并扩展回原始 STFT 网格，沿频率方向细化，并用频率共享的解码头解码复数掩码。在 DnR v2、MUSDB18-HQ 和 EchoSet 上，DBS 相比对应的静态模型提升了分离性能，其中 DBS-BSRNN 在 DnR v2 上的平均 SI-SDR 相对提升达 29.0%。这些结果以及在 DnR v2 上的边界分析表明，在固定频带词元预算下，内容自适应的频率划分对多种分离器骨干网络都有益。代码和音频演示见文中所给网址。

📄 English Abstract

Audio source separation models must preserve fine spectral structure while keeping the cost of the separator tractable. Band-split architectures reduce this cost by manually partitioning the spectrum into predefined frequency bands and encoding each band as a token, but the same boundaries are used for every mixture. Because informative spectral structure varies with source content, fixed boundaries can merge discriminative components or spend tokens where little resolution is needed. We introduce Dynamic Band Splitting (DBS), an end-to-end framework that adapts boundary locations and band widths to each mixture under a fixed band-token budget. DBS predicts content-conditioned boundary scores within frequency blocks, selects a prescribed number of boundaries using hard Top-$K$, and applies content-dependent pooling to obtain compact band tokens. After separation, it smooths and expands these tokens to the original STFT grid, refines them along frequency, and decodes complex masks with a frequency-shared head. DBS improved separation over the corresponding static models on DnR v2, MUSDB18-HQ, and EchoSet, with DBS-BSRNN achieving relative gain of 29.0% in average SI-SDR on DnR v2. These results and boundary analyses on DnR v2 show that content-adaptive frequency partitioning benefits multiple separator backbones under a fixed band-token budget. Code and audio demos are available at this https URL

6. RoTA-Bench：通过交互式投影对齐评估 3D 空间推理

💡 一句话解读：这篇研究要解决的问题是：视觉语言模型能否把二维看到的图像转成三维理解，并据此做出精确动作。为此作者提出 RoTA-Bench 测试基准，让模型通过连续旋转物体、依据视觉反馈调整动作，使物体的投影与目标轮廓对齐，从而考察其三维空间推理与精细控制能力。

视觉语言模型能否将二维观测转化为支持精确动作的三维理解？这需要推断物体几何、预判旋转效果，并通过视觉反馈修正动作。我们提出 RoTA-Bench，通过交互式三维到二维投影对齐来评估这些能力：给定目标轮廓，模型观察一个旋转后的物体，并在有限动作预算内选择一系列旋转以恢复目标投影。我们的自动化流程从二维图标构建保持轮廓的三维物体，并采用可逆初始化保证任务可解，共生成 674 个任务及一个分层抽样的 100 任务测试集。实验发现：(i) 人与模型之间仍存在显著差距：GPT-5.6-Sol 仅成功完成 17% 的任务，而 20 名本科生参与者合计以 100% 成功率完成测试集四次；(ii) 图像空间与位姿空间的进展可能背离：在相同状态下，模型比人类更常出现轮廓重叠改善而位姿对齐变差的情况（19.4% 对 10.9%）；(iii) 精细控制对交互历史敏感：移除该信息后，模型 96.8% 的动作停留在 30 度的粗粒度步长，任务成功率也明显下降。这些发现凸显了将视觉证据转化为精确三维控制的挑战，其难度超出识别或匹配单张图像。

📄 English Abstract

Can vision-language models turn 2D observations into 3D understanding that supports precise action? This requires inferring object geometry, anticipating the effects of rotation, and refining actions through visual feedback. We introduce RoTA-Bench to evaluate these abilities through interactive 3D-to-2D projection alignment. Given a target silhouette, a model observes a rotated object and chooses successive rotations to recover the target projection within a limited action budget. Our automated pipeline constructs silhouette-preserving 3D objects from 2D icons and uses reversible initialization to guarantee solvability, yielding 674 tasks and a stratified 100-task test set. Empirically, we find that (i) a substantial human-model gap persists: GPT-5.6-Sol succeeds on only 17% of tasks, whereas 20 undergraduate participants collectively complete the test set four times with a 100% success rate; (ii) image-space and pose-space progress can diverge: at matched states, the model more often improves silhouette overlap while worsening pose alignment than humans (19.4% vs. 10.9%); and (iii) fine-grained control is sensitive to interaction history: removing it leaves 96.8% of the model's actions at a coarse 30-degree step size and sharply reduces observed task success. These findings highlight the challenge of converting visual evidence into precise 3D control, beyond recognizing or matching individual images.

7. 3DCodeVerse：探索将 3D 世界作为代码

💡 一句话解读：让AI写3D内容往往难以编辑和验证。本研究构建了大规模可执行3D代码数据集、自动生成与修正框架，并用实验证明微调能提升开源模型的3D编程能力。

我们提出3DCodeVerse，一个使编码智能体能够大规模创建可编辑、可验证3D内容的研究生态系统。3DCodeVerse包含大规模高质量的可执行3D代码数据集、用于空间推理与自动验证的编码框架，以及3D编程的实践指导。首先，我们构建了3DCodeVerse-2M，一个大规模精选的独立可执行3D代码数据集；该语料通过收集开源数据、转换专门表示以及利用编码智能体生成新程序而建成，每个程序都打包了在其声明环境中执行所需的源文件与相关资源，以及可用的描述和验证记录。其次，我们开发了3DCodeVerse Harness，一个用于生成和精炼3D内容的多智能体有向无环图框架，它利用执行检查、空间测量和视觉反馈在生成过程中无需人工反馈即可自动评估和修正生成的程序。第三，我们对3D编程进行了系统的实证研究，建立了微调开源模型的基线，并通过框架的组件级分析得出了实践指导。大量实验表明，在我们的数据集上微调显著提升了开源模型的3D编程能力。值得注意的是，当两者均由Claude Opus 5.5使用时，我们的3DCodeVerse框架在成对人工评估中以90%的胜率击败Claude Design（3D Object）。

📄 English Abstract

We introduce 3DCodeVerse, a research ecosystem that enables coding agents to create editable and verifiable 3D content at scale. 3DCodeVerse comprises large-scale, high-quality datasets of executable 3D code, a coding harness for spatial reasoning and automated verification, and practical guidance for 3D coding. First, we construct 3DCodeVerse-2M, a large-scale curated dataset of standalone 3D executable code. We build this corpus by collecting open-source data, converting specialized representations, and generating new programs with coding agents. Each program is packaged with the source files and associated assets required for execution in its declared environment, along with available descriptions and verification records. Second, we develop the 3DCodeVerse Harness, a multi-agent directed acyclic graph framework for generating and refining 3D content. It uses execution checks, spatial measurements, and visual feedback to automatically evaluate and revise generated programs without human feedback during generation. Third, we conduct systematic empirical studies of 3D coding, establishing baselines with fine-tuned open models and deriving practical guidance through component-level analyses of our harness. Extensive experiments demonstrate that fine-tuning on our dataset significantly improves the 3D coding capabilities of open-source models. Notably, our 3DCodeVerse harness achieves a 90% win rate against Claude Design (3D Object) in pairwise human evaluations, when both are utilized by Claude Opus 5.5.

8. SCOPE：面向文本到图像生成中提示工程的语义边界信用优化

💡 一句话解读：现有方法把图像反馈平均分给提示词里的每个小词，忽略了对象、数量等其实成段出现。SCOPE让改写模型同时输出增强提示词和它的语义分段，按段汇总更新权重，无需额外奖励或渲染图像，在多个生成器上平均提升约7%~9%。

强化学习（RL）使语言模型能够通过为冻结的图像生成器改写提示词来改进文本到图像（T2I）生成。现有方法通常将源自图像的单一信号施加到各个子词（subword）上，尽管对象、数量、属性和关系构成连贯的语义片段（semantic span）。我们提出 SCOPE，一种基于组相对策略优化（GRPO）的目标函数，使更新权重与这一结构对齐。改写器联合生成增强提示词及其语义划分。SCOPE 在每个片段内对策略对数概率的变化进行池化，为其词元损失推导出一个共享系数，同时为每个完成保留单一优势值。它既不需要单独的片段奖励，也不需要额外的图像渲染。在 Qwen-Image、SD3 和 FLUX.1-dev 上，SCOPE 在 GeneVal 和 T2I-CompBench 上相较原始 GRPO 分别取得 9.48% 和 7.62% 的平均相对提升。消融研究表明，语义边界与片段内池化均对这些提升有贡献，支持将语义结构作为基于 RL 的提示工程的一种实用归纳偏置（inductive bias）。

📄 English Abstract

Reinforcement learning (RL) enables language models to improve text-to-image (T2I) generation by rewriting prompts for a frozen image generator. Existing methods typically apply a single signal derived from the image to individual subword tokens, although objects, counts, attributes, and relations form coherent semantic spans. We propose SCOPE, an objective based on Group Relative Policy Optimization (GRPO) that aligns update weighting with this structure. The rewriter jointly generates an enhanced prompt and its semantic partition. SCOPE pools shifts in policy log probabilities within each span to derive a shared coefficient for its token losses while retaining a single advantage for each completion. It requires neither separate span rewards nor additional image renders. Across Qwen-Image, SD3, and FLUX.1-dev, SCOPE achieves average relative gains over vanilla GRPO of 9.48% on GeneVal and 7.62% on T2I-CompBench. Ablation studies show that both semantic boundaries and pooling within spans contribute to these gains, supporting semantic structure as a practical inductive bias for prompt engineering with RL.

9. 面向高效自回归视频生成的推测强制

💡 一句话解读：视频生成模型对每个画面单元都做同样多的计算，但其实很多单元一开始就基本定型了，继续算纯属浪费。作者提出“推测强制”：把中间结果当草稿，提前判断哪些单元可以定稿冻结、不再计算，从而在保持画质的同时大幅提速。

少步自回归视频生成器支持流式生成，但它们对每个 token 都执行最终的去噪步骤，而不论其实际需要多少细化。我们发现这种统一计算往往是不必要的：许多 token 在最初几步就已接近其最终的全步输出，而另一些则需要一直细化到最后。我们提出 Speculative Forcing 框架，将中间干净估计视为草稿，并预测各个 token 何时可以被安全地提前接受。一旦被接受，该 token 即被冻结为最终输出、不再进行后续去噪，而其缓存的键和值仍可作为其余 token 的上下文使用。我们探索了两种验证器设计：一种无需训练，基于与干净内容的相似度及跨步稳定性；另一种是学习得到的，读取冻结生成器的中间特征来估计每个 token 与全步输出之间的剩余差距。在短视频、长视频和世界生成任务上，Speculative Forcing 在保持全步采样质量的同时显著减少了计算量。在 LongLive 上，它以每 token 约 1.7 个去噪步骤达到四步生成器的水平，在 41.7 和 37.8 fps 下运行速度提升超过 1.5 倍。在相机运动不断改变可见内容的 Lingbot-World 2.0 上，它以每 token 约 2.5 个去噪步骤保持质量，并以 24.5 fps 运行，提速 1.35 倍。

📄 English Abstract

Few-step autoregressive video generators enable streaming generation, yet they run every token through the final denoising step, regardless of how much refinement it needs. We find this uniform computation is often unnecessary: many tokens already closely match their eventual full-step outputs in the initial steps, while others require refinement until the end. We introduce Speculative Forcing, a framework that treats intermediate clean estimates as drafts and predicts when individual tokens can be safely accepted early. Once accepted, a token is frozen as its final output and undergoes no further denoising, while its cached keys and values remain available as context for the remaining tokens. We explore two alternative verifier designs: a training-free one based on similarity to clean content and stability across steps, and a learned one that reads the frozen generator's intermediate features to estimate each token's remaining gap to its full-step output. Across short-video, long-video, and world generation, Speculative Forcing preserves the quality of full-step sampling while substantially reducing computation. On LongLive, it matches the four-step generator with ~1.7 denoising steps per token, running over 1.5x faster at 41.7 and 37.8 fps. On Lingbot-World 2.0, where camera motion continually changes the visible content, it maintains quality with ~2.5 denoising steps per token and runs 1.35x faster at 24.5 fps.

10. MJev-Spatial：通过原生注意力匹配实现系统 1 空间智能

💡 一句话解读：视觉语言模型回答空间问题（如物体关系、距离、位置）时，习惯像聊天一样逐字生成答案，速度慢且浪费。本研究把空间判断统一变成“从候选里选一个”的分类题：选项、数值区间或图像区域都当作类别，直接用模型已有的注意力打分，一次前向就出结果，不生成任何文字，速度提升数倍且准确率更高。

空间智能使智能体能够感知物理世界并采取行动，其中许多判断必须在交互速率下完成：判断哪种空间关系成立、物体有多远、或应把物体放在哪里。视觉语言模型（VLM）通过自由形式的文本生成给出这些答案，即便输出空间很紧凑（几个选项之一、一个标量或一个二维点），仍要付出自回归解码和答案解析的代价。为此，我们提出 MJev-Spatial，一种系统 1 多模态决策模型，在统一的分类框架内处理全部三类空间理解任务。具体而言，MJev-Spatial 将每个候选视为一个类别，利用骨干网络预训练的查询和键投影对其打分，不增加额外参数；候选分别是选择题的给定选项，或经由粗到细分类细化的数值分箱和图像区域（分别用于数值预测和点预测）。推理时，每次分类决策只需一次前向传播，且不生成任何答案 token。为高效训练异构空间数据，我们在一次前向传播中用教师强制联合监督所有由粗到细阶段，并结合长度感知的全局批处理和跨 GPU 负载均衡，使训练吞吐量比原始批处理策略提高 30.8%。MJev-Spatial 基于 Qwen3.5-4B 构建，并在由现有空间数据集转换而来的 1230 万问答对上训练，在八个被测模型中取得最高总分（65.64%），在十二个基准中的七个上排名第一，并在全部十二个基准上优于未修改的自回归基座模型，同时运行速度比最快的自回归基线快 2.5 倍、比基座模型快 5.6 倍。

📄 English Abstract

Spatial intelligence lets an agent perceive and act in the physical world, and many of its judgments must be made at interaction rates: which spatial relation holds, how far an object is, or where to place it. Vision-language models (VLMs) produce these answers by free-form text generation, paying for autoregressive decoding and answer parsing even though the output space is compact: one of a few options, a scalar, or a 2D point. In light of this, we introduce MJev-Spatial, a System 1 multimodal decision model that tackles all three types of spatial understanding tasks within a unified classification framework. Specifically, MJev-Spatial treats each candidate as a class and scores it using the backbone's pretrained query and key projections, without adding parameters. The candidates are the given options for choice tasks, or value bins and image regions refined through coarse-to-fine classification for numerical and point prediction, respectively. At inference, each classification decision takes a single forward pass and generates no answer tokens. To train on heterogeneous spatial data efficiently, we supervise all coarse-to-fine stages jointly with teacher forcing in one forward pass, and combine this with length-aware global batching and cross-GPU load balancing, raising training throughput by 30.8% over the original batching strategy. Built on Qwen3.5-4B and trained on 12.3M question-answer pairs converted from existing spatial datasets, MJev-Spatial achieves the highest overall score (65.64%) among eight tested models, ranks first on seven of twelve benchmarks, and outperforms its unmodified autoregressive base model on all twelve, while running 2.5 times faster than the fastest autoregressive baseline and 5.6 times faster than the base model.

11. OpenGCUA：迈向通用计算机使用的全开放技术栈

💡 一句话解读：现有能操作电脑的AI助手（如GPT-6 Astra）大多是闭源的，难以研究其设计。为此，作者开源了OpenGCUA全栈项目，让模型能通过写代码来操作电脑，并把原本由固定程序做的决策交给模型，从而提升任务完成效率。

计算机使用正迎来转折点：关注点正从仅限图形用户界面（GUI）的交互，扩展到通过视觉与程序化接口在用户计算机上完成任务的系统。我们将通用计算机使用智能体（GCUA）定义为一种系统，其中模型自主选择如何跨视觉与程序化接口进行观察、行动和组织工作。然而，前沿实现（如GPT-6 Astra的计算机使用系统）大多仍闭源，限制了对它们设计的系统研究。我们推出Open Generalist CUA（OpenGCUA），这是一个与社区共同开发并开放GCUA全栈的持续项目。本报告呈现OpenGCUA的首个版本。代码执行是我们智能体设计的关键特性：它让中间结果决定后续操作，并使模型能将其现有编程能力直接应用于计算机交互。我们还提出从执行框架到模型的决策转移（harness-to-model decision transfer），将选定的观察和任务状态决策从固定的执行框架规则转移到模型。在OSWorld-v2上，我们的完整配置提升了所有八个评估模型（包括GPT-6 Astra和Claude Opus 5）的任务得分，平均部分得分提高+19.6分，并且与批量仅GUI执行相比，每个任务报告步骤平均减少53.0%。我们发布了实现与提示词、所有八个模型的评估轨迹，以及真实设备执行框架和跨设备SDK的初步版本。该执行框架解决了基准测试未完全涵盖的真实设备挑战，而SDK为计算机、移动设备和浏览器提供了通用编程接口。在我们的真实设备执行框架下，四个真实macOS任务中，选定的Qwen3.8-Max运行所报告的推理token数是GPT-6 Astra的19.4倍，突显了模型在真实设备执行效率上的显著差异。

📄 English Abstract

Computer use has reached a turning point: the focus is expanding from GUI-only interaction to systems that complete tasks on users' computers through visual and programmatic interfaces. We define a generalist computer-use agent (GCUA) as a system in which the model chooses how to observe, act, and organize its work across visual and programmatic interfaces. Yet frontier implementations, such as GPT-6 Astra's computer-use system, remain largely closed, limiting systematic study of their design. We introduce Open Generalist CUA (OpenGCUA), an ongoing project to develop and open the full stack for GCUAs with the community. This report presents the first release of OpenGCUA. Code execution is a key feature of our agent design: it lets intermediate results determine subsequent operations and enables models to apply their existing programming capabilities directly to computer interaction. We also propose harness-to-model decision transfer, moving selected observations and task-state decisions from fixed harness rules to the model. On OSWorld-v2, our full configuration improves task scores across all eight evaluated models, including GPT-6 Astra and Claude Opus 5, raising mean partial score by +19.6 points and reducing reported steps per task by an average of 53.0% relative to batched GUI-only execution. We release our implementation and prompts, evaluation trajectories from all eight models, and preliminary versions of a real-device harness and a cross-device SDK. The harness addresses real-device challenges not fully captured by benchmarks, while the SDK provides a common programming interface across computers, mobiles and browsers. Across four real macOS tasks under our real-device harness, selected Qwen3.8-Max runs use 19.4 times as many reported reasoning tokens as GPT-6 Astra, highlighting substantial differences between models in real-device execution efficiency.

12. Mosaic：多会话智能体强化学习

💡 一句话解读：现有智能体强化学习默认整条任务只在一个连续对话里完成，但压缩上下文、分叉子智能体等做法会让任务跨多个会话，训练与推理对不上。为此作者提出多会话智能体强化学习设定，并给出 Mosaic 方法：不训练评论家模型，而是在整条轨迹和单个会话两个层面分别估计优势，按每个会话的边际贡献分配奖励，从而更合理地使用会话。

现有大多数智能体强化学习方法假设单会话轨迹，即交互在一个持续累积的上下文中进行。然而，上下文压缩（context compaction）和子智能体分叉（subagent forking）等智能体技术可能产生跨越多个会话的轨迹，造成单会话训练假设与推理时执行之间的不匹配。为此，我们引入多会话智能体强化学习（Multi-Session Agentic Reinforcement Learning），这是一个更一般的设定，支持一条轨迹中包含任意数量的会话。在该设定下，一种被广泛采用的方法是将轨迹级优势广播到所有会话，但这无法区分各会话的贡献，也无法指导轨迹内的会话分配。为解决该问题，我们提出 Mosaic，一种采用无评论家（critic-free）分层优化的多会话智能体强化学习方法：在轨迹层面，它将会话使用信号纳入优势估计；在会话层面，它根据估计的边际贡献塑造每个会话的优势。在 BrowseComp 上使用 Qwen3.5-4B 的实验表明，Mosaic 在上下文压缩和子智能体分叉两种场景下均优于广播方法，且平均使用的会话数更少。进一步分析表明，我们的方法在会话效用和会话分配上均取得改进，我们认为这是未来多会话智能体强化学习进展的一个有前景的方向。

📄 English Abstract

Most existing agentic reinforcement learning methods assume single-session trajectories, where interaction proceeds within one continuously accumulated context. However, agentic techniques such as context compaction and subagent forking may produce trajectories spanning multiple sessions, creating a mismatch between the single-session training assumption and inference-time execution. Therefore, we introduce Multi-Session Agentic Reinforcement Learning, a more general setting that supports an arbitrary number of sessions within a trajectory. Under this setting, a widely adopted method is to broadcast trajectory-level advantages to all sessions, but this does not distinguish contributions from various sessions or guide session allocation within trajectories. To address this issue, we propose Mosaic, a multi-session agentic RL method with critic-free hierarchical optimization. At the trajectory level, it incorporates session-usage signals into advantage estimation. At the session level, it shapes each session's advantage according to the estimated marginal contribution. Experiments on BrowseComp with Qwen3.5-4B show that Mosaic outperforms broadcasting in both context compaction and subagent forking scenarios while using fewer sessions on average. Further analyses show that our method achieves improvements in session utility and allocation, which we suggested is a promising target for future advances in multi-session agentic RL.

13. Intern Lumina U2：面向全视觉理解与图像生成的多码本扩散大语言模型

💡 一句话解读：现有统一多模态模型要么用单一视觉编码（表达力弱），要么图文分属不同表示空间（训练易冲突）。本文提出 Intern Lumina U2：用多个互补码本共同表示每个视觉位置，把图像、视频、3D 都放进与文字一致的离散空间，由扩散语言模型统一建模，从而兼顾表达力与训练一致性。

统一多模态生成涵盖离散、连续与混合三类范式。单码本视觉分词器的离散方法天然契合文本建模，但表征能力有限；连续与混合方法保留更丰富的视觉表征，却在异构空间中以不同训练目标建模文本与图像，可能因表征空间不对齐而引入模态间的优化冲突。这些选择引出一个核心问题：如何在单一骨干网络中，于共享优化空间内高效地同时建模高容量视觉表征与文本内容。我们提出 Intern Lumina U2，据我们所知是首个在全离散空间中表示多码本视觉 token 的扩散语言模型。每个视觉位置由来自互补码本的一组码表示，在保持与离散文本建模兼容的同时提升表征容量。输入端在各位置拼接嵌入；输出端由轻量多码本自回归头依次预测每个位置内的码，而扩散骨干并行处理所有位置。基于 AToken 的统一 4D 表示，Intern Lumina U2 在单一骨干中支持图像、视频与 3D 输入，并在视觉理解、图像生成与图像编辑方面取得统一模型中的领先性能。我们还在昇腾与 NVIDIA 平台上构建了全栈训练、推理与评测系统，以验证模型的跨平台有效性。此外，我们通过稳定性感知采样策略提升推理效率，该策略结合置信度平台引导的解掩码方法与漂移触发的缓存刷新机制，使图像生成加速 4.45 倍。

📄 English Abstract

Unified multimodal generation covers discrete, continuous, and hybrid paradigms. Discrete approaches with single-codebook visual tokenizers align naturally with text modeling but have limited representational capacity. Continuous and hybrid approaches retain richer visual representations, but they model text and images in heterogeneous spaces with different training objectives, potentially introducing optimization conflicts between modalities with misaligned representation spaces. These choices expose a central question: how to efficiently model high-capacity visual representations alongside textual content in a shared optimization space within a single backbone. We present Intern Lumina U2, to our knowledge the first diffusion language model to represent multi-codebook visual tokens in a fully discrete space. Each visual position is represented by a group of codes from complementary codebooks, increasing its representational capacity while preserving compatibility with discrete text modeling. At the input, the embeddings are concatenated at each position; at the output, a lightweight multi-codebook autoregressive head predicts the codes within each position sequentially, while the diffusion backbone processes all positions in parallel. Built on the unified 4D representation of AToken, Intern Lumina U2 supports image, video, and 3D inputs within a single backbone and achieves leading performance among unified models in visual understanding, image generation, and image editing. We further build a full-stack training, inference, and evaluation system on both Ascend and NVIDIA platforms to verify the cross-platform effectiveness of our model. We also improve the inference efficiency through a stability-aware sampling strategy, which combines confidence-plateau-guided unmasking approach with drift-triggered cache refreshing mechanisms, accelerating image generation by 4.45x.

14. AutoSFT：面向监督微调中自动化数据研究的智能体系统

💡 一句话解读：现有自动训练系统面对完整训练流程时，往往只会反复修改训练数据，而且只能做一种操作（比如从固定数据里挑样本）。本研究要解决的是：能否让自动系统像人类数据工程师一样，同时会调整比例、删减、修复、替换和合成数据。为此提出 AutoSFT 多智能体系统，把执行与分析分开，从多个角度收集方案，并调用合成子智能体补足数据池覆盖不到的能力。

语言模型智能体正日益自动化后训练（post-training），但当被赋予完整训练栈时，它们会收敛到监督微调（SFT），并把大部分精力用于修改训练数据。现有系统只给智能体一种控制方式：从固定池中选行，或调用生成器；而人类数据工程师会调整比例、剪枝、修复、替换和合成数据。我们为这些改动设计了一个数据动作空间，并追问自动系统能否覆盖全部动作而非其中一种，以及这需要怎样的结构。我们先用单个智能体运行该循环，结果它只做到调整类别比例这一种动作，且上下文被执行输出填满并在搜索中途被丢弃。因此我们提出 AutoSFT——一个多智能体系统：由元智能体将执行与分析分离，从独立角度收集提案，并为数据池无法覆盖的能力调用合成子智能体，同时在每次比较中固定模型、训练器和预算。逐项加入各组件均有帮助：执行器将上下文损失减半，分析子智能体把最佳 IFEval 提升 +6.5，合成再带来 +3.4 IFEval 和 +2.9 IFBench。同一循环可迁移到代码任务，其中分析在单智能体毫无提升的情况下带来 +3.7 MBPP+。所有结果均来自每轮从基座模型重新训练的运行，因此每个组件的效果都可单独归因。我们的发现表明，自动化 SFT 数据研究更依赖覆盖多样的数据动作，而非强化单一动作。

📄 English Abstract

Language model agents increasingly automate post-training, but when given the full training stack they converge on supervised fine-tuning and spend most effort revising training data. Existing systems give the agent one kind of control, selecting rows from a fixed pool or calling a generator, whereas a human data engineer adjusts proportions, prunes, repairs, replaces and synthesizes the data. We design a data action space for these changes and ask whether an automated system can make the full range rather than one of them, and what structure that requires. We first run this loop with a single agent, and it reaches only one action, adjusting category proportions, while its context fills with execution output and is discarded mid-search. We therefore introduce \textbf{AutoSFT}, a multi-agent system where a meta agent separates execution from analysis, gathers proposals from independent angles, and calls a synthesis subagent for capabilities the pool cannot cover, with the model, the trainer, and the budget fixed within each comparison. Adding each component in turn helps: the executor halves context loss, the analysis subagents lift the best IFEval by $+6.5$, and synthesis adds $+3.4$ IFEval and $+2.9$ IFBench. The same loop transfers to code, where analysis adds $+3.7$ MBPP+ over a single agent that does not improve. Every result comes from runs that retrain from the base model each round, so each component's effect is individually attributable. Our findings suggest that automating SFT data research depends more on covering diverse data actions than on a stronger single action.

15. MAI-Gym：面向移动与计算机环境的 GUI 智能体评估与强化学习训练的开源一体化平台

💡 一句话解读：训练能操作手机和电脑界面的AI助手，一直受困于真实环境成本高、可练习任务太少。作者做了一个开源平台MAI-Gym，能批量模拟各种App和任务，让AI在模拟环境里反复练习并自动判分，还训练出9B小模型，成绩超过同类开源模型，且模拟中学到的技能能迁移到真机。

带可验证奖励的在线强化学习（RL）推动了数学、编程和工具使用领域的快速进展，但将其成功扩展到GUI智能体仍面临挑战，原因在于交互环境成本高昂且真实训练任务稀缺。我们提出MAI-Gym，一个开源的一体化平台，统一了跨移动端、计算机和网页环境的GUI智能体评估、数据收集与在线RL训练。通过共享的rollout流水线，MAI-Gym确保评估与训练中智能体行为一致，支持十个既有基准，并可扩展至20,000个并行环境实例。除可扩展基础设施外，MAI-Gym还能自动合成多样化的GUI应用与高难度任务，包括可重置的模拟应用以及具有执行验证任务校验器的连贯跨应用世界。利用MAI-Gym，我们训练了MAI-Gym-9B，一个面向移动端与计算机使用的统一9B智能体，在10B参数以下的开源GUI智能体中取得最优性能：MobileWorld上78.4%，OSWorld-Verified上76.2%。重要的是，在模拟的火车票预订应用中进行的RL可迁移至真实对应场景，使物理手机上的任务成功率从28.8%提升至48.1%。我们发布该框架、合成技能、模拟应用、2,000个已验证任务及模型检查点，以支持可复现研究和可扩展GUI智能体训练的进一步进展。

📄 English Abstract

Online reinforcement learning (RL) with verifiable rewards has driven rapid progress in mathematics, coding, and tool use, but scaling this success to GUI agents remains challenging due to the high cost of interactive environments and the scarcity of realistic training tasks. We introduce MAI-Gym, an open all-in-one platform that unifies GUI agent evaluation, data collection, and online RL training across mobile, computer, and web environments. Through a shared rollout pipeline, MAI-Gym ensures consistent agent behavior across evaluation and training, supports ten established benchmarks, and scales to 20,000 parallel environment instances. Beyond scalable infrastructure, MAI-Gym enables automated synthesis of diverse GUI applications and challenging tasks, including resettable mocked apps and coherent cross-app worlds with execution-validated task verifiers. Using MAI-Gym, we train MAI-Gym-9B, a unified 9B agent for mobile and computer use, achieving state-of-the-art performance among open GUI agents under 10B parameters: 78.4% on MobileWorld and 76.2% on OSWorld-Verified. Importantly, RL in a mocked train-ticketing app transfers to its real-world counterpart, improving task success on a phys- ical phone from 28.8% to 48.1%. We release the framework, synthesis skills, mocked apps, 2,000 verified tasks, and the model checkpoint to support repro- ducible research and further advances in scalable GUI agent training.

Daily Papers
Daily Papers · 2026-10-18
https://www.jzzzhang35.top/2026/10/18/daily-papers-2026-10-18/
作者
zjz
发布于
2026年10月18日
许可协议