Daily Papers · 2026-10-22
📄 论文日报 | 来源：Hugging Face Daily Papers
🗓 Episode：
1. 聚焦式全模态GUI智能体
- 英文标题：Focused Omni-Modal GUI Agents
- arXiv：2610.26203
💡 一句话解读：图形界面智能体要同时看屏幕、听声音、读历史，但每一步真正有用的信息很少且不断变化，全塞进去反而互相干扰、越错越偏。作者提出 FocusGUI：一个小插件按当前步骤挑出最相关的信息，压成紧凑表示后轻插进冻结模型，不改变原模型参数，只用动作监督训练。
图形用户界面（GUI）智能体越来越多地在跨截图、音频、视频、交互历史和设备状态的流式全模态观测上执行动作。然而，在每一步都统一暴露完整上下文，忽视了相关性是短暂且稀疏的：决定某一步动作的信号在下一步可能就变成干扰。由于已执行动作会反馈回上下文，这些错误会累积成“序列分心”，使智能体发生不可逆的轨迹漂移。我们将这一根本需求表述为“聚焦感知”：一种步自适应、动作充分且非侵入式的表示。我们在 FocusGUI 中实现了这一点，它是一个轻量适配器，针对特定步骤的决策参考动态地对多模态通道进行优先级排序。它并非统一处理所有输入，而是将最相关的证据压缩为紧凑的潜在表示，并微妙地注入冻结解码器的嵌入中，在不改变其原生参数空间的情况下引导模型注意力。FocusGUI 仅通过动作监督训练，既不需要模态相关性标签，也不需要微调骨干网络。在四个全模态骨干网络上，它平均将整体类型匹配（Type Match）比冻结骨干提高 33.2，将精确匹配（Exact Match）提高 3.0 倍，并将类型匹配比最强的现有增强方法提高 30.1。
📄 English Abstract
Graphical User Interface (GUI) agents increasingly act over streaming omni-modal observations spanning screenshots, audio, video, interaction history, and device states. However, exposing this complete context uniformly at every step ignores that relevance is transient and sparse: the signal determining one action can become clutter at the next. Because executed actions feed back into the context, these errors compound into Sequential Distraction, driving the agent into irreversible trajectory drift. We formulate the underlying requirement as Focused Perception: a representation that is step-adaptive, action-sufficient, and non-intrusive. We realize this in FocusGUI, a lightweight adapter that dynamically prioritizes multimodal channels against a step-specific decision reference. Instead of uniformly processing all inputs, it compresses the most relevant evidence into a compact latent and subtly injects it into the frozen decoder's embeddings, guiding the model's attention without altering its native parameter space. FocusGUI trains from action supervision alone, requiring neither modality-relevance labels nor backbone fine-tuning. Across four omni-modal backbones, it raises overall Type Match by +33.2 and Exact Match by 3.0 times over the frozen backbone on average, and Type Match by +30.1 over the strongest existing enhancement method.
2. PAR：面向状态跟踪长度泛化的进度对齐循环
- 英文标题：PAR: Progress-Aligned Recurrence for Length Generalization in State Tracking
- arXiv：2610.27628
💡 一句话解读：循环模型测试时多跑几轮不一定能解决更长的问题：最终答案监督没说清每轮该推进到哪，反复更新还会让状态数值越滚越大。作者提出进度对齐循环（PAR），让第k轮预测对应前k步操作后的状态，并把循环状态缩回第一轮的数值大小，从而在远超训练长度的问题上保持准确。
循环语言模型在测试时可以运行更多迭代，但额外的迭代未必有助于解决更长的问题。对于序列状态跟踪，最终答案监督并未规定每次迭代应推进多远，且重复更新会改变循环状态的尺度。我们提出进度对齐循环（Progress-Aligned Recurrence, PAR），训练第 k 次迭代的预测去匹配前 k 次操作之后的状态，并将循环状态重新缩放到其首次迭代的范数。在 swap-S5 任务上，一个最多用 32 步训练过的 Qwen2.5-0.5B 模型在 512 步（即训练长度的 16 倍）时达到 95.5% 的平均全状态准确率，而同一骨干上 RLT 报告为 55.7%、显式思维链仅为 0.8%。受控的 S5 消融实验表明，去掉范数锚定会把外推倍数从 16 倍降到 2.2 倍以下，而用最终答案监督替代进度对齐目标则至多降到 8 倍。一个学习得到的完成读出模块可在未提供停止深度时选择何时作答。当中间标签不可用时，前缀自蒸馏仅利用最终答案即可从较短的问题前缀中诱导出这种对应关系。这些增益适用于多种序列状态跟踪任务，确立了实现深度外推所需的任务结构。这些发现支持将中间预测与状态尺度一起训练，从而使额外的循环深度对更长的问题有用。
📄 English Abstract
Looped language models can run more iterations at test time, but additional iterations need not help them solve longer problems. For sequential state tracking, final-answer supervision does not specify how far each iteration should advance, and repeated updates can change the scale of the recurrent state. We introduce \textbf{Progress-Aligned Recurrence (PAR)}, which trains the prediction at iteration $k$ to match the state after the first $k$ operations and rescales recurrent states to their first-iteration norms. On swap-$S_5$, a Qwen2.5-0.5B model trained on at most 32 moves achieves 95.5% mean full-state accuracy at 512 moves, or $16\times$ the training length, compared with 55.7% reported for RLT and 0.8% for explicit chain of thought on the same backbone. Controlled $S_5$ ablations show that removing norm anchoring reduces the extrapolation factor from $16\times$ to below $2.2\times$, and replacing progress-aligned targets with final-answer supervision reduces it to at most $8\times$. A learned completion readout selects when to answer without a supplied stopping depth. When intermediate labels are unavailable, prefix self-distillation induces the correspondence from shorter problem prefixes using only final answers. The gains apply across sequential state-tracking tasks, establishing the task structure required for depth extrapolation. These findings support training intermediate predictions and state scale together to make additional recurrent depth useful for longer problems.
3. Matrix-Game 4.0：利用视频模型打造实时可玩游戏世界
- 英文标题：Matrix-Game 4.0: Harnessing Video Models for Real-Time Playable Game Worlds
- arXiv：2610.27999
💡 一句话解读：现有视频模型只能生成可控视频，无法变成真正可玩、规则一致的游戏世界。本文把游戏逻辑交给轻量引擎执行，用多模态编码代理把玩家想法转成游戏代码，再用视频模型把低多边形画面实时渲染成高清画面，从而在单张显卡上实现低延迟、可自定义的实时游戏。
从可控视频迈向真正可玩的世界，需要持久的世界状态、可执行的游戏规则和低延迟交互。我们提出了下一代 Matrix-Game 的路线图，通过可执行的游戏框架将游戏执行与视觉生成分离，并推出首个基于该设计的系统 Matrix-Game 4.0。轻量级游戏引擎维护世界状态并执行交互规则，多模态编码代理将玩家想法转化为游戏代码和动态场景描述，而基于自回归视频模型的渲染器则以动态描述和记忆参考为条件，将引擎渲染的低多边形代理画面转化为精细视觉。该框架为视觉生成提供持续引导，并支持用自然语言创建和修改场景与玩法。我们的渲染器在对齐的低多边形与高保真视频对上训练，并蒸馏为因果生成器，以四帧为块、仅需一至两步去噪进行生成。结合针对性的推理优化，Matrix-Game 4.0 可在单张 NVIDIA RTX5090 上实现 480p、60 FPS 的游戏体验，按键到显示延迟为 112 毫秒。单步生成在稳态下达到 83 FPS，峰值 GPU 内存占用 18.2GiB。在固定提示下，稳态吞吐量在 RTX5080 上达到 67 FPS，在 RTX5070Ti 上达到 58 FPS。我们展示了具有引擎强制机制的可定制射击和驾驶游戏，并发布代码、模型权重和低多边形游戏资源，以支持可玩游戏世界的研究与创作。
📄 English Abstract
Moving from controllable videos to genuinely playable worlds requires persistent world state, executable game rules, and low-latency interaction. We outline a roadmap for the next generation of Matrix-Game that separates game execution from visual generation through an executable game harness, and introduce Matrix-Game 4.0 as the first system built on this design. A lightweight game engine maintains world state and executes interaction rules, a multimodal coding agent translates player ideas into game code and dynamic scene captions, and a renderer based on autoregressive video model transforms engine-rendered low-poly proxies into detailed visuals, conditioned on dynamic captions and memory references. This framework provides continuous guidance for visual generation and enables natural-language creation and modification of scenes and gameplay. Our renderer is trained on aligned low-poly and high-fidelity video pairs and distilled into a causal generator that generates in four-frame chunks with one or two denoising steps. Together with targeted inference optimizations, Matrix-Game 4.0 enables 480p, 60 FPS gameplay on a single NVIDIA RTX5090 with a key-press-to-display latency of 112ms. Single-step generation achieves 83FPS steady-state throughput with 18.2GiB of peak GPU memory usage. With fixed prompts, steady-state throughput also reaches 67FPS on an RTX5080 and 58FPS on an RTX5070Ti. We demonstrate customizable shooter and driving games with engine-enforced mechanics, and release code, model weights, and low-poly game assets to support research and creation of playable game worlds.
4. Ropedia W0：借助人类经验跨具身扩展移动操作世界模型
- 英文标题：Ropedia W0: Scaling Loco-Manipulation World Models across Embodiments with Human Experience
- arXiv：2610.26855
💡 一句话解读：现有世界模型很难同时处理“走路”和“动手”两类动作，且不同身体（人、机器人）的动作方式差异大、数据少。作者构建了大规模人类与机器人第一视角数据，用统一方式表示身体动作，训练出一个能跨人、跨机器人预测动作后果的世界模型，并支持实时交互。
世界模型能让具身智能体预判动作后果，但由于动作空间异构、多模态监督有限，跨多种具身形态联合建模移动（locomotion）与操作（manipulation）仍具挑战。本文提出 Ropedia W0，这是首个覆盖人类与多种具身机器人的第一视角移动操作世界基础模型。我们构建了 Xp-Data，包含 1 万小时第一视角人类数据和 3 千小时机器人数据，并配有统一的多模态标注与系统化质量控制；采用统一运动学表示，以相机位姿编码移动、以投影骨架编码操作，机器人骨架则结合具身特定的前向运动学与大量人工关键点标注和校正。我们还提出基于高质量灵巧人形遥操作数据的长时程移动操作基准 Xp-Teleop。评估表明，该模型在五个跨具身基准（DreamDojo、Ψ0、ω-HOME、Xp-Teleop 和 AgiBot）以及留出的人类第一视角视频上均取得最先进性能，涵盖零样本推理、少样本和全数据后训练。扩展人类预训练数据可同时提升人类与机器人评估表现。经蒸馏后，模型支持实时、分钟级的长时推演，可实现实时 VR 交互。在策略评估、基于模型的规划以及世界-动作建模等应用上，进一步证明了该模型对具身决策的实用价值。
📄 English Abstract
World models enable embodied agents to anticipate action consequences, yet jointly modeling locomotion and manipulation across diverse embodiments remains challenging due to heterogeneous action spaces and limited multimodal supervision. We present Ropedia W0, the first egocentric loco-manipulation world foundation model spanning humans and robots with diverse embodiments. We curate Xp-Data, comprising 10k hours of egocentric human data and 3k hours of robot data with unified multimodal annotations and systematic quality control. A unified kinematic representation encodes locomotion through camera poses and manipulation through projected skeletons, with robot skeletons combining embodiment-specific forward kinematics with extensive manual keypoint annotation and correction. We also introduce Xp-Teleop, a long-horizon loco-manipulation benchmark based on high-quality dexterous humanoid teleoperation data. Evaluations demonstrate state-of-the-art performance across five cross-embodiment benchmarks (DreamDojo, $\Psi_0$, $\omega$-HOME, Xp-Teleop, and AgiBot) and on held-out human egocentric video, from zero-shot inference to few-shot and full-data post-training. Scaling human pre-training data improves performance on both human and robot evaluations. After distillation, our model supports real-time, minute-long rollouts, enabling live VR interaction. Applications in policy evaluation, model-based planning, and world-action modeling further demonstrate the model's utility for embodied decision-making.
5. S-Agent 2.0：基于通用编程智能体的空间推理
- 英文标题：S-Agent 2.0: Spatial Reasoning with a General Coding Agent
- arXiv：2610.28354
💡 一句话解读：现有 AI 做空间推理时，即使 3D 感知工具给出了正确数据，模型也常得不出正确答案，问题出在“工具读数”到“可用结论”之间没人衔接。作者把通用编程智能体配上命令行感知工具，并让模型把每次读数记进持久场景记忆、用日常空间语言解释，从而补上这一环，显著提升多个基准成绩。
空间推理要求模型判断物体在三维空间中的位置、相互关系及其运动方式。近期的智能体式视觉语言模型（agentic VLM）通过让模型用 3D 感知工具主动收集证据而非一次性作答，在此类任务上取得大幅提升。然而在许多问题上，工具返回了正确证据，模型却仍得不出正确答案：模型与工具之间“原始读数变为可用结论”的缺口未被填补。我们提出 S-Agent 2.0，一种空间推理智能体，为现成的通用编程智能体配备命令行感知工具包，并围绕一个原则设计接口：帮助 VLM 闭合证据回路——获取场景证据、将其存入持久场景记忆，并用平实的空间语言解释每个读数。该设计保留了编程智能体的全部灵活性，且其处理的每个读数都锚定于工具的几何结果。在 MMSI-Bench 和 ReVSI 上，该工具包使强开源模型 Qwen3.8-27B 相比无工具的同一智能体分别提升 13.9 和 20.5 分，使强闭源模型 Gemini-3.8-Flash 分别提升 11.2 和 9.6 分。以开源骨干运行时，S-Agent 2.0 在 MMSI-Bench 上达到 66.5、在 ReVSI 上达到 82.3，超过同一骨干上的专用系统 SpatialClaw（62.6 和 74.5）；搭配 Gemini-3.8-Flash 时达到 74.1 和 90.5，而后者为 67.0 和 79.7。随着骨干模型能力增强，两种工具带来的增益都会缩小，但 SpatialClaw 在 ReVSI 上缩小得多得多（从 +13.2 降至 +2.7），而我们的方法仍保持 +9.6。S-Agent 2.0 还可跨不同的通用编程智能体框架（如 Codex 和 Claude Code）工作。
📄 English Abstract
Spatial reasoning asks a model to work out where objects are in 3D space, how they relate to one another, and how they move. Agentic VLMs have recently brought large improvements on such tasks by letting the model actively gather evidence with 3D perception tools instead of answering in a single pass. Even so, on many questions the tools return the right evidence and the model still does not arrive at the correct answer: the gap between model and tools, where raw readings become usable conclusions, is left unfilled. We present S-Agent 2.0, a spatial reasoning agent that equips an off-the-shelf general coding agent with a command-line perception toolkit, and we design the interface around one principle: help the VLM close the evidence loop: acquire scene evidence, remember it in a persistent scene memory, and interpret each reading in plain spatial language. This design keeps the full flexibility of a coding agent, and every reading it acts on stays grounded in the tools' geometry. On MMSI-Bench and ReVSI, the toolkit improves a strong open-source model, Qwen3.8-27B, by +13.9 and +20.5 points over the same agent without tools, and a strong closed-source model, Gemini-3.8-Flash, by +11.2 and +9.6 points. With the open backbone S-Agent 2.0 reaches 66.5 on MMSI-Bench and 82.3 on ReVSI, ahead of the specialized system SpatialClaw on the same backbone (62.6 and 74.5), and with Gemini-3.8-Flash it reaches 74.1 and 90.5 against its 67.0 and 79.7. Both tool gains shrink as the backbone grows more capable, but SpatialClaw's shrinks far more on ReVSI, from +13.2 to +2.7, while ours stays at +9.6. S-Agent 2.0 also works across different general coding agent harnesses, such as Codex and Claude Code.
6. Sage-A1.5：从长程执行到基于能力组合的科学探索
- 英文标题：Sage-A1.5: From Long-Horizon Execution to Scientific Exploration with Capability Composition
- arXiv：2610.28578
💡 一句话解读：现有AI智能体在长任务中容易走弯路、难协作。Sage-A1.5通过让多个子智能体分工并行、自我检查纠错，并把能力用于开放式科学探索，使复杂任务更可靠、可验证。
本报告介绍 Sage-A1.5，一个基于 A1 构建的 35B 混合专家（Mixture-of-Experts）智能体模型，通过自适应能力组合（adaptive capability composition）提升长程问题求解能力。与 A1 相比，Sage-A1.5 不仅学习解决单个任务，还学习在执行过程中组织工作并修正方法。其三项改进包括：智能体编排（agentic orchestration），通过任务委派和具有隔离上下文的子智能体间并行协作扩展单智能体执行；智能体验证（agentic verification），通过进度检查、证据验证和反思性策略修正增强长程可靠性；开放科学发现（open scientific discovery），将智能体能力从回答预设问题扩展到开展探索性研究。我们通过复杂智能体任务和多学科科学应用展示这些能力，并希望 Sage-A1.5 为社区提供开放科学发现的可及基础。
📄 English Abstract
In this report, we introduce Sage-A1.5, a 35B Mixture-of-Experts agentic model that builds on A1 to advance long-horizon problem solving through adaptive capability composition. Compared with A1, Sage-A1.5 learns not only to solve individual tasks, but also to organize work and revise its approach during execution. Sage-A1.5 highlights three improvements: agentic orchestration, verification, and open scientific discovery. First, agentic orchestration extends single-agent execution through task delegation and parallel collaboration among subagents with isolated contexts, enabling complex tasks to be decomposed and addressed concurrently and supporting efficient execution while reducing interference across concurrent subagents. Second, agentic verification strengthens long-horizon reliability through progress checking, evidence validation, and reflective strategy revision, allowing the agent to recognize unproductive search paths and pursue alternative approaches. Third, open scientific discovery extends agentic capabilities beyond answering predefined questions to conducting exploratory research, connecting scientific inquiry with interactive analysis and inspectable research artifacts. We showcase these capabilities through complicated agentic tasks and multidisciplinary scientific applications. Besides, we hope Sage-A1.5 provides the community with an accessible foundation for open scientific discovery, offering a practical path from long-horizon task execution to collaborative, verifiable, and inspectable scientific exploration.
7. VLA-Rolling：通过自适应动作复用实现更少去噪步骤与更好控制
- 英文标题：VLA-Rolling: Fewer Denoising Steps, Better Control via Adaptive Action Reuse
- arXiv：2610.25043
💡 一句话解读：机器人视觉-语言-动作模型每次规划动作都要从纯噪声反复去噪，但只执行开头一小段，剩下的“动作尾巴”被丢掉。本研究发现问题在于这段尾巴其实保留了有用的去噪进度，于是提出VLA-Rolling：把上一轮未执行的动作尾巴继承下来，逐动作估计置信度，再按置信度分配去噪步数，从而用更少步骤生成更准的动作。
基于流匹配（flow-matching）的视觉-语言-动作（VLA）模型通过迭代去噪高斯噪声来生成一个动作块（action chunk），但通常只执行其中很短的前缀，随后便结合更新后的观测重新规划。在这种滚动时域（receding-horizon）范式下，未执行的部分——即动作尾部（action tail）——会被丢弃，下一个动作块则从新的噪声重新生成。我们表明，这一被丢弃的尾部保留了可迁移的去噪进度：使用动作尾部初始化，可以在更少的去噪步骤下达到甚至超过原生 10 步高斯基线的效果。然而，最优的去噪预算在不同样本间差异显著，说明尾部复用应根据更新后的环境进行自适应调整。基于这一观察，我们提出 VLA-Rolling，一个轻量级推理框架，通过三个阶段复用动作尾部：动作尾部继承、逐动作置信度估计，以及通过重新加噪与去噪预算分配实现的置信度引导精化，同时底层 VLA 保持冻结。在 LIBERO、MetaWorld、LIBERO-Plus 和 RoboTwin 2.0 上对 π0.5、SmolVLA、X-VLA 和 MolmoAct2 的实验中，VLA-Rolling 在全部 16 个模型—基准组合中降低了动作块推理开销，最高实现 1.48 倍加速，同时在 14 个组合中提升成功率，最高提升 15.08 个百分点。这些结果表明，动作尾部并非只是被丢弃的预测，而是可复用的中间状态，能够实现更高效、更准确的 VLA 推理，并支持以更少的去噪步骤生成更好的动作。
📄 English Abstract
Flow-matching vision-language-action (VLA) models generate an action chunk by iteratively denoising Gaussian noise, but typically execute only a short prefix before replanning with updated observations. Under this receding-horizon paradigm, the unexecuted remainder—the action tail—is discarded, and the next chunk is regenerated from fresh noise. We show that this discarded tail retains transferable denoising progress: action-tail initialization can match or outperform a native 10-step Gaussian baseline using fewer denoising steps. However, the optimal denoising budget varies substantially across samples, indicating that tail reuse should be adapted to the updated environment. Based on this observation, we introduce VLA-Rolling, a lightweight inference framework that reuses action tails through three stages: action-tail inheritance, per-action confidence estimation, and confidence-guided refinement through renoising and denoising budget allocation, while the underlying VLA remains frozen. Across $\pi_{0.5}$, SmolVLA, X-VLA, and MolmoAct2 on LIBERO, MetaWorld, LIBERO-Plus, and RoboTwin 2.0, VLA-Rolling reduces action-chunk inference cost in all 16 model–benchmark pairs, achieving up to a $1.48\times$ speedup, while improving success rate in 14 pairs by up to $15.08$ percentage points. These results show that action tails are not merely discarded predictions, but reusable intermediate states that enable more efficient and accurate VLA inference. These results demonstrate that the action tail can support the generation of better actions with fewer denoising steps. Our project is available at https://chenbao123ww.github.io/VLA-Rolling/.
8. CoEvo-RSI：当执行框架演化时，哪些模型应当跟进？
- 英文标题：CoEvo-RSI: When Harnesses Evolve, Which Models Should Follow?
- arXiv：2610.25078
💡 一句话解读：这篇研究要解决的是AI智能体“大脑”（模型）和“身体”（执行框架）不匹配的问题：框架不断升级，模型却跟不上，或模型有能力却被框架限制。方法是让一个元智能体借助共享的历史存档和执行反馈，双向继承历史设计，同时协调框架演进与模型选择、合并和适配，使两者对齐。
模型与执行框架的协同演化已成为递归自我改进（RSI）智能体的一种重要范式，二者共同演化其执行脚手架与内在能力。我们研究了该范式中长期存在却常被忽视的“大脑—身体失配”问题：不断演化的执行框架（身体）要求模型（大脑）具备其尚未学会的行为，或限制其已具备的能力。我们提出 CoEvo-RSI，一个通过智能体驱动的双向历史继承来推进模型—框架协同演化的 RSI 框架。借助共享的演化存档与执行反馈，一个元智能体协调框架的历史设计开发，并进行模型检查点选择或合并，以及在所选框架下的后续适配，从而使继承的能力与不断变化的执行需求对齐，以减少大脑—身体失配。仅使用对自身拒绝采样轨迹的监督微调（SFT），CoEvo-RSI 就使 Qwen3.5-9B 在多种垂直应用中与更大模型相竞争，包括多智能体故障归因、角色感知的文本到 SQL 以及智能家居指令遵循。它在 TraceElephant 上以 +14.16 的智能体准确率超越 DeepSeek-V4-Pro，在 HomeBench VM 上以 +4.90 的成功率超越 GLM-5.3，凸显了大脑—身体对齐的必要性以及双向历史继承推进智能体协同演化的潜力。
📄 English Abstract
Model–harness co-evolution has emerged as a prominent paradigm for recursive self-improving (RSI) agents, jointly evolving their execution scaffolds and intrinsic capabilities. We study the persistent yet often overlooked brain–body mismatch problem in this paradigm, where an evolving harness (body) demands behaviors its model (brain) has not learned or restricts capabilities it already possesses. We introduce CoEvo-RSI, a RSI framework that advances model–harness co-evolution through agent-driven, bidirectional historical inheritance. Drawing on a shared evolutionary archive and execution feedback, a meta-agent coordinates harness development from historical designs with model checkpoint selection or merging and subsequent adaptation under the selected harness, aligning inherited capabilities with evolving execution requirements to reduce brain–body mismatch. Using only supervised fine-tuning (SFT) on its own rejection-sampled trajectories, CoEvo-RSI makes Qwen3.5-9B competitive with larger models across diverse vertical applications, including multi-agent failure attribution, role-aware text-to-SQL, and smart-home instruction following. It surpasses DeepSeek-V4-Pro on TraceElephant (+14.16 agent accuracy) and GLM-5.3 on HomeBench VM (+4.90 success rate), underscoring the need for brain–body alignment and the potential of bidirectional historical inheritance to advance agent co-evolution.
9. 通过点轨迹预测学习四维世界
- 英文标题：Learning the 4D World through Point Trajectory Forecasting
- arXiv：2610.28594
💡 一句话解读：多视角视频预测常因各相机画面不一致而失真。作者提出视频点模型（VPM），在预测未来视频的同时追踪场景中固定点的三维运动轨迹，用轨迹约束让各视角描述同一三维运动；还构建了配套数据集和评测基准，并扩展到机器人控制，提升了视频质量、运动准确性和操作成功率。
可靠的多视角预测要求不同相机的视频描述同一个不断演化的三维场景。为此，我们提出视频点模型（Video-Point Model, VPM），这是一种以多视角视频历史和语言指令为条件的世界模型，可联合预测任意视角的未来视频以及持续场景点的三维轨迹。我们扩展了一个预训练世界模型，加入相机条件化的多视角处理以融合跨相机的互补观测，并增加轨迹头来预测物理点如何随时间运动。每条轨迹在所有视角中跟踪同一个点，因此将这些轨迹与视频生成一起监督，就把每个相机的表观变化锚定到对三维运动的共享描述上。为支持训练与评估，我们构建了 VPM-Data（将同步多视角视频与持续点轨迹配对的仿真语料）和 VP-WorldBench（覆盖仿真与真实环境的预测基准）。我们进一步将 VPM 扩展到机器人控制，把机器人动作作为生成器中的一种 token 模态加入；动作 token 与点监督共享主干，并与未来视频联合去噪，从而使策略建立在同一三维感知表征之上。在 VP-WorldBench 和现有 4D 生成基准上，点监督相比仅视频的对照模型提升了视频保真度和三维运动精度；在仿真和真实机器人实验中，它还提高了启用动作的模型的操作成功率。
📄 English Abstract
Reliable multi-view forecasting requires videos from different cameras to describe the same evolving 3D scene. To this end, we introduce Video-Point Model (VPM), a world model that conditions on multi-view video histories and a language instruction to jointly forecast future videos from arbitrary viewpoints and the 3D trajectories of persistent scene points. We extend a pretrained world model with camera-conditioned multi-view processing that integrates complementary observations across cameras, and add a trajectory head that predicts how physical points move through time. Each trajectory tracks a single point across all views, and supervising these trajectories alongside video generation therefore anchors appearance changes in every camera to a shared description of 3D motion. To support training and evaluation, we build VPM-Data, a simulation corpus pairing synchronized multi-view videos with persistent point trajectories, and VP-WorldBench, a forecasting benchmark spanning simulated and real environments. We further extend VPM to robot control by adding robot actions as a token modality in the generator. Action tokens share the point-supervised backbone and are denoised jointly with future videos, thereby grounding the policy in the same 3D-aware representation. On VP-WorldBench and existing 4D generation benchmarks, point supervision improves both video fidelity and 3D motion accuracy over a video-only counterpart, and in simulated and real-robot experiments, it also raises the manipulation success of the action-enabled model.
10. SolarRenderer：少步生成式参考渲染
- 英文标题：SolarRenderer: Few-Step Generative Reference Rendering
- arXiv：2610.28630
💡 一句话解读：游戏引擎等工具能给出场景的布局、动作和镜头，但要把这些粗略结构变成逼真视频，还需要渲染器补全外观。现有渲染器绑定单一引擎、需要几十步去噪或昂贵蒸馏，且训练条件与目标视频不对齐。SolarRenderer 用两个轻量 LoRA 改造冻结的 33B 视频生成模型，把结构视频直接送入原生参考通路、仅用纯文本描述，从而用单一控制 LoRA 适配任意结构表示；配合四步 LoRA 和分布匹配，四步即可渲染，比 50 步快 12.3 倍，并支持实时流式 768p 渲染。
游戏引擎和编码智能体可以指定场景的布局、运动和相机，但要将这种粗粒度结构转化为逼真视频，需要一个既能遵循该结构、又能补全其未定外观的渲染器。现有渲染器受限于单一引擎的表示方式，需要数十步去噪或代价高昂的自有蒸馏，并且从与目标视频不对齐的条件中学习。我们提出 SolarRenderer，一种结构条件视频渲染器，通过两个轻量 LoRA 适配冻结的 MiniMax-H3 33B 主干。结构视频在其所描述帧的位置进入主干的原生参考通路，无需类型标识符、仅以纯文本描述，从而使单个控制 LoRA 能遵循任意结构表示。从预训练的四步 LoRA 出发，结构引导的分布匹配可在四步内完成渲染，比 50 步采样快 12.3 倍，同时保留所学控制。一种块因果变体将相同设计扩展到流式场景，可实时渲染 768p 视频。为训练模型，我们构建了来自游戏引擎、模拟器和 YouTube 视频的 136K 帧对齐片段语料库。在分布内测试集和包含 16 种结构表示的泛化基准上，实验表明 SolarRenderer 在两者上均达到最先进结果，并可泛化到 11 种未见过的表示，包括姿态骨架、手绘和点云。
📄 English Abstract
Game engines and coding agents can specify the layout, motion, and camera of a scene, yet turning such coarse structure into realistic video requires a renderer that follows the structure and completes the appearance it leaves open. Existing renderers are tied to the representation of one engine, need tens of denoising steps or a costly distillation of their own, and learn from conditions that are not aligned with their target videos. We present SolarRenderer, a structure-conditioned video renderer that adapts a frozen MiniMax-H3 33B backbone with two lightweight LoRAs. Structural videos enter the native reference pathway of the backbone at the positions of the frames they describe, without a type identifier and described only in plain text, so that a single control LoRA follows arbitrary structural representations. Starting from a pretrained four-step LoRA, structure-guided distribution matching renders in four steps, 12.3 times faster than 50-step sampling, while preserving the learned control. A block-causal variant extends the same design to streaming and renders 768p video in real time. To train the model, we build a frame-aligned corpus of 136K clips from game engines, simulators, and YouTube videos. Experiments on an in-distribution test set and a generalization benchmark of 16 structural representations show that SolarRenderer achieves state-of-the-art results across both, and generalizes to 11 unseen representations, including pose skeletons, drawings, and point clouds.
11. SWIFT：基于目标外推的自引导弱到强扩散模型
- 英文标题：SWIFT: Self-guided Weak-to-strong dIFfusion models via Target extrapolation
- arXiv：2610.25844
💡 一句话解读：训练高质量扩散模型很贵，推理时加引导更贵。本文提出Swift：用训练早期的弱模型作为“向导”，把去噪目标往远离弱模型预测的方向外推，放大欠拟合部分，从而在不增加推理成本的情况下训练出更强模型。
训练高质量扩散模型计算代价高昂，且推理时引导常增加采样成本。本文提出Swift，一种简单有效的自引导训练方法，利用冻结的弱去噪器改进更强模型而不增加推理开销。对每个噪声输入，Swift将干净回归目标从弱去噪器的预测中外推，放大仍欠拟合的分量。弱引导可以是早期训练检查点，主模型训练时无需反向传播，之后即被丢弃。所得模型保留原始采样器，支持条件和无条件生成，无需标签丢弃或无条件下分支。我们通过RKHS中有限时间群体梯度流的谱分析解释该机制，并证明在早停弱引导下，非空的外推强度范围能比匹配的标准训练获得严格更低的后验均值误差，尽管回归目标偏离了后验均值。在CIFAR-10和ImageNet-256上的实验表明，该方法在不同模型规模和表示下均有改进。在一种ImageNet-256配置中，Swift用约一半的总训练计算量（包括引导准备和训练开销）超越了基线的最佳FID。在RAEv2上，无推理时引导达到FID 1.15，有引导达到0.99。这些结果确立了目标外推作为弱到强扩散训练的有效方法，并实现高效的单模型推理。
📄 English Abstract
Training high-quality diffusion models is computationally expensive, and inference-time guidance often adds further sampling cost. In this work, we introduce Swift, a simple yet effective self-guided training method that uses a frozen weak denoiser to improve a stronger model without adding inference overhead. For each noisy input, Swift extrapolates the clean regression target away from the weak denoiser's prediction, amplifying components that remain underfit. The weak guide can be an early training checkpoint, requires no backpropagation during main-model training, and is discarded afterward. The resulting model retains the original sampler and supports both conditional and unconditional generation without label dropout or an unconditional branch. We explain this mechanism through a spectral analysis of finite-time population gradient flow in an RKHS. With an early-stopped weak guide, we prove that a nonempty range of extrapolation strengths yields strictly lower posterior-mean error than matched standard training, despite shifting the regression target away from the posterior mean. Experiments on CIFAR-10 and ImageNet-256 demonstrate improvements across model scales and representations. In one ImageNet-256 configuration, Swift surpasses the baseline's best FID using roughly half the total training compute, including guide preparation and training overhead. On RAEv2, it achieves FID 1.15 without inference-time guidance and 0.99 with guidance. These results establish target extrapolation as an effective approach to weak-to-strong diffusion training with efficient single-model inference.
12. 扩散模型的校正训练
- 英文标题：Correction Training for Diffusion Models
- arXiv：2610.14136
💡 一句话解读：扩散模型生成图像时，每一步都基于自己上一步不完美的结果继续预测，误差会越积越多，导致画质变差，通常得靠额外的引导技巧（CFG）来补救。作者为此训练了一个“校正器”：它专门学习如何把模型自己的预测修正得更接近真实图像，在采样每一步反复精修，从而提升画质、保留多样性，还能把校正能力蒸馏回原模型，不增加推理开销。
扩散模型中的迭代采样会因暴露偏差（exposure bias）导致误差累积：去噪器训练时输入的是真实图像与噪声的插值，而推理时输入却由自身前几步不完美的预测构成，这是训练时从未见过的情形，可能是扩散模型在缺乏 CFG 等额外辅助时往往表现不佳的原因之一。我们为扩散模型引入一个校正步骤，以缩小这种训练与推理之间的差距。校正器以基础模型的预测为输入，并针对相同的地面真值目标进行训练，从而使校正成为一个显式的学习任务，其训练输入由基础模型提供。在采样过程中，我们在每个去噪步骤迭代地精修去噪器的干净样本预测，从而能在中间去噪步骤校正预测，防止早期伪影残留到最终样本中。尽管校正器仅在基础去噪器的预测上训练，推理时可对其自身输出反复应用以获得更好结果。这一简单框架在显著提升样本质量的同时基本保持了多样性。我们在 DiT、SiT 和 JiT 上评估了该方法，在无引导生成尤其是无条件生成（此前难以在不使用引导的情况下生成高质量样本）上大幅优于先前报道的 FID 分数，性能可与无分类器引导（CFG）相媲美；将校正与 CFG 结合还能进一步提升多样性并减少 CFG 典型的过饱和现象。最后我们表明，校正带来的额外推理开销可通过将精修后的预测蒸馏回基础扩散模型来摊销，从而在不增加推理时开销的情况下保留质量收益。
📄 English Abstract
Iterative sampling in diffusion models can lead to error accumulation due to exposure bias. The denoiser is trained on real images interpolated with noise, but at inference its inputs are constructed from its own imperfect predictions from the earlier steps, something it has never seen at training time. This might be one reason diffusion models often struggle without additional help, such as CFG. We introduce a correction step for diffusion models which attempts to reduce this training-inference gap. The corrector takes the base model's prediction as input and is trained against the same ground-truth target. This makes correction an explicit learning task, with training inputs provided by the base model. During sampling, at each denoising step, we iteratively refine the denoiser's clean-sample prediction. This allows us to correct predictions at intermediate denoising steps, preventing early artifacts from persisting into the final sample. Although the corrector is trained only on predictions from the base denoiser, it can be applied repeatedly to its own outputs at inference for better results. This simple framework leads to substantial improvement in sample quality while largely preserving diversity. We evaluate our method across DiT, SiT, and JiT. Our method substantially improves upon previously reported FID scores for unguided generation and, notably, unconditional generation, where producing high-quality samples without guidance has been particularly challenging. Our method achieves performance competitive with classifier-free guidance (CFG), and combining correction with CFG further improves diversity and reduces oversaturation typical of CFG. Finally, we show that the additional inference cost of correction can be amortized by distilling the refined predictions back into the base diffusion model, retaining the quality gains without additional inference-time overhead.
13. MMEnterprise-Bench：在持续演进的多模态记录上评估企业智能体
- 英文标题：MMEnterprise-Bench: Evaluating Enterprise Agents over Evolving Multimodal Records
- arXiv：2610.27911
💡 一句话解读：企业里同一件事的信息会在文档、聊天和图表中不断被修改，留下多个版本，AI助手要回答问题时容易搞混或答错。本研究构建了一个包含1613个问题的测试集，专门考察AI能否找齐证据、处理跨来源和时间冲突、并识别证据不足的情况；结果发现现有最强模型在这些方面仍明显吃力。
企业智能体需要基于随团队协作不断演进的工作场所记录来回答问题。随着信息在文档、聊天和图表中被共享、更正和修订，同一信息会在工作区中积累出多个版本。因此，可靠的问答要求智能体在充分证据上进行推理、解决跨来源与时间上的冲突，并识别出可用证据不足以确定答案的情况。为系统评估这些能力，我们提出了 MMEnterprise-Bench，这是一个包含 1,613 个问题的基准，问题基于模拟协作项目中不断演进的多模态记录，覆盖证据推理（Evidence Reasoning）、冲突消解（Conflict Resolution）和不可回答性（Unanswerability）三类。我们评估了 10 个专有和开源多模态大语言模型，发现即使是最强的模型，在证据包含跨来源或时间冲突、或跨越多张图像时也表现吃力。此外，增加检索预算或采用更大的检索器能提升证据召回率，但并未稳定转化为更高的答案准确率。这些发现凸显了在演进式工作场所记录上进行推理的持续挑战，并确立了 MMEnterprise-Bench 作为开发更可靠企业智能体的测试平台。代码与数据见 https://mlvlab.github.io/MMEnterprise-Bench/。
📄 English Abstract
Enterprise agents answer questions over workplace records that evolve through team collaboration. As information is shared, corrected, and revised across documents, chats, and figures, multiple versions of the same information accumulate in the workspace. Reliable question answering therefore requires agents to reason over sufficient evidence, resolve cross-source and temporal conflicts, and recognize when the available evidence is insufficient to establish an answer. To systematically evaluate these capabilities, we introduce MMEnterprise-Bench, a benchmark comprising 1,613 questions grounded in evolving multimodal records from simulated collaboration projects, covering three categories: Evidence Reasoning, Conflict Resolution, and Unanswerability. We evaluate 10 proprietary and open-source multimodal LLMs and find that even the strongest models struggle when evidence contains cross-source or temporal conflicts or spans multiple images. Moreover, increasing the retrieval budget or employing a larger retriever improves evidence recall but does not consistently translate into higher answer accuracy. These findings highlight the persistent challenges of reasoning over evolving workplace records and establish MMEnterprise-Bench as a testbed for developing more reliable enterprise agents. Code and data are available at https://mlvlab.github.io/MMEnterprise-Bench/.
14. LLM2R：从大语言模型到3D重装配器
- 英文标题：LLM2R: From Large Language Models to 3D Reassemblers
- arXiv：2610.18604
💡 一句话解读：现有方法在零件很多时难以准确拼装3D模型。作者把拼装任务变成让大语言模型按顺序读入各零件信息，再加一组“总结符号”汇总全局信息，一次前向就预测所有零件的位置和朝向，零件越多优势越明显。
3D零件装配旨在通过为每个零件预测刚体SE(3)变换，从一组未装配零件重建完整形状。由于零件的位置很少能由其自身几何决定，该任务需要对整个零件集合进行推理，而依赖成对关系或迭代生成与验证的现有方法难以扩展到零件数量较多的物体。我们提出LLM2R，将零件装配转化为使用预训练仅解码器大语言模型（LLM）的序列建模：Point-BERT编码器和Q-Former将每个零件压缩为一块几何token，所有零件的块被拼接，经低秩适配（LoRA）调整的因果LLM在一次前向传播中处理该序列。然而，因果注意力使零件的隐藏状态只能观察到其之前的零件。因此，LLM2R在最后一个零件之后附加一块可学习的总结token：这些token位于每个零件之后，其状态以恒定token开销聚合整个装配，而零件感知的姿态头在预测每个零件的旋转和平移之前，将该零件的状态与这些总结token融合。由此，单一架构即可处理语义和几何装配，并在一次非自回归前向中预测所有姿态。LLM2R在全部三个PartNet类别上取得最高的零件准确率和成功率，在PartNeXt上取得最佳的旋转误差、平移误差和零件准确率，并在Breaking Bad上取得最低的平移误差和最快的推理速度，仅有一种 fracture 专用方法更准确。其优势随零件数量增加而扩大：在100个零件时，它在Breaking Bad上比GARF的零件准确率高33.5个百分点，在PartNet上比Assembler高29.1个百分点，并能零样本迁移到未见类别，零件准确率比三个基线中最强者高16个百分点以上。
📄 English Abstract
3D part assembly aims to reconstruct a complete shape from a set of unassembled parts by predicting a rigid SE(3) transformation for each part. Because the placement of a part is rarely determined by its own geometry, the task demands reasoning over the entire part set, and existing methods, which rely on pairwise relations or iterative generation and verification, scale poorly to objects with many parts. We present LLM2R, which casts part assembly as sequence modeling with a pretrained decoder-only large language model (LLM): a Point-BERT encoder and a Q-Former compress each part into a block of geometric tokens, the blocks of all parts are concatenated, and a causal LLM adapted with Low-Rank Adaptation (LoRA) processes the sequence in one forward pass. Causal attention, however, lets the hidden states of a part observe only the parts that precede it. LLM2R therefore appends a block of learnable summary tokens after the last part: these tokens follow every part, so their states aggregate the whole assembly at a constant token cost, and a part-aware pose head fuses each part's states with them before predicting its rotation and translation. One architecture thus handles semantic and geometric assembly and predicts all poses in a single non-autoregressive pass. LLM2R achieves the highest part accuracy and success rate on all three PartNet categories, the best rotation error, translation error, and part accuracy on PartNeXt, and the lowest translation error and fastest inference on Breaking Bad, where only a fracture-specialized method is more accurate. Its advantage grows with the number of parts: at 100 parts it exceeds GARF by 33.5 points in part accuracy on Breaking Bad and Assembler by 29.1 points on PartNet, and it transfers zero-shot to unseen categories with more than 16 points higher part accuracy than the strongest of three baselines.
15. PhysSkin++：基于自监督神经蒙皮的实时可泛化物理动画
- 英文标题：PhysSkin++: Real-Time and Generalizable Physics-Based Animation via Self-Supervised Neural Skinning
- arXiv：2610.26067
💡 一句话解读：这篇研究要解决的是：如何让实时物理动画能快速适配各种不同形状的3D模型，而不需要为每个模型单独训练网络。他们提出PhysSkin++，用自监督学习从静态几何中自动学习变形规律，无需模拟数据，就能实现跨形状的实时动画，并支持网格、点云等多种表示。
在多样化的3D形状和空间离散化之间泛化实时基于物理的动画仍然具有挑战性。本文提出PhysSkin++，一个基于物理信息的框架，它从静态几何中学习连续的神经蒙皮场（neural skinning fields），以构建紧凑的变形子空间。具体而言，我们提出了一种神经蒙皮场自编码器（Neural Skinning Fields Autoencoder），无需针对每个对象进行网络训练即可预测依赖于形状的变形子空间。为了在没有模拟轨迹的情况下学习这些子空间，我们开发了一种基于物理信息的自监督学习策略，该策略结合了蒙皮场归一化与冲突感知梯度校正，从而在物理和正交约束下稳定联合优化。学习到的表示支持带有用户控制和接触交互的高效降阶动力学，并可直接将动画绑定到网格、点云和3D高斯。对于预设的骨骼动画，我们进一步引入了一种自监督互补性目标，以学习互补的弹性响应，而无需在运行时强制互补性约束。实验证明了学习到的蒙皮场的跨形状泛化能力、实时降阶模拟，以及在可控、碰撞感知和骨骼互补动画中的应用。
📄 English Abstract
Generalizing real-time physics-based animation across diverse 3D shapes and spatial discretizations remains challenging. In this paper, we introduce PhysSkin++, a physics-informed framework that learns continuous neural skinning fields from static geometries to construct compact deformation subspaces. Specifically, we propose a Neural Skinning Fields Autoencoder that predicts shape-dependent deformation subspaces without per-object network training. To learn these subspaces without simulation trajectories, we develop a Physics-Informed Self-Supervised Learning strategy that combines skinning-field normalization with conflict-aware gradient correction, stabilizing joint optimization under physical and orthogonality constraints. The learned representation supports efficient reduced dynamics with user control and contact interactions, together with direct animation binding to meshes, point clouds, and 3D Gaussians. For prescribed rig animation, we further introduce a self-supervised complementarity objective to learn complementary elastic responses without enforcing complementarity constraints at runtime. Experiments demonstrate cross-shape generalization of the learned skinning fields, real-time reduced simulation, and applications to controllable, collision-aware, and rig-complementary animation.