Daily Papers · 2026-09-29
📄 论文日报 | 来源：Hugging Face Daily Papers
🗓 Episode：2026.09.28 | 层融合正则缩小重建生成差；血缘数据流加速保序容错
1. FuseReg：正则化层融合缓解表征自编码器中的重建-生成差距
- 英文标题：FuseReg: Regularizing Layer Fusion Mitigates the Reconstruction-Generation Gap in Representation Autoencoders
- arXiv：2609.31620
💡 一句话解读：表征自编码器（RAE）用预训练视觉编码器的特征做图像重建和生成，但选哪些层做共享隐空间是个两难：浅层重建细节好，深层生成指标好。固定选层会顾此失彼。FuseReg 改为在随机层子集上训练，让同一个解码器能灵活适配任意层组合，从而同时提升重建和生成质量。
表征自编码器（RAEs）复用预训练视觉编码器的特征作为重建和扩散隐变量，将强视觉表征融入图像生成。然而，RAEs 仍需决定哪些编码器层构成生成器和像素解码器共享的隐空间。这一选择涉及权衡：较浅层往往能更好地保留精细像素细节，而较深层往往能产生更好的生成指标。因此，固定的启发式层融合会耦合这两个受益于不同信息的阶段。我们提出 FuseReg，用对编码器层随机子集的训练替代启发式特征选择。我们从理论上分析了其内在机制：子集采样在期望上保留全层隐变量均值，同时显式惩罚对跨层不一致的敏感性。在 ImageNet-256 上使用 DINOv3-L 时，单个 FuseReg 解码器无需重新训练即可从全层、稀疏和单层融合中重建，其 PSNR 高于专用于固定融合的解码器。这种灵活性也有益于生成：仅替换解码器即可在保持 RAEv2 DiT-XL 生成器不变的情况下，将无引导 gFID 降低 27%。将 FuseReg 同时应用于两个阶段，在 DiT-Base 上还能将无引导 gFID 降低 29%。重建和生成的收益也扩展到其他编码器家族。FuseReg 在不增加训练成本或改变架构的情况下缩小了重建-生成差距。
📄 English Abstract
Representation autoencoders (RAEs) reuse features from a pretrained visual encoder as reconstruction and diffusion latents, integrating strong visual representations into image generation. However, RAEs still need to decide which encoder layers form the shared latent space for the generator and pixel decoder. This choice involves a trade-off: shallower layers tend to preserve fine pixel details better, while deeper layers tend to yield better generation metrics. A fixed heuristic layer fusion therefore couples two stages that benefit from different information. We introduce FuseReg, which replaces heuristic feature selection with training over random subsets of encoder layers. We theoretically analyze the underlying mechanism: subset sampling preserves the full-layer latent mean in expectation while explicitly penalizing sensitivity to cross-layer disagreement. On ImageNet-256 with DINOv3-L, a single FuseReg decoder reconstructs from full, sparse, and single-layer fusions without retraining, achieving higher PSNR than decoders specialized to fixed fusions. This flexibility also benefits generation: decoder replacement alone reduces unguided gFID by 27% with an unchanged RAEv2 DiT-XL generator. Applying FuseReg to both stages also reduces unguided gFID by 29% on DiT-Base. The reconstruction and generation benefits also extend to other encoder families. FuseReg narrows the reconstruction-generation gap without additional training cost or architectural changes.
2. RayOrch：面向基础模型数据准备的受血缘控制多粒度数据流编程与执行
- 英文标题：RayOrch: Programming and Executing Lineage-Controlled Multi-Grain Dataflows for Foundation-Model Data Preparation
- arXiv：2609.18703
💡 一句话解读：给大模型准备训练数据时，一条原始文档或视频往往要拆成很多小片段，数量还不固定，处理起来容易乱。RayOrch 让程序明确声明这种拆分和合并关系，由系统自动记录每个片段的归属、顺序和完成状态，从而高效调度 GPU 并行处理，速度比现有方案快很多。
为基础模型准备高质量训练数据，需要可扩展的流水线，将异构文档和视频转换为结构化记录。这类流水线会把每个父项扩展为一个有序且依赖输入的序列，其子项数量可能呈长尾分布。GPU 应当跨父项批量处理子项，同时保留父子关系、子项顺序、完成状态和结果路由。现有系统要么把并行性隐藏在粗粒度作业之后，要么暴露扁平记录，迫使应用自行管理血缘关系与重新分组。我们提出 RayOrch，一种编程模型和分布式执行引擎，在整个执行过程中保留父子关系。程序声明有序的可变基数扩展及与之匹配的聚合操作，编译器验证每一对扩展与聚合，运行时则记录子项成员关系、直接父项、不可变序号和终止状态。每次调用的 FIFO 就绪队列跨父项批量处理就绪子项；聚合操作依据声明的成员关系和序号重建结果，而非依赖批次边界或完成顺序。父项一旦所有必需子项都进入终止状态即可推进。带类型的父项作用域失败会抑制失败父项尚未派发的兄弟项，同时允许无关父项继续执行。在 NVIDIA H20 GPU 上，RayOrch 将 MinerU 从 4 块 GPU 扩展到 64 块 GPU 时实现了 15.14 倍加速，将视频流水线从 8 块扩展到 64 块 GPU 时实现了 7.82 倍加速。在 MinerU 上，其端到端时间比 Ray Data 减少 13.1%，比 Daft 减少 29.0%；在 Docling 上比 Ray Data 减少 16.0%。代码见 https://github.com/OpenDCAI/RayOrch 。
📄 English Abstract
Preparing high quality training data for foundation models requires scalable pipelines that transform heterogeneous documents and videos into structured records. Such pipelines expand each parent item into an ordered and input dependent sequence of children, whose counts may be long tailed. GPUs should batch children across parents while preserving parent relationships, child order, completion status, and result routing. Existing systems either hide parallelism behind coarse grained jobs or expose flat records that force applications to manage lineage and regrouping. We present RayOrch, a programming model and distributed execution engine that preserves parent child relations throughout execution. Programs declare ordered variable cardinality expansions and matching gathers. The compiler validates each pair, while the runtime records child membership, immediate parents, immutable ordinals, and terminal states. Per Call FIFO Ready Queues batch ready children across parents. Gathers reconstruct results from declared membership and ordinals rather than batch boundaries or completion order. Parents can advance as soon as all required children become terminal. Typed parent scoped failures suppress undispatched siblings of the failed parent while allowing unrelated parents to continue. On NVIDIA H20 GPUs, RayOrch achieves 15.14 times speedup when scaling MinerU from 4 to 64 GPUs and 7.82 times speedup when scaling a video pipeline from 8 to 64 GPUs. It reduces end to end time by 13.1 percent versus Ray Data and 29.0 percent versus Daft on MinerU, and by 16.0 percent versus Ray Data on Docling. Code available at https://github.com/OpenDCAI/RayOrch .
3. 对数线性复杂度的块稀疏注意力
- 英文标题：Block Sparse Attention with Log-Linear Complexity
- arXiv：2609.31093
💡 一句话解读：长文本处理时，注意力机制计算量随长度平方增长，太慢。块稀疏注意力只算部分块，但选块本身还是平方复杂度。本文提出PISA，用金字塔式逐层筛选，像从粗到细找目标，把选块复杂度降到N log N，并写了高效GPU内核，在常识推理和检索任务上表现不错。
将语言模型扩展到长上下文受限于自注意力的二次计算成本。块稀疏注意力（block sparse attention）提供了一种高效替代方案，但选择保留哪些块仍是瓶颈。传统块选择需要对所有查询-块对进行评分，因此相对于序列长度仍是二次复杂度。为解决此问题，我们提出PISA，一种采用金字塔Top-K选择策略的块稀疏注意力机制。其核心思想是在不同层级逐步缩小候选范围，从而更高效地找到最相关的键。具体而言，我们构建从粗到细的键层级结构，并从最粗层级开始选择。在每个层级，对有限候选集应用LogSumExp评分，选出进入下一更细层级的候选，直至最细层级。通过池化，我们构建O(log N)层键，整体复杂度为O(N log N)，其中N为序列长度。我们为训练和推理开发了硬件感知的Triton内核，融合了层级路由和LogSumExp评分，无需具体化查询-键评分矩阵。我们在语言建模任务上评估了该方法。与基线相比，我们的方法在常识推理等基准上性能相当，同时在检索任务上取得更好结果。
📄 English Abstract
Scaling language models to long contexts is limited by the quadratic cost of self-attention. Block sparse attention offers an efficient alternative, but selecting the retained blocks remains a bottleneck. Conventional block selection requires scoring all query-block pairs and therefore remains quadratic in sequence length. To address this issue, we propose PISA, a block-sparse attention mechanism that employs a pyramid Top-$K$ selection strategy. The main idea is to gradually narrow down the candidates across different levels, making it more efficient to find the most relevant keys. Specifically, we construct a coarse-to-fine hierarchy of keys and perform selection from the coarsest level. At each level, LogSumExp scoring is applied to a bounded candidate set to select candidates for the next finer level, continuing until the finest level is reached. Through pooling, we construct $O(\log N)$ levels of keys, yielding an overall complexity of $O(N\log N)$, where $N$ denotes the sequence length. We develop hardware-aware Triton kernels for both training and inference, fusing hierarchical routing and LogSumExp scoring without materializing the query-key score matrix. We further evaluate our method on language modeling tasks. Compared with the baseline, our method achieves comparable performance on benchmarks such as commonsense reasoning while delivering better results on retrieval tasks.
4. InternW0-Δ：一个连接预测动态与动作、基于20K+小时开放数据的世界动作模型
- 英文标题：InternW0-$\Delta$: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data
- arXiv：2609.31394
💡 一句话解读：这篇研究要解决的是：如何让机器人既看懂画面变化、又知道该怎么动手。现有方法各管一摊，难以统一。作者提出一个叫 InternW0-Δ 的模型，把视频预测、场景理解、三维几何和动作生成拼进一个框架，还让模型在训练时学会预判画面变化，但实际用时不必先画出未来视频，从而更快更准地控制机器人。
世界动作模型（WAM）将视觉动态与动作生成联合建模，用于通用机器人操作。其核心挑战在于把大规模预训练模型中的先验——包括视觉动态、场景语义、几何与运动——整合进一个统一框架来生成机器人动作。我们提出 InternW0-Δ，一个在异构语料上预训练的统一 WAM，在仿真基准和真实机器人平台上均优于先前方法。InternW0-Δ 在混合 Transformer（MoT）框架内融合了预训练视觉动态、场景级语义、4D 几何与运动先验以及动作生成：预训练视频专家与动作专家在冻结 VLM 的语义引导下交互，预训练 4D 基础模型则通过仅训练时蒸馏注入几何与运动先验。我们还提出 Causal Imprint，它从仅训练时的未来监督中学习与未来相关的场景变化，并直接向动作专家提供预测性表征，推理时无需生成未来视频。为进行大规模联合训练，我们构建了一个异构语料库，涵盖机器人演示、UMI 数据、第一人称人类演示和 Ego2Robot 数据，并在统一的状态-动作表示下进行整理与对齐。该语料库包含超过 2 万小时的处理后训练数据，据我们所知是同类中最大的开源语料库。我们在该语料库上预训练 InternW0-Δ，并在仿真基准和真实机器人平台上展示了强劲性能。我们将在许可允许的范围内开源训练代码、模型权重、基础设施、数据处理流程及处理后数据。项目页面：https://internrobotics.github.io/InternW0-Delta/
📄 English Abstract
World Action Models (WAMs) jointly model visual dynamics and action generation for generalist robot manipulation. A central challenge is to integrate priors from large-scale pretrained models—including visual dynamics, scene semantics, geometry, and motion—into a unified framework for robot action generation. We introduce InternW0-$\Delta$, a unified WAM pretrained on a heterogeneous corpus that outperforms prior methods across simulation benchmarks and real-robot platforms. InternW0-$\Delta$ combines pretrained visual dynamics, scene-level semantics, 4D geometric and motion priors, and action generation within a Mixture-of-Transformers (MoT) framework. A pretrained video expert and an action expert interact under semantic guidance from a frozen VLM, while a pretrained 4D foundation model injects geometric and motion priors through training-only distillation. We further introduce Causal Imprint, which learns future-relevant scene changes from training-only future supervision and provides predictive representations directly to the action expert without future-video rollout at inference. For large-scale joint training, we construct a heterogeneous corpus of robot demonstrations, UMI data, egocentric human demonstrations, and Ego2Robot data, curated and aligned under a common state-action representation. The resulting corpus contains over 20K hours of processed training data, to our knowledge the largest open-source corpus of its kind. We pretrain InternW0-$\Delta$ on this corpus and demonstrate strong performance across simulation benchmarks and real-robot platforms. We will open source the training code, model weights, infrastructure, data-processing pipeline, and processed data where licenses permit. Project page: https://internrobotics.github.io/InternW0-Delta/
5. Tactile-JEPA：面向分布式触觉传感器的拓扑感知自监督表示学习
- 英文标题：Tactile-JEPA: Topology-Aware Self-Supervised Representation Learning for Distributed Tactile Sensors
- arXiv：2609.24385
💡 一句话解读：机器人靠触觉感知接触，但现有触觉编码器多从零训练，且自监督方法主要针对视觉触觉传感器，忽略了分布式电子皮肤。Tactile-JEPA利用传感器在表面的空间排布，通过预测被遮挡传感单元的表示来预训练，并用双尺度遮挡同时捕捉局部接触和全局状态。
触觉感知是机器人在视觉遮挡下执行富接触灵巧操作的重要模态。尽管预训练图像编码器已成为机器人学习流程的标准配置，触觉编码器通常仍从原始、含噪信号从头训练，这可能限制其表达能力。现有的自监督学习（SSL）方法主要面向基于视觉的触觉传感器，而分布式电子皮肤在很大程度上仍未得到解决。然而，这类传感器具有一个独特性质：其传感元件稀疏且不规则地分布在所覆盖的表面上，这使得直接复用视觉SSL方法并非最优。我们提出Tactile-JEPA，一种高效的自监督预训练方法，利用触觉传感器的空间排布来学习拓扑感知的表示。具体而言，它被训练为从 unmasked 的剩余部分预测被遮挡传感元件的嵌入，并使用传感器连接图来指导空间遮挡。我们的分析表明，有效的触觉表示需要同时捕捉局部接触细节和触觉表面的全局状态，我们通过双尺度遮挡实现了这一点。在涵盖磁性和压阻式传感器、不同机器人形态以及单传感器和双传感器配置的三个多样化数据集上，Tactile-JEPA相比先前的最先进方法将力估计误差降低了6.3%，手内方向误差降低了20.8%，并在其他下游应用（包括策略学习）中取得了持续增益。总体而言，我们的结果表明，触觉感知的收益关键取决于编码器预训练的质量，而Tactile-JEPA直接解决了这一问题。代码可在 https://github.com/E-Kovtun/tactile 获取。
📄 English Abstract
Tactile sensing is an essential modality for robots performing contact-rich, dexterous manipulation, particularly under visual occlusion. While pre-trained image encoders are standard in robot learning pipelines, tactile encoders are still commonly trained from scratch from raw, noisy signals, which might limit their expressivity. Existing self-supervised learning (SSL) approaches focus predominantly on vision-based tactile sensors, leaving distributed electronic skins largely unaddressed. These sensors, however, have a distinctive property: their sensing elements are sparse and irregularly arranged over the surface they cover, which makes direct reuse of visual SSL methods suboptimal. We present Tactile-JEPA, an efficient self-supervised pre-training method that uses the spatial arrangement of tactile sensors to learn topology-aware representations. Specifically, it is trained to predict the embeddings of masked sensing elements from the unmasked remainder, using the sensor connectivity graph to guide spatial masking. Our analysis shows that effective tactile representations require capturing both local contact details and the global state of the tactile surface, which we achieve through dual-scale masking. Across three diverse datasets spanning magnetic and piezoresistive sensors, different robot embodiments, and single- and paired-sensor configurations, Tactile-JEPA reduces force estimation error by 6.3% and in-hand orientation error by 20.8% over the prior state-of-the-art, with consistent gains in other downstream applications, including policy learning. Overall, our results demonstrate that the benefit of tactile sensing depends critically on the quality of encoder pre-training, a problem which Tactile-JEPA addresses directly. Code is available at https://github.com/E-Kovtun/tactile.
6. 利用预训练扩散模型与多模态条件增强摄影测量数字表面模型
- 英文标题：Enhancing Photogrammetric Digital Surface Models with Pretrained Diffusion Models and Multimodal Conditioning
- arXiv：2609.31199
💡 一句话解读：卫星立体摄影测量能低成本生成大范围数字表面模型，但结果常含噪声、异常值和空洞。该研究用预训练扩散模型，结合摄影测量DSM和Pléiades影像进行多模态条件精炼，改进Stable Diffusion 3架构，在法国城市测试中显著降低了高程误差。
大规模数字表面模型（DSM）可通过卫星影像的立体摄影测量以低成本生成，但所得三维地图常受噪声、异常值和空洞污染。另一方面，航空LiDAR能以高得多的成本提供高精度高程测量。本研究探讨了以摄影测量DSM和Pléiades影像为条件的扩散模型，用于精炼垂直配准的DSM。我们引入了一种改进的Stable Diffusion 3架构，包含剪枝的文本流和分块归一化策略，从而能够在LiDAR数据上稳定训练，并实现从自然图像到高程图的迁移。在法国城市的实验表明，多模态条件提高了高程精度，在上下文城市中将密集城市区域的RMSE从6.00米降至3.45米，在留出的波尔多市则从4.16米降至2.77米。
📄 English Abstract
Large-scale Digital Surface Models (DSMs) can be produced cost-effectively from satellite images via stereo-photogrammetry. However, the resulting 3D maps are often contaminated by noise, outliers, and voids. On the other hand, aerial LiDAR provides high-accuracy elevation measurements at a substantially higher cost. In this work, we study diffusion models conditioned both on photogrammetric DSMs and Pl'eiades imagery to refine vertically co-registered DSMs. We introduce a modified Stable Diffusion 3 architecture with a pruned text stream and a patch-wise normalization strategy, enabling stable training on LiDAR data and transfer from natural images to elevation maps. Experiments in French cities demonstrate that multimodal conditioning improves elevation accuracy, reducing Dense Urban RMSE from 6.00 to 3.45 m in the in-context cities and from 4.16 to 2.77 m in the held-out city of Bordeaux.
7. FoMo：生成轨迹中的分叉时刻作为感知距离
- 英文标题：FoMo: Forking Moment in Generative Trajectory as a Perceptual Distance
- arXiv：2609.25716
💡 一句话解读：这篇研究要解决的是：训练图像质量评价模型需要大量人工打分，既贵又吵。它提出用扩散模型生成图像时的“分叉时刻”来自动生成质量距离标签——两张图越早分道扬镳，感知差距越大，从而无需人工标注就能训练模型。
基于参考的图像质量评价（IQA）指标旨在反映人类如何感知一对图像之间的感知距离。为学习人类视觉系统（HVS）的工作机制，近期基于参考的 IQA 指标严重依赖人工标注数据。基于平均意见得分（MOS）的逐点打分（为每张图像分配一个标量质量值）在标注时更受青睐，但大规模收集成本极高，且因人类判断不一致而存在噪声。作为替代，二选一强制选择（2AFC）的成对标签因可靠高效而流行，但仅能捕捉成对图像间的相对比较。本文提出一种全自动数据生成流程，无需任何人工标注即可生成图像对之间的逐点感知距离标签。我们的方法利用扩散模型的生成动态作为感知距离代理，其中图像的粗粒度结构在早期时间步生成，精细细节在后期时间步生成。在生成过程中较早分叉的图像仅共享粗粒度结构，感知上相距较远；较晚分叉的图像仅精细细节不同。我们证明扩散轨迹与人类视觉系统高度一致，并将该分叉时刻（FoMo）作为基于参考的距离标签，用于监督基于参考的 IQA 指标训练。这些逐点标签支持任意图像对之间的通用比较，从而带来信息丰富的训练目标。跨多种骨干架构的大量实验证实了该生成流程的有效性，在多个基准上超越了人工标注数据集。
📄 English Abstract
Reference-based image quality assessment (IQA) metrics aim to reflect how humans perceive the perceptual distance between a pair of images. To learn how the human visual system (HVS) operates, recent reference-based IQA metrics heavily rely on human-annotated data. Mean opinion score (MOS)-based pointwise scoring, which assigns a scalar quality value per image, is preferable for annotation but is prohibitively expensive to collect at scale and is known to be noisy due to inconsistent human judgments. As an alternative, two-alternative forced choice (2AFC) pairwise labels have gained popularity due to their reliability and efficiency, but they capture only relative comparisons between pairs. In this paper, we propose a fully automated data generation pipeline that generates pointwise perceptual distance labels between image pairs without any human annotation. Our approach exploits the generative dynamics of diffusion models as a perceptual distance proxy, where the coarse structure of an image is generated in the early timesteps and the fine details are generated in the later timesteps. Images that fork early in the generation process share only coarse structure and are perceptually far apart; images that fork late differ only in fine detail. We demonstrate that the diffusion trajectory aligns well with the human visual system, and use this forking moment, FoMo, as a reference-grounded distance label to supervise the training of a reference-based IQA metric. The pointwise labels, which support universal comparison between arbitrary image pairs, enable an information-rich training objective. Extensive experiments across diverse backbone architectures confirm the effectiveness of our generation pipeline, outperforming human-annotated datasets in multiple benchmarks.
8. 真实环境中的 Jev：Jev 模型功能、应用与生态的数据驱动分析
- 英文标题：Jev in the Wild: A Data-Driven Analysis of the Jev Model’s Functionality, Applications and Ecosystem
- arXiv：2609.30216
💡 一句话解读：这篇研究想搞清楚：一个叫Jev的快速决策模型，在真实世界里到底被用在了哪些地方，以及大家的关注点是否和项目数量一致。他们分析了GitHub上2170个公开项目，发现Jev生态增长很快，主要用来做判断和打分，但关注度集中在路由和接口代理上，和项目数量并不匹配。
Jev 是一种快速、低成本的决策模型，能够以选项、二元判断和评分来回答自然语言问题。随着其公共生态迅速扩张，人们尚不清楚 Jev 在不同应用中的使用方式，以及公众关注度与项目分布之间的关系。为回答这些问题，我们对截至 2026 年 9 月 22 日从 GitHub 收集的 2170 个公开 Jev 项目进行了大规模数据驱动分析。我们发现 Jev 公共生态早期增长迅速，既有新项目涌现，也有对现有代码库的集成。在不同领域中，项目将 Jev 用于多种决策目的并组合其接口。属性判断和评分被广泛使用，而动作选择、内容过滤以及模型与工具选择的使用则因领域而异。这些模式表明，Jev 充当了一个可复用的决策组件，其功能随周边工作流而变化。与此同时，公众关注度集中在路由和接口代理上，且与项目数量并不一致。我们的发现提供了 Jev 新兴生态的定量视角，并为跨多样应用场景的通用决策模型的设计与评估提供了参考。
📄 English Abstract
Jev is a fast, low-cost decision model that answers natural-language questions with choices, binary judgments, and scores. As its public ecosystem grows rapidly, it remains unclear how Jev is used across applications and how public attention relates to project distribution. To answer these questions, we conduct a large-scale, data-driven analysis of 2,170 publicly available Jev projects collected from GitHub as of September 22, 2026. We find rapid early growth in Jev’s public ecosystem, with both new projects and integration into existing repositories. Across diverse domains, projects use Jev for multiple decision purposes and combine its interfaces. Attribute judgment and scoring are widely used, while the use of action selection, content filtering, and model and tool selection varies across domains. These patterns suggest that Jev serves as a reusable decision component whose functionality varies with the surrounding workflow. Meanwhile, public attention is concentrated in routing and interface agents and does not track project counts. Our findings provide a quantitative view of Jev’s emerging ecosystem and inform the design and evaluation of general-purpose decision models across diverse application contexts.
9. TrackEverything：通过去重3D场景表示实现长时程密集跟踪
- 英文标题：TrackEverything: Long Horizon Dense Tracking via De-Duplicating 3D Scene Representations
- arXiv：2609.30222
💡 一句话解读：现有方法要么只能长时间跟踪少量点，要么只能短时间跟踪全部点，无法兼顾。本研究把视频看成三维世界的二维投影，用三维场景坐标来记录轨迹，通过去重、只对运动点解码轨迹等技巧，首次在40GB显存内实现对超1000帧视频中所有可见点的长时程跟踪。
现有的点跟踪模型面临一个根本性权衡：要么在长时程上跟踪稀疏的查询点，要么仅在短片段中跟踪所有点。我们提出 TrackEverything，一种 3D 点跟踪器，通过将视频表示为世界坐标系中的持久 3D 场景轨迹来打破这一权衡。基于视频是底层 3D 世界的 2D 投影这一洞见，TrackEverything 将模型复杂度与视频时长解耦，使其转而随独特的物理场景几何规模扩展。我们的方法引入三项关键创新：第一，在滑动窗口边界采用基于体素化的去重机制来合并位置重合的轨迹，防止对同一表面的重复观测被冗余累积；第二，将跟踪分解为一个端点细化器（预测每个点的目的地以及静态与动态分类），随后由一个轻量级轨迹细化器专门为动态点解码稠密轨迹；第三，提出 3D WAFT，用场景点云中的高效特征采样替代内存开销巨大的 4D 相关体。据我们所知，TrackEverything 是首个能在 40 GB GPU 显存内跟踪超过 1000 帧视频中所有可见点的 3D 跟踪器。在 TAPVid-3D 上，TrackEverything 在短片段上以超过 20% 的 APD 优于所有开源全帧稠密 3D 跟踪器，同时在长序列上与最先进的稀疏跟踪器保持竞争力，尽管其跟踪的点数远多于后者。
📄 English Abstract
Existing point tracking models face a fundamental tradeoff: they can either track a sparse set of query points over long horizons, or track all points across only short clips. We introduce TrackEverything, a 3D point tracker that breaks this trade-off by representing videos as persistent 3D scene tracks in world coordinates. Grounded in the insight that videos are 2D projections of an underlying 3D world, TrackEverything decouples model complexity from video duration, allowing it to scale with unique physical scene geometry instead. Our approach introduces three key innovations. First, we employ a voxelization-based de-duplication mechanism at sliding-window boundaries to merge co-located tracks, preventing repeated observations of the same surface from redundantly accumulating. Second, we decompose tracking into an endpoint refiner that predicts each point’s destination and static-versus-dynamic classification, followed by a lightweight trajectory refiner that decodes dense trajectories exclusively for dynamic points. Third, we propose 3D WAFT, replacing memory-prohibitive 4D correlation volumes with efficient feature sampling in the scene cloud. To the best of our knowledge, TrackEverything is the first 3D tracker capable of tracking all visible points across videos exceeding 1000 frames within 40 GB of GPU memory. On TAPVid-3D, TrackEverything outperforms all open-source all-frame dense 3D trackers by more than 20% APD on short clips, while remaining competitive with state-of-the-art sparse trackers on long sequences, despite tracking far more points.
10. SLCA-GRPO：解决工具调用强化学习中的跨段信用误归因
- 英文标题：SLCA-GRPO: Resolving Cross-Segment Credit Misattribution in Tool-Calling RL
- arXiv：2609.29050
💡 一句话解读：工具调用智能体的输出混合了工具调用和自然语言总结，现有强化学习（如GRPO）把整条轨迹的奖励统一分配给所有词，导致总结部分的噪声干扰工具决策，造成信用误归因。本文提出SLCA-GRPO，通过分段锁定信用分配，将执行优势给工具词、偏好优势给总结词，并构建模拟器支持训练，在多个基准上提升准确率并降低成本。
工具调用智能体产生异构输出，将结构化工具调用与面向用户的自然语言总结交织在一起。这种输出异构性在标准同策略强化学习（RL）中呈现出一个结构性失效模式：像GRPO这样的算法不加区分地将同质的轨迹级标量优势广播给所有词元。因此，来自总结生成的梯度噪声泄漏到工具决策词元中，导致跨段信用误归因和脆弱的优化。在这项工作中，我们提出了SLCA-GRPO，一个包含分段锁定信用分配（SLCA）的框架。为了在没有昂贵真实API的情况下实现可扩展探索和稳定训练，我们首先构建了模式引导的LLM模拟器（SGLS）作为基础训练设施。在此基础上，SLCA在单组轨迹的结构段级别解耦优势估计，而不需要从中间状态进行额外轨迹采样。在分层奖励（HierR）的支持下，SLCA将执行优势路由到工具词元，将偏好优势路由到总结词元，从而在每次策略更新中消除优势污染（主要的跨段信用误归因渠道）。在7B骨干模型上，SLCA-GRPO加速了收敛，并在相同训练预算下，在域内评估中比标准GRPO、ToolPO和RLTR高出+2.53个百分点，在伯克利函数调用排行榜（BFCL）上高出+1.36个百分点，在τ²-Bench上高出+9.15个百分点，以更少的工具冗余和成本实现了更高的准确率。
📄 English Abstract
Tool-calling agents produce heterogeneous outputs, interleaving structured tool invocations with user-facing natural language summaries. This output heterogeneity presents a structural failure mode in standard on-policy Reinforcement Learning (RL): algorithms like GRPO indiscriminately broadcast a homogeneous trajectory-level scalar advantage to all tokens. Consequently, gradient noise from summary generation leaks into tool-decision tokens, causing cross-segment credit misattribution and brittle optimization. In this work, we propose SLCA-GRPO, a framework incorporating Segment-Locked Credit Assignment (SLCA). To enable scalable exploration without costly real APIs and stable training, we first construct the Schema-Guided LLM Simulator (SGLS) as foundational training infrastructure. Building on this, SLCA decouples advantage estimation at the structural segment level within a single group of rollouts, without requiring additional rollouts from intermediate states. Supported by Hierarchical Rewards (HierR), SLCA routes execution advantages to tool tokens and preference advantages to summary tokens, eliminating advantage contamination (the dominant cross-segment credit misattribution channel) within each policy update. On a 7B backbone, SLCA-GRPO accelerates convergence and outperforms standard GRPO, ToolPO, and RLTR by +2.53 pp on in-domain evaluation, +1.36 pp on the Berkeley Function-Calling Leaderboard (BFCL), and +9.15 pp on $\tau^2$-Bench under the same training budgets, achieving higher accuracy with reduced tool redundancy and costs.
11. CARD：面向个性化文本生成的聚类级适配与奖励引导解码
- 英文标题：CARD: Cluster-level Adaptation with Reward-guided Decoding for Personalized Text Generation
- arXiv：2601.06352
💡 一句话解读：这篇论文要解决的是：让大语言模型为每个用户做个性化写作时，既想贴合个人风格，又难以大规模部署。作者提出 CARD 框架：先把风格相近的用户聚成组，训练组专属的小型适配模块；再用对比用户原文和组内生成文本的方式，自动学出个人偏好；最后在生成时只做轻量调整，不动原模型。
将大语言模型适配到单个用户仍具挑战，原因在于细粒度个性化与可扩展部署之间存在矛盾。我们提出 CARD，一个通过渐进式细化实现有效个性化的分层框架。CARD 首先根据共享的风格模式对用户进行聚类，并学习组特定的 LoRA 适配器，从而实现稳健的泛化与较强的低资源性能。为捕捉每个聚类内部的个体差异，我们提出一种隐式偏好学习机制，将用户撰写的文本与聚类级生成结果进行对比，使模型无需人工标注即可推断用户特定的风格偏好。在推理阶段，CARD 仅通过轻量级用户偏好向量和低秩 logit 修正（low-rank logit corrections）在解码时注入个性化，同时保持基础模型冻结。在 LaMP 和 LongLaMP 基准上的实验表明，CARD 相比基线取得了更优的生成质量，同时显著提升了实际个性化文本生成的效率与可扩展性。
📄 English Abstract
Adapting large language models to individual users remains challenging due to the tension between fine-grained personalization and scalable deployment. We present CARD, a hierarchical framework that achieves effective personalization through progressive refinement. CARD first clusters users according to shared stylistic patterns and learns group-specific LoRA adapters, enabling robust generalization and strong low-resource performance. To capture individual differences within each cluster, we propose an implicit preference learning mechanism that contrasts user-authored text with cluster-level generations, allowing the model to infer user-specific style preferences without manual annotation. At inference time, CARD injects personalization exclusively at decoding via lightweight user preference vectors and low-rank logit corrections, while keeping the base model frozen. Experiments on the LaMP and LongLaMP benchmarks show that CARD achieves superior generation quality compared to baselines, while significantly improving efficiency and scalability for practical personalized text generation.
12. 隐式个性化与显式风格会冲突吗？PsPLUG：用于平衡定制化 LLM 中个性化与风格的轻量级插件
- 英文标题：Do Implicit Personalization and Explicit Styles Conflict? PsPLUG: A Lightweight Plug-in for Balancing Personalization and Style in Customized LLMs
- arXiv：2601.06362
💡 一句话解读：这篇研究要解决的问题是：让大模型既懂用户个人偏好，又能按指定风格说话时，风格指令往往会“冲掉”个性化。作者提出一个轻量插件 PsPLUG，先按风格生成，再补上用户特有的“小修正”，还能在推理时调节个性化强度，从而兼顾两者。
个性化大语言模型通常被期望遵循显式风格指令，但我们发现这类指令会削弱个性化方法旨在保留的用户特有特征。我们将这种失败模式称为“个性化崩塌”（personalization collapse）：显式风格控制可能与隐式用户偏好相冲突。为解决这一挑战，我们提出 PsPLUG，一种轻量级插件，它在考虑所请求风格之后学习用户特有的残差（residual）。PsPLUG 还允许我们在推理时调节个性化强度。实验表明，显式风格指令会削弱现有方法的个性化，而 PsPLUG 在提供对个性化与风格遵循之间平衡的精确控制的同时，能更好地保留用户偏好。
📄 English Abstract
Personalized large language models are often expected to follow explicit style instructions, yet we find that such instructions can undermine the user-specific characteristics that personalization methods aim to preserve. We call this failure mode personalization collapse: explicit style control can conflict with implicit user preferences. To address this challenge, we propose PsPLUG, a lightweight plug-in that learns a user-specific residual after accounting for the requested style. PsPLUG also allows us to tune personalization strength at inference time. Our experiments show that explicit style instructions can diminish personalization in existing methods, whereas PsPLUG better preserves user preferences while providing precise control over the balance between personalization and style adherence.
13. AgentWorld：多智能体大语言模型长时程协作基准测试
- 英文标题：AgentWorld: Benchmarking Long-Horizon Collaboration of Multi-agent LLMs
- arXiv：2609.31590
💡 一句话解读：现有测试多智能体协作的基准，要么只考竞争、要么只跑20步以内、要么只看个人成绩，没法真正衡量大模型智能体一起干活的能力。为此，作者构建了AgentWorld：一个MMORPG沙盒里的100个人工标注任务，需要3到20个角色不同的智能体通过聊天、共同计划和分资源来合作50多轮，并提出了一个能算出“团队努力有多少真正起了作用”的因果协作有效性指标。
现有的多智能体基准测试主要考察竞争环境、20步以内的短时程交互，或只是简单汇总个体表现，无法分离并凸显基于大语言模型（LLM）的智能体真正的协作能力。我们提出了AgentWorld，一个包含100个人工标注任务（另有100个增强变体）的基准，用于评估长时程、多智能体协作。任务在一个丰富的MMORPG沙盒中跨越50多个交互轮次，需要3至20个具有不对称角色与能力的智能体，在无法访问彼此内部状态的黑盒设定下，通过通信、联合规划和资源共享进行协调。为了在传统的二元任务成功之外量化协作有效性，我们提出了因果协作有效性（Causal Collaboration Effectiveness, CCE），这是一种基于图的指标，追踪智能体动作之间的因果依赖，并衡量团队努力中实际对结果有贡献的比例。对Gemini 3 Flash、Claude Haiku 4.5、GPT-5 Mini和DeepSeek R1-70B的实验表明，即使最佳模型也仅达到52.0%的任务成功率，并存在系统性失败模式，包括通信中断、角色混淆以及无法跨轮次维持共享计划。AgentWorld完全开源。
📄 English Abstract
Existing multi-agent benchmarks primarily test in competitive settings, short-horizon interactions under 20 steps, or simply aggregate individual performance, failing to isolate and highlight genuine collaboration capabilities of LLM-based agents. We introduce AgentWorld, a benchmark of 100 human-annotated tasks (with 100 augmented variants) for evaluating long-horizon, multi-agent collaboration. Tasks span 50+ interaction rounds across a rich MMORPG sandbox and require 3-20 agents with asymmetric roles and abilities to coordinate through communication, joint planning, and resource sharing under a blackbox setting where each agent acts independently without access to others’ internal states. To quantify collaboration effectiveness in addition to conventional binary task success, we propose Causal Collaboration Effectiveness (CCE), a graph-based metric that traces causal dependencies between agent actions and measures what fraction of a team’s effort actually contributed to the outcome. Experiments with Gemini 3 Flash, Claude Haiku 4.5, GPT-5 Mini, and DeepSeek R1-70B show that even the best model achieves only 52.0% task success, with systematic failure modes including communication breakdowns, role confusion, and inability to maintain shared plans across rounds. AgentWorld is fully open-source.
14. 游戏竞技场：竞争环境中的大语言模型策略评估
- 英文标题：Game Arena: Strategic LLM Evaluation in Competitive Environments
- arXiv：2609.31473
💡 一句话解读：现有的大模型评测多是静态考卷，模型越来越强后分数容易“刷满”，看不出真实差距。为此研究者搭建了“游戏竞技场”，让大模型在象棋、扑克、狼人杀等游戏里真刀真枪对战，对手越强难度越高，从而持续、公平地衡量模型的策略与应变能力。
我们推出 Kaggle Game Arena，一个开放且不断扩展的平台，通过竞技类游戏来评估大语言模型（LLM）。与静态基准不同，游戏竞技场让模型在结构化环境中进行正面对决，其博弈强度会随模型进化而自然提升，从而避免性能饱和。本技术报告详述了 Game Arena 背后的基础设施，并介绍了三个试点游戏环境：国际象棋、扑克和狼人杀。这些环境涵盖完全信息、不完全信息以及多人博弈场景，可用于系统研究模型在不确定性下的战略规划、适应与鲁棒性。针对每个游戏，我们详细描述了环境、评估指标以及跨模型完整竞赛的结果。凭借稳健的基础设施和基于大规模真实标注的评估，Game Arena 确保了可复现性、透明度，以及随时间推移对新游戏和新变体的可推广性。
📄 English Abstract
We introduce Kaggle Game Arena, an open and ever-expanding platform to evaluate large language models (LLMs) through competitive games. Different from static benchmarks, game arena enables models to play head-to-head matchups in structured environments where the gameplay strength naturally increases as models evolve, preventing performance saturation. This technical report details the infrastructure behind Game Arena and describes the three pilot game environments: Chess, Poker, and Werewolf. These environments span perfect information, imperfect information, and multiplayer game settings, enabling a systematic study of models’ strategic planning, adaptation, and robustness under uncertainty. For each game, we provide a detailed description of the environment, evaluation metrics, and results from running full competitions across models. Through robust infrastructure and large-scale ground-truth based evaluation, Game Arena ensures reproducibility, transparency and generalizability to new games and variants over time.
15. IndicBankBench：评估印度零售银行中语言模型助手的安全性与可靠性
- 英文标题：IndicBankBench: Evaluating Safety and Reliability of Language Model Assistants in Indian Retail Banking
- arXiv：2609.29167
💡 一句话解读：银行助手需要根据账户信息回答问题甚至执行操作，但只看最终回答会漏掉很多错误，比如重复询问已有信息、用错账户或说对却写错。为此，作者构建了IndicBankBench，包含799个印度零售银行场景，分四个阶段评估，并用严格通过率（三次全对）和至少一次通过率来对比。结果显示，至少一次通过率会高估可靠性，严格通过率仅43.7%-58.2%。
银行助手必须利用账户特定信息来回答请求，并在许多情况下通过工具采取行动。仅评估最终响应会遗漏重要错误：助手可能询问它已经拥有的信息、依赖过时上下文、选择错误账户，或在说出正确值后写入无效值。我们推出了IndicBankBench，这是一个包含799个案例的基准，涵盖印度零售银行的五个运营领域、一个能力/拒绝领域和二十个主要维度。案例在四个阶段进行评估：安全性、行动与工具使用、响应充分性和咨询质量。工具使用和大多数安全检查是确定性的。一个窄解析器仅处理模糊的写入前确认案例，而单独的LLM评判器评估语义响应充分性。我们对每个案例运行三次，并报告严格的pass^3，即要求所有试验均成功。在评估的十一个模型中，严格可靠性范围为43.7%至58.2%，而至少一次成功率范围为60%至74%。这一差距表明，至少一次成功率可能夸大可靠的银行行为。案例级诊断还区分了提出不必要问题的系统与采取行动但未能协调客户上下文或完全解决请求的系统。我们发布了案例、模拟环境和评估工具。
📄 English Abstract
Banking assistants must use account-specific information to answer requests and, in many cases, take actions through tools. Evaluating only the final response misses important errors. An assistant may ask for information it already has, rely on stale context, select the wrong account, or write an invalid value after stating the correct one. We introduce IndicBankBench, a 799-case benchmark for Indian retail banking spanning five operational domains, a capability/refusal domain, and twenty primary axes. Cases are evaluated at four stages: safety, action and tool use, response adequacy, and advisory quality. Tool use and most safety checks are deterministic. A narrow resolver handles only ambiguous confirmation-before-write cases, while a separate LLM judge evaluates semantic response adequacy. We run every case three times and report strict pass^3, which requires success on all trials. Across the eleven evaluated models, strict reliability ranges from 43.7% to 58.2%, whereas at-least-once success ranges from 60% to 74%. This gap shows that at-least-once success can overstate dependable banking behavior. The case-level diagnostics also distinguish systems that ask unnecessary questions from those that act but fail to reconcile customer context or fully resolve the request. We release the cases, mock environment, and evaluation harness.