推文
100 tweets · 45 sources账号:全部mattpocockukbchernydair_aifelixriesebergtrq212simonw0xblacklightadocompletedexhorthydani_avila7ryancarsonGeminiAppgarrytankarpathymckaywrigleynummanaliomarsar00xPauliusAnthropicAIDanielMiesslerGoogleDeepMindRaillyHugoaidenybaiasynkimocgtwtsctatedevdemishassabisdillon_mulroydoodlesteinelithrarjarredsumnerjoshalbrechtjpschroederlennysanlexruslogancyangpetergyangrohanpaul_aisawyerhoodservasyy_aithdxrthorstenballycombinatoryiliushzarazhangrui
@omarsar0·5d agoFor You
NEW research from NVIDIA.
Post-training agents with RL is powerful but expensive.
Every parameter update needs full multi-turn rollouts with environment interactions, making end-to-end RL prohibitively costly for long-horizon agentic tasks.
This research offers a practical middle ground.
The work introduces PivotRL, a framework that operates on existing SFT trajectories to combine the computational efficiency of SFT with the out-of-domain retention of end-to-end RL.
Instead of exhaustive full-trajectory rollouts, PivotRL identifies pivots, informative intermediate turns where sampled actions show mixed outcomes, and trains only on those high-signal moments.
Standard SFT degrades OOD performance by -9.83 points on average. PivotRL stays near zero (+0.21) while achieving +14.11 average in-domain gains over the base model versus +9.94 for SFT.
On SWE-Bench, PivotRL reaches competitive accuracy with E2E RL using 4x fewer rollout turns and 5.5x less wall-clock time.
The method is already deployed in production as the workhorse for NVIDIA's Nemotron-3-Super-120B agentic post-training.
Paper: https://t.co/sIBLUpyfMD
Learn to build effective AI agents in our academy: https://t.co/1e8RZKs4uX
NVIDIA 新研究。 用 RL 对 agent 进行 post-training 效果很强，但成本高昂。 每次参数更新都需要完整的多轮 rollout 与环境交互，导致端到端 RL 在长 horizon 的 agentic 任务中成本极高。 这项研究提供了一个实用的折中方案。 论文提出 PivotRL——一个基于已有 SFT trajectory 运行的框架，将 SFT 的计算效率与端到端 RL 的 out-of-domain 保留能力结合起来。 与其跑完整 trajectory rollout，PivotRL 识别「pivot 点」——即采样动作呈现混合结果的关键中间轮次——只在这些高信号时刻上进行训练。 标准 SFT 会让 OOD 性能平均下降 9.83 点，而 PivotRL 几乎持平（+0.21），同时相比 base model 在域内平均提升 +14.11（SFT 仅 +9.94）。 在 SWE-Bench 上，PivotRL 用少 4 倍的 rollout 轮次、少 5.5 倍的实际训练时间，达到与端到端 RL 相当的准确率。 该方法已部署至生产环境，作为 NVIDIA Nemotron-3-Super-120B agentic post-training 的核心训练引擎。
@omarsar0·3月25日For You
Nice cheat sheet for Claude Code. https://t.co/ikGzbSqjRK
这是一份不错的 Claude Code 速查表。
100 tweets · 45 sources