WORLDWIDE_SanGuai / Research bulletin

[ worldwide.arakne.de ]

世界很大,讀一小點。

SanGuai 的三日研究佈告板。每天更新,讀完就走。

EST. 202672H上架 72 小時
人工智慧 / 2611.17167

演員、評審或教師?理解長程代理中的決策模型

Actor, Judge, or Teacher? Understanding Decision Models in Long-Horizon Agents

作者: Haochen Liu, Qianhao Zhu, Zixuan Wang, Dingming Li, Tongbo Chen, Peng Qiu, Yingqi Qu, Weiming Lu et al.論文發布: 2026-11-12離架倒數: 38 小時

研究速讀

這篇論文探討決策模型在長程代理中的三種角色:行動者、評審與教師。作者在 ALFWorld 與 WebShop 上比較三個決策模型與三個通用大型語言模型,發現表現不佳的行動者仍可作為有效教師,因為事後評判能揭示行動時難以預見的後果。

值得留意

  1. 作者在 ALFWorld 上比較三個決策模型與三個通用大型語言模型作為行動者與評審,並以 JEV 作為教師。
  2. 作為行動者時,六個模型都傾向選擇較容易的子目標,而偏離必須先完成的子目標。
  3. 作為教師時,JEV 讓 Qwen2.5-1.5B 策略經 GRPO 訓練後,在 ALFWorld 的成功率由 72.8% 提升至 82.9%,在 WebShop 由 56.8% 提升至 71.4%。

閱讀限制

僅閱讀標題與摘要,無法核實實驗設定、基準比較與統計顯著性,也無法確認程式碼的實際可用性;所有結果均為作者陳述。

原始摘要 [EN]

Decision models answer a question with a typed output, such as a choice among options with a probability for each, so one model can choose an agent's actions, evaluate actions already taken, or score every action to teach a policy with reinforcement learning. Prior work has studied them mainly as actors, judging their usefulness by how well they act. We ask whether a decision model that acts poorly can still teach well. We compare three decision models and three general LLMs as actors and judges on ALFWorld, and use JEV as a teacher on ALFWorld and WebShop. As actors, all six models are pulled away from a subgoal that must come first by an easier one. As judges that see the whole trajectory, the same models credit the steps toward that subgoal, and asking whether each action helped, rather than whether the task will succeed, separates useful from useless actions. As a teacher, JEV raises a Qwen2.5-1.5B policy over GRPO from 72.8% to 82.9% success on ALFWorld and from 56.8% to 71.4% on WebShop. These results suggest that a poor actor can still teach well, because judging an action after the episode ends reveals the consequences that acting must anticipate. Code is available at https://github.com/ZJU-REAL/decision-agents.

AI 摘要僅根據公開標題與摘要整理;研究結論來自作者,請以原文為準。