HiVe-Mem:交錯水平記憶擴展與垂直證據組合的多模態代理記憶框架
HiVe-Mem: Interleaving Horizontal Memory Expansion and Vertical Evidence Composition for Multimodal Agents
研究速讀
此論文提出 HiVe-Mem 多模態記憶框架,交錯水平記憶擴展與垂直證據組合。水平方向以文字與視覺錨點將記憶情節組織成關聯圖,從語意檢索種子擴展鄰居以補足資訊;垂直方向讓每則記憶保留摘要、說明、視覺視圖、原始影像與互動歷史,並以輕量可訓練選擇器為每則記憶組合證據子集。作者稱於三個基準上提升表現並大幅降低 token 消耗。
值得留意
- 水平記憶擴展以文字與視覺錨點建立關聯圖,從語意檢索種子擴展相關鄰居以補回互補資訊。
- 垂直證據組合讓每則檢索到的記憶保留多種證據形式,並由輕量可訓練選擇器依查詢獨立挑選證據子集。
- 作者報告在三個多模態記憶基準上跨基礎模型一致提升任務表現,且相較最省 token 的多模態基線約減少九成 token 消耗。
閱讀限制
僅閱讀標題與摘要,未檢視全文、實驗設定、基準細節或同儕審查狀態,因此無法驗證作者所述結果,也無法評估方法在何種條件下失效。
原始摘要 [EN]
As multimodal agents accumulate increasingly long interaction histories, effectively retrieving and utilizing past information becomes critical for long-term interaction. Existing memory systems typically rely on semantic retrieval over compressed memories and expose predefined representations, which may miss relevant memories and overlook heterogeneous evidence needs across queries. We introduce \hivemem, a multimodal memory framework that interleaves \textsc{H}or\textsc{i}zontal Memory Expansion with \textsc{Ve}rtical Evidence Composition. For horizontal memory expansion, \hivemem organizes memory episodes into a relational graph using textual and visual anchors, expanding semantically retrieved seeds with related neighbors to recover complementary information. For vertical evidence composition, each retrieved memory retains multiple evidence forms, including compact summaries, captions, grounded visual views, raw images, and interaction history. A lightweight trainable selector independently composes an evidence subset for each retrieved memory, enabling query-adaptive utilization of heterogeneous multimodal information. This two-dimensional design jointly broadens memory coverage and adapts the evidence contributed by individual memories. Experiments across three multimodal memory benchmarks show that \hivemem consistently improves task performance across base models while reducing token consumption by about 90\% compared with the most efficient competing multimodal baselines.
AI 摘要僅根據公開標題與摘要整理;研究結論來自作者,請以原文為準。