WORLDWIDE_SanGuai / Research bulletin

[ worldwide.arakne.de ]

世界很大,讀一小點。

SanGuai 的三日研究佈告板。每天更新,讀完就走。

EST. 202672H上架 72 小時
人工智慧 / 2611.16517

UniSpark:探索混合注意力於多模態理解與生成

UniSpark: Exploring Hybrid Attention for Multimodal Understanding and Generation

作者: Jiancheng Pan, Yuzheng Liu, Hengyuan Cao, Qiyuan Zhang, Yangyan Li, Biao Gong, Zheng Zhang, Ka Long Cheng et al.論文發布: 2026-11-12離架倒數: 62 小時

研究速讀

作者提出 UniSpark-9B 統一多模態模型,以混合專家 Transformer 為骨幹,結合全注意力與 GatedDeltaNet 線性注意力,並以自迴歸文字預測與視覺流匹配聯合支援視覺問答、文生圖、指令編輯與交錯圖文生成。作者稱其在 MME-S 得 2470.3、GenEval 得 0.83。

值得留意

  1. 模型約 9B 參數,每個分支皆採混合專家 Transformer 骨幹,結合全注意力與 GatedDeltaNet 線性注意力。
  2. 訓練資料依作者所述包含 1.597 億生成樣本、2.134 億理解樣本與 62.48 萬交錯序列,並以四階段課程聯合訓練。
  3. 作者報告 MME-S 分數 2470.3、GenEval 0.83,並以質性範例展示交錯圖像生成、影片生成與影片編輯。

閱讀限制

僅閱讀標題與摘要,無法核對實驗設定、基準細節或分數是否經獨立驗證,所有數據均為作者自述。

原始摘要 [EN]

We present UniSpark-9B, a unified multimodal model that investigates hybrid attention for visual understanding and generation. Built on a Mixture-of-Transformer-Experts backbone with approximately 9B parameters in each branch, UniSpark combines full attention for explicit token interactions with GatedDeltaNet linear attention for compact context propagation. Autoregressive text prediction and visual flow matching jointly support visual question answering, text-to-image generation, instruction-guided editing, and interleaved text-image prediction. In interleaved generation, noisy visual targets are isolated from subsequent predictions, while clean visual representations provide context for later outputs. A blockwise protocol enforces this distinction through attention masks in full-attention layers and recurrent-state retention rules in linear-attention layers. The selected training sources contain 159.7M generation samples, 213.4M understanding samples, and 624.8K interleaved sequences. A four-stage curriculum jointly trains multimodal understanding and visual generation, refining answer accuracy and instruction following while extending image synthesis to 2K. UniSpark scores 2470.3 on MME-S and 0.83 on GenEval. Qualitative examples further illustrate its capabilities in interleaved image generation, video synthesis, and video editing. Together, these results demonstrate that hybrid attention provides an effective backbone for unified multimodal understanding and generation.

AI 摘要僅根據公開標題與摘要整理;研究結論來自作者,請以原文為準。