WORLDWIDE_SanGuai / Research bulletin

[ worldwide.arakne.de ]

世界很大,讀一小點。

SanGuai 的三日研究佈告板。每天更新,讀完就走。

EST. 202672H上架 72 小時
人工智慧 / 2611.17336

回到像素:將潛空間影片擴散模型提升至像素空間

Back to Pixels: Lifting Latent Video Diffusion into Pixel Space

作者: Yanke Song, Shuo Chen, Chris Rockwell, Guande He, Nan Liu, Desai Xie, Tianpei Gu, Yong-Hyun Park et al.論文發布: 2026-11-12離架倒數: 62 小時

研究速讀

作者提出 PixVid,將預訓練的潛空間影片擴散模型改為直接在像素空間逐幀生成。方法保留擴散 Transformer,以卷積頭取代 VAE 解碼器,並指出預測乾淨影片而非速度、搭配較大的噪聲位移是品質關鍵。作者稱以 Wan2.1-14B 微調後,在約 130 萬訓練片段內可與潛空間基線競爭。

值得留意

  1. 保留擴散 Transformer,用卷積頭取代 VAE 解碼器,直接輸出像素
  2. 作者稱預測乾淨影片而非速度、加上較大噪聲位移,是提升品質的關鍵
  3. 作者報告可達兩倍解析度而每幀 token 數不變,並改善逐幀條件控制與互動回應

閱讀限制

僅依標題與摘要整理,未讀全文,無法核實實驗設定、比較基準與作者所述結果。

原始摘要 [EN]

Frontier video generation models work in a VAE latent space that compresses video in both space and time. This compression made training tractable, but now limits high-resolution, controllable, and interactive generation -- doubling resolution quadruples token count, conditions apply to groups of frames rather than individual ones, and each response must wait for the rest of its multi-frame block. We present PixVid, a simple yet effective method that lifts a pretrained latent video model to directly generate every frame in pixel space. PixVid keeps the diffusion transformer and replaces the VAE decoder with a convolutional head that renders its outputs into pixels. We find that predicting clean video rather than velocity, together with a larger noise shift, is key to quality. Finetuned from Wan2.1-14B, PixVid is competitive with the latent baseline within sim 1.3M training clips. Working framewise in pixels, it generates at 2times higher resolution with the same per-frame token count, keeps depth-conditioned video editing aligned under fast motion, and responds to each action at the next frame. Project page: https://chenshuo20.github.io/PixVid/

AI 摘要僅根據公開標題與摘要整理;研究結論來自作者,請以原文為準。