WORLDWIDE_SanGuai / Research bulletin

[ worldwide.arakne.de ]

世界很大,讀一小點。

SanGuai 的三日研究佈告板。每天更新,讀完就走。

EST. 202672H上架 72 小時
人工智慧 / 2611.15844

Open Vision Banana:將預訓練影像生成器轉為視覺感知系統的完全開放配方

Open Vision Banana: A Fully Open Recipe for Generative Visual Perception

作者: Arpit Jadon, Valentinos Pariza, Jona Ruthardt, Ryousuke Yamada, A. Sophia Koepke, Andrei Bursuc, Yuki M. Asano論文發布: 2026-11-11離架倒數: 63 小時

研究速讀

作者提出 Open Vision Banana,一套完全開放且可重現的配方,把預訓練影像生成器改造成多任務視覺感知系統。研究分析色彩編碼、提示設計、參數調適與任務平衡等關鍵設計,並指出這些選擇會互相影響。以 4B 與 9B 開放權重模型為基礎,使用公開資料集訓練,作者報告其零樣本 Cityscapes 表現超越 Vision Banana,並提出測試時圖例集成方法。

值得留意

  1. 作者指出原 Vision Banana 未公開資料、權重、程式碼與實作細節,因此提出完全開放的替代配方。
  2. 研究辨識出色彩編碼、提示設計、參數調適與任務平衡等影響泛化的設計選擇,且這些選擇彼此交互作用。
  3. 作者報告 OVB 在 Cityscapes 零樣本 mIoU 為 73.9,高於 Vision Banana 報告的 69.9,並提出圖例集成測試時方法。

閱讀限制

僅閱讀標題與摘要,無法確認實驗設定、比較基準是否對等,也無法驗證作者報告的數字與可重現性。

原始摘要 [EN]

Recent work suggests that pretrained image generators encode real-world understanding beyond image synthesis. Vision Banana exploits this by framing vision tasks as conditional image editing. However, its data, weights, code, and implementation details are not public. We present Open Vision Banana (OVB), a fully open and reproducible recipe for converting pretrained generators into unified multi-task perception systems. Our analyses identify the design choices that matter most for generalization, including color encoding, prompt design, parameter adaptation, and task balancing. These choices interact, so individually helpful choices can hurt when combined with others. Starting from 4B and 9B parameter open-weight base models and using 2.1M-3.8M training pairs from public datasets, OVB surpasses Vision Banana's reported Cityscapes zero-shot performance (73.9 vs. 69.9 mIoU) and rivals state-of-the-art affine-invariant depth estimators. Its non-zero-shot variants come within five points of Vision Banana on referring and reasoning segmentation but remain further behind on instance segmentation. All models largely retain the base model's image-editing abilities. With legend ensembling, we introduce a test-time scaling method for generator-based perception. It segments an image under several class-color legends and fuses the results by majority vote, correcting unstable color-class bindings and improving mIoU by up to 8.7 points on Cityscapes and 15.0 points on COCO-Object without retraining. We release the dataset construction and processing pipeline, the training and evaluation code, as well as the final, intermediate, and ablation checkpoints. Project website: https://openvisionbanana.github.io

AI 摘要僅根據公開標題與摘要整理;研究結論來自作者,請以原文為準。