YuE2:以符號規劃統一符號與音訊音樂生成
YuE2: Unifying Symbolic and Audio Music Generation at Frontier Quality
研究速讀
作者提出 YuE2,以符號規劃串接樂譜與音訊生成:單一 AR-NAR Mixture-of-Transformers 先寫出含旋律與和聲的可讀樂譜,再展開為語意 token 並生成完整歌曲音訊。摘要稱專家偏好此規劃方式,且在 WildSongBench 與聽測中表現具競爭力;惟這些結果皆為作者自述。
值得留意
- 以單一 AR-NAR Mixture-of-Transformers 先產生可讀樂譜,再轉為語意 token 並合成整首歌曲音訊。
- 作者稱同檢查點下專家較偏好含符號規劃的版本,整體偏好率為 49.3% 對 34.6%。
- 摘要稱 WildSongBench 的 SongBench Global Avg 為 6.73,best-of-8 達 6.96,並稱 MERT2 在 15 項 MARBLE 指標中 14 項超越先前最佳。
閱讀限制
僅閱讀標題與摘要,無法查核實驗設計、基線選擇、評分細節與統計顯著性,所有數據均為作者陳述。
原始摘要 [EN]
Symbolic models make melody, harmony, rhythm, and form explicit but typically stop before a finished recording; audio models produce complete songs while leaving composition implicit. We introduce YuE2, which unifies symbolic and audio music generation at frontier quality through symbolic planning. A single AR-NAR Mixture-of-Transformers (MoT) first writes a readable score specifying melody and harmony, expands it into semantic music tokens, and realizes it as full-song audio. In comparisons using the same checkpoint, experts prefer symbolic planning for overall quality and musicality, with 49.3% of overall preferences versus 34.6% without planning. Experts also favor the unified model over a separate language model and diffusion Transformer. On WildSongBench, YuE2 scores 6.73 on SongBench Global Avg, exceeding all evaluated public baselines. Selecting from eight candidates (best-of-8), YuE2 reaches 6.96, the highest observed mean among all evaluated systems. Expert listening further establishes its competitiveness with proprietary song generators, favoring best-of-8 over Suno v4.5 and yielding nearly balanced preferences against Suno v5. To learn this generation process from recordings without aligned scores, we introduce MERT2 and SheetSage2 to supply semantic and symbolic supervision. MERT2 sets a new state of the art in music representation learning, surpassing previous best results on 14 of 15 MARBLE metrics; SheetSage2 leads 12 of 15 benchmark-metric pairs in our lead-sheet transcription comparison. The same checkpoint follows score edits while largely preserving unedited musical content and generates zero-shot covers without cover-specific training. Its readable score also enables agentic music editing, with external language models translating user feedback into revisions of the composition.
AI 摘要僅根據公開標題與摘要整理;研究結論來自作者,請以原文為準。