Ant Group UBTree Improves Speculative Decoding: Parallel Screening of Draft Trees, Enabling Large Models to Verify More Token at Once
2026-10-29 18:32:44
According to CoinMeta, Ant Bird has open-sourced a large model inference and decoding solution UBTree. This solution parallelizes the screening of draft trees, allowing smaller models to "draft" in advance, followed by the large model verifying multiple token at once, thereby reducing the number of computational rounds required for individual generation. UBTree improves the screening method by first predicting in parallel which token may appear in subsequent positions, and then using a lightweight model to determine whether adjacent token can be reasonably connected. After adjustments, using a dflash selector to construct the tree resulted in an average acceleration of 5.95 times. By switching to a ubtree selector and adjusting the training method, this was further increased to 6.88 times, representing a further improvement of about 16%.
Bullish 0
Bearish 0
Source:Internet
This content is for market information only and does not constitute investment advice.