Ant Group open-sources the UBTree speculative decoding solution, with average test speedup increased to 6.88 times

AT A GLANCE

Ant Bailing has open-sourced UBTree, improving speculative decoding efficiency by selecting draft trees in parallel; the article states that Qwen3-4B the average speedup across seven tests reached 6.88 times.

Article

Ant Group’s UBTree improves speculative decoding: parallel draft-tree selection lets large models verify more at once Token Dongcha Beating AI news: Ant Group’s Bailing has open-sourced the UBTree speculative decoding solution for large models.

Speculative decoding is like having a small model prepare a “draft” in advance, then having a large model verify multiple Token at once, reducing the number of computation rounds required to generate Token.

But drafts also have an accuracy problem. The earlier the small model makes a mistake, the less content can be directly accepted afterward. Existing techniques prepare multiple candidate paths simultaneously, organize them into a draft tree, and then prune branches that are unlikely to be accepted.

UBTree improves the selection method. It first predicts in parallel the possible Token at several subsequent positions, then uses a lightweight model to determine whether adjacent Token can connect reasonably. Different combinations can be scored simultaneously, eliminating the need to repeatedly run the model along each path. The more promising branches are then selected, assembled into a draft tree, and sent to the large model for verification.

The team also adjusted the training method for the selection model so that it focuses not only on the highest-probability answer but can also identify alternative paths worth retaining, giving the large model an opportunity to accept more Token in each round.

Across seven tests in Qwen3-4B, the average speedup from building the tree with the DFlash 2 selector was 5.95 times. After switching to the UBTree selector and adjusting the training method, this increased to 6.88 times, equivalent to a further improvement of approximately 16% over the original approach.

Original link https://m.theblockbeats.info/flash/374374

Key points

Ant Bailing has open-sourced UBTree, a speculative decoding solution for large models.
UBTree first predicts in parallel the tokens that may appear at multiple positions Token, then uses a lightweight model to determine whether adjacent Token can be connected, while simultaneously scoring different combinations.
The selector model's training approach was adjusted to simultaneously identify alternative paths worth retaining.
Across Qwen3-4B seven tests, the average speedup of DFlash 2 selector tree construction was 5.95 times, while that of the UBTree selector was 6.88 times.
The text states that UBTree further improves by approximately compared with the original solution 16%。

AI-assisted interpretation

The following is analysis, separate from reported facts. Verify important claims independently.

Speculative decoding allows a small model to generate a draft first, which the large model then checks multiple Token. UBTree attempts to filter the draft tree more quickly while retaining valuable alternative paths, allowing the large model to potentially accept more in each round Token。

Why it matters to readers

If the test results reported in the article can be reproduced, this type of method may reduce the number of computation rounds required for large-model generation, thereby reducing inference latency or computational costs.

It demonstrates one way to improve the runtime speed of large models: rather than having the model generate tokens one by one, first prepare multiple candidates in parallel and then verify them collectively.

Risks and unknowns

The information comes from a media brief paraphrase; the original text did not provide links to the paper, code repository, or the team's original announcement.
The article reported only Qwen3-4B seven tests; the effects on other models, tasks, and hardware are unknown.
Whether an independent team can reproduce it 6.88 times the average speedup, not yet certain.
It remains unclear whether UBTree has been adopted by production inference services or open-source inference frameworks.

Ant Group open-sources the UBTree speculative decoding solution, with average test speedup increased to 6.88 times

Article

AI-assisted interpretation

Why it matters to readers

Risks and unknowns

Related Developments

Related concepts

Keep following

Security Center