SGLang Adds native decision-making API, planned for official release with v0.5.21

Article

SGLang Add Jev-style decision-making to ordinary large models—make choices directly without fine-tuning

Beating AI breaking news: large-model inference/deployment framework SGLang adds native decision-making API, allowing off-the-shelf models such as Qwen3.8-LLM and multimodal models to make choices, judgments, and scores directly without modifying weights or fine-tuning again. Developers provide the current situation and several candidate answers, and the model directly returns the probability of each option instead of generating a passage of text first and then parsing it.

It uses the “next-token probability” that large models already calculate. For example, if the candidate answers are A, B, and C, an ordinary chat model will continue generating text;SGLang instead directly reads the model’s probabilities for A, B, and C to see which answer it favors. The model itself has not changed; only the way the results are read has changed.SGLang used Qwen3.8- for a Pokémon FireRed demonstration. Based on the real-time game state, the model decides whether to attack, switch Pokémon, or heal, with each decision taking under ms, and cleared the 4 Elite Four and Champion in one run. Qwen3.8- itself supports image input, so this method can also handle multimodal decision-making.SGLang also added a /v1/systemone endpoint compatible with the TypeSafe SDK used by Jev. Applications already using this SDK can change the service address to their own SGLang instance and continue making calls. The new feature has entered the nightly version and is planned for official release with v0.5.21.

Original link https://m.theblockbeats.info/flash/369767

Key points

SGLang Add native decision-making API。
This API enables off-the-shelf LLM and multimodal models to directly make choices, judgments, and scores, without changing weights or retraining.
The model can directly return the probabilities of the candidate options.
SGLang Use Qwen3.8- to demonstrate decision-making in Pokémon FireRed, with each decision taking less than 100ms.
SGLang Add the /v1/systemone interface, compatible with the one used by Jev TypeSafe SDK。
The new feature has entered the nightly version and is planned for official release with v0.5.21.

AI-assisted interpretation

The following is analysis, separate from reported facts. Verify important claims independently.

Ordinary models typically generate text first and then have a program parse the answer.SGLang Instead, read the model's next-token probabilities for the candidate answers, such as comparing which of A, B, and C has the highest probability; the model weights themselves remain unchanged.

Why it matters to readers

Developers can directly use the probabilities of candidate options for selection, judgment, and scoring, reducing the need to generate text first and then parse it.

This means that off-the-shelf large language models can also be used for simple decision-making tasks, without necessarily retraining the model.

Risks and unknowns

The evidence comes from a single-media report and does not provide SGLang an official announcement or code release record.
The official release date for v0.5.21 has not yet been clearly stated in the evidence.
The interface’s full compatibility and independent test results have not yet been confirmed in the evidence.

SGLang Adds native decision-making API, planned for official release with v0.5.21

Article

AI-assisted interpretation

Why it matters to readers

Risks and unknowns

Related Developments

Related concepts

Keep following

Security Center