StepFun open-sources Step 5 Preview 600B model weights, supporting million Token context
BIBIBI
AT A GLANCE
StepFun open-sourced Step 5 Preview weights and inference code, stating that it supports 1000000 Token context.
Article
StepFun Step 5 Open source 600B Weights: 92 deep architecture, million Token Indexing overhead reduced to 1/8 Beating AI news: StepFun has open-sourced Step 5 Preview model weights,FP8 and BF16 versions are now available on Hugging Face and ModelScope, with the accompanying inference code also released.Step 5 Preview Total parameters: 600000000000, with only approximately 27000000000 parameters activated per generation, supporting 1000000 Token context. A newly published technical blog reveals that the model uses a 92-layer “narrow and deep” architecture, allowing information to pass through more layers to support multi-step reasoning in complex tasks. It also introduces Sparse GQA, which merges adjacent 8 Token into a single index, reducing the overhead of indexing and Top-k filtering to Token of that of the per-1/8 during the reinforcement learning phase, enabling Agent to organize progress when tasks run too long and then continue execution.
The model uses a customized StepFun Model License 1.0, which permits general commercial use. However, hardware manufacturers such as mobile phone and automobile companies that integrate the model’s capabilities into commercial products must sign a separate license agreement. Enterprises operating MaaS(model-as-a-service) businesses whose own and affiliated companies’ total revenue exceeds 12 over a consecutive period of 20000000 USD months must also obtain a separate commercial license.
Original link https://m.theblockbeats.info/flash/371938
Key points
01
StepFun has open-sourced the Step 5 Preview model weights.
02
FP8 and BF16 versions are now available on Hugging Face and ModelScope, with the accompanying inference code also released.
03
Step 5 Preview Total parameters: 600000000000, with only approximately 27000000000 parameters activated per generation, supporting 1000000 Token context.
04
The technical blog reveals that the model uses a 92-layer “narrow and deep” architecture.
05
The model introduces Sparse GQA, merging adjacent 8 Token into a single index, reducing the overhead of indexing and Top-k filtering to Token of that of the per-1/8。
06
During reinforcement learning, the team incorporated context compression into training, enabling Agent to organize progress and continue execution when tasks run too long.
07
StepFun Model License 1.0 It permits general commercial use, but hardware manufacturers such as mobile phone and automobile companies that integrate the model’s capabilities into commercial products must sign a separate license agreement.
08
Enterprises operating MaaS businesses whose own and affiliated companies’ total revenue exceeds 12 over a consecutive period of 20000000 USD months must obtain a separate commercial license.
AI-assisted interpretation
The following is analysis, separate from reported facts. Verify important claims independently.
This is a report that the model weights and inference code have been made public. The model does not use all 600000000000 parameters for every generation, but activates approximately 27000000000 parameters; it claims to support a maximum context of 1000000 Token.Sparse GQA By merging adjacent 8 Token into a single index, it reduces indexing and filtering overhead. The model may be used commercially in general, but certain hardware manufacturers and MaaS enterprises meeting the revenue criteria require additional licenses.
Why it matters to readers
After the weights and inference code were released, developers can try deploying and studying the model; its million Token context, sparse indexing design, and commercial licensing restrictions relate respectively to long-text processing, inference overhead, and conditions for commercial use.
Beginners should distinguish between “total parameters 600000000000” and “approximately 27000000000 parameters activated per generation,” and verify the model files, runtime requirements, and formal license terms before deployment or commercial use.
Risks and unknowns
The current evidence consists only of a news brief; no Hugging Face or ModelScope page or technical blog text was provided for verification.
There is no third-party verification information on model performance, actual inference costs, or 1000000 Token context capability.
StepFun Model License 1.0 The complete terms, additional licensing conditions, and fees for have not been specified in the evidence.
Related Developments
Loading event timeline…
Related concepts
FP8
This term is not in the glossary yet. Browse related concepts in the glossary.
