Zhipu releases specialized legal model GLM-Legal, saying its pass rate rose to 19.2%
BIBIBI
AT A GLANCE
Zhipu releases specialized legal model GLM-Legal; reports indicate advantages in pass rate and cost, but longer average completion times.
Article
Zhipu’s legal model nearly triples its pass rate, while testing costs only 70 one-third that of comparable competitors
Dongcha Beating AI news reports that Zhipu has released a specialized legal model GLM-Legal, further trained based on GLM-5.3-Flash.
In an evaluation covering 24 types of legal services and Harvey LAB tasks, its task pass rate rose from 6.7%to 19.2%—nearly triple the original version and ahead of models such as Kimi K3、Claude Fable 5—placing seventh on the Vals AI leaderboard.
Zhipu demonstrated several practical use cases. When reviewing a software services agreement,GLM-Legal recommended shortening the termination notice period from 60 days to 30 days and requiring the service provider to provide up to 90 days of transition support. In another software project dispute involving 40 documents, it found that the company reported the project as completed to the client 60%while internal records showed only approximately 41%. It also identified CEO emails instructing staff to conceal risks and continue charging fees, and analyzed potential legal liabilities in conjunction with the contract terms.
According to an independent evaluation by Vals AI,GLM-Legal requires only 0.15 USD to complete a test, while the similarly performing Gemini 4 Argon requires 10.55 USD—a cost difference of approximately 70 times. However, the average time taken by GLM-Legal is 72 minutes, considerably slower than the latter’s 44 minutes.
Zhipu said the model underwent specialized training on legal data and reinforcement learning in tens of thousands of simulated legal environments to improve its ability to complete complex tasks.
Original link https://m.theblockbeats.info/flash/371482
Key points
01
Zhipu releases specialized legal model GLM-Legal, further trained based on GLM-5.3-Flash.
02
The report says that, in an evaluation covering 24 types of legal services and Harvey LAB tasks,GLM-Legal’s task pass rate rose from 6.7%to 19.2%, placing it seventh on the Vals AI leaderboard.
03
In the contract review case demonstrated by Zhipu,GLM-Legal recommended shortening the termination notice period from 60 days to 30 days and requiring the service provider to provide up to 90 days of transition support.
04
The report cited an independent evaluation by Vals AI saying that GLM-Legal requires 0.15 USD，Gemini 4 Argon requires 10.55 USD, a cost difference of approximately 70 times.
05
GLM-Legal’s average completion time is 72 minutes, while Gemini 4 Argon’s is 44 minutes.
06
Zhipu said the model underwent specialized training on legal data and reinforcement learning in tens of thousands of simulated legal environments.
07
Currently,GLM-Legal requires an application for API trial access.
AI-assisted interpretation
The following is analysis, separate from reported facts. Verify important claims independently.
This is a specialized AI model for legal tasks. The evaluation results reported show that its task pass rate rose from 6.7%to 19.2%, while its per-test cost was also lower than that of the similarly performing Gemini 4 Argon; the trade-off is a longer average completion time. Zhipu also used contract review and software project dispute cases to demonstrate how the model handles legal materials.
Why it matters to readers
If the evaluation results reported can be reproduced consistently,GLM-Legal may handle some complex legal tasks at lower testing costs; however, 19.2%’s task pass rate also indicates that most tasks in the evaluation still failed, so suitability for real legal work cannot be determined solely from its cost advantage.
Beginners should focus on comparing three metrics: task pass rate, per-test cost, and average completion time. Here,GLM-Legal has lower costs but is slower, and its pass rate remains below. Applying for API trial access does not mean the model is already broadly publicly available.
Risks and unknowns
The original article does not provide the Harvey LAB or Vals AI complete evaluation methodology, samples, or links to the original results, so the relevant figures cannot be independently verified on this basis.
The application cases demonstrated by Zhipu are materials from the provider, and the original article does not state whether they underwent third-party verification.
The original article does not explain GLM-Legal’s accuracy, hallucination rate, privacy protection, or compliance performance in real-world legal operations.
Related Developments
Loading event timeline…
The original article does not specify the API trial-access scope, pricing, or official launch date.
Comparisons such as “ahead of Kimi K3、Claude Fable 5” appear only in the report’s wording and lack detailed breakdown data.
Related concepts
Task pass rate
This term is not in the glossary yet. Browse related concepts in the glossary.
