Xplore release RWE-Bench：GPT-6 Astra single-attempt pass rate 37%，Claude Opus 5.5 rises to after multiple attempts 70%

Article

AI Begin challenging real-world engineering design:GPT-6 Astra pass rate 37%，5 models scored zero

Beating AI breaking news,AI startup Xplore released RWE-Bench, testing large models’ ability to solve real-world engineering problems. 11 models challenged 20 mechanical, electrical, and aerospace tasks,GPT-6 Astra ranked first with 37%average pass rate per single attempt, while Claude Opus 5.5 was 29%，GPT-6.1 Sol was 25%. Another 5 models failed all tasks, including DeepSeek V4.1 Flash、GLM-5.3 and Grok 4.7.

The tasks covered engine crankshaft design, circuit-board routing, and turbine-blade heat-dissipation simulation. Crankshaft design required balancing size, weight, strength, and fatigue life; routing an eight-layer circuit board with nearly 300 components already installed required connecting 71 signal lines while controlling timing error and impedance; and simulating turbine-blade heat dissipation required correctly calculating heat transfer between high-temperature airflow and the blades.

Each model used its corresponding Agent toolchain to operate specialized engineering software directly. Each task could run for at most 24 hours, with 5 independent attempts. Submitted designs were automatically verified by engineering software and physics solvers; failure to meet any hard requirement counted as a failure. Across all 1100 tests, 4 tasks remained unsolved. However, after allowing 5 attempts per task, Claude Opus 5.5 task pass rate rose to 70%, surpassing Astra’s 60%.

Xplore hopes to further turn these engineering tasks into reinforcement-learning training environments, allowing AI to improve their engineering capabilities through repeated design, simulation, and error correction.

Original link https://m.theblockbeats.info/flash/374164

Key points

Xplore released RWE-Bench, designed to test large models’ ability to solve real-world engineering problems.
11 models challenged 20 mechanical, electrical, and aerospace tasks.
In the single-attempt average test,GPT-6 Astra’s pass rate 37%，Claude Opus 5.5 was 29%，GPT-6.1 Sol’s was 25%。
After allowing 5 attempts per task, Claude Opus 5.5 pass rate rose to 70%，GPT-6 Astra’s was 60%。
5 models failed all tasks; the article lists DeepSeek V4.1 Flash、GLM-5.3 and Grok 4.7。
Tasks included engine crankshaft design, circuit-board routing, and turbine-blade heat-dissipation simulation.

AI-assisted interpretation

The following is analysis, separate from reported facts. Verify important claims independently.

RWE-Bench The models directly use specialized engineering software to complete designs, and the results are automatically checked by engineering software and physics solvers. A single attempt measures the initial success rate, while multiple attempts indicate whether a model can improve its results through repeated design, simulation, and error correction.

Why it matters to readers

This shows that different models perform differently on real-world engineering tasks and also indicates that multiple attempts may change the rankings.

This is not merely a test of question answering; it requires models to complete design, simulation, and validation, making it closer to an actual engineering workflow.

Risks and unknowns

The information comes from a media brief and does not provide Xplore’s original benchmark report or complete test records.
The article does not list all 11 models or all 20 tasks.
37%、70%and 60%respective pass rates have not been independently reproduced based on the evidence.
It remains unclear whether the evaluation rules for multiple attempts represent actual deployment conditions.

Xplore release RWE-Bench：GPT-6 Astra single-attempt pass rate 37%，Claude Opus 5.5 rises to after multiple attempts 70%

Article

AI-assisted interpretation

Why it matters to readers

Risks and unknowns

Related Developments

Related concepts

Keep following

Security Center