Perplexity and NVIDIA proposed JOLT a self-distillation training method

Article

Perplexity improves AI Self-distillation: training not only the student, but also the teacher

Dongcha Beating AI news: in self-distillation training for large models, the same model plays roles: the student solves problems independently, while the teacher can see the reference answer in advance and follows the content written by the student, step by step Token to provide guidance.

The roles share parameters, eliminating the need to run an additional, larger teacher model, while also obtaining training signals more granular than final scoring. However, the additional information available to the teacher may lead it to adopt reasoning paths that the student cannot complete independently, thereby hindering learning.

This year,June，UCLA and other teams proposed SDPG, retaining final-outcome rewards while allowing only solution processes that perform above their group average to receive distillation guidance, then gradually reducing the distillation weight later in training to prevent excessive imitation from hindering the model's own exploration. If all attempts in a group fail, the distillation signal is turned off; the same applies when all answers are correct.

Perplexity and NVIDIA newly proposed JOLT, which also brings the teacher into training. The teacher independently solves problems using additional information and receives rewards based on the final outcome; meanwhile,KL divergence constrains the output difference between teacher and student, preventing guidance from deviating too far from the student's current capabilities. The student still follows its own solution process, step by step Token receiving teacher feedback. The roles share parameters, and problem-solving ability and teaching approach are adjusted together during training.

In the Qwen3-0.6B MATH test,JOLT accuracy reached 72.14%, higher than the OPSD of the prior self-distillation method 61.20%, and also exceeding the GRPO of reinforcement learning 68.36%. This JOLT experiment did not separately add a final-outcome reward for the student; only the teacher branch directly used the outcome reward.

At the same 63.44%accuracy,GRPO actually consumed approximately 8400000 training Token，JOLT is estimated from the training curve to require only 1300000, a difference of approximately 6.5 times.

However, when switched to the AppWorld agent task,JOLT without a student outcome reward was only 51.92%, lower than the GRPO of 62.20%. After also adding the final task reward for the student,JOLT+ increased only to 72.09%. The teacher's step-by-step guidance still cannot fully replace feedback from the student actually completing the task.

Key points

In the described large-model self-distillation training, the student solves problems independently, while the teacher sees the reference answer in advance and follows the student's content, step by step Token to provide guidance.
The roles share parameters, eliminating the need to run an additional, larger teacher model.
The report states that this year June，UCLA and other teams proposed SDPG, allowing only solution processes that perform above their group average to receive distillation guidance, and gradually reducing the distillation weight later in training.
Perplexity and NVIDIA proposed JOLT, incorporating the teacher into training as well and using KL divergence to constrain output differences between teacher and student.
In the Qwen3-0.6B MATH test,JOLT accuracy was 72.14%，OPSD was 61.20%，GRPO was 68.36%。
At 63.44%accuracy,GRPO actually consumed approximately 8400000 training Token；JOLT is estimated from the training curve to require 1300000, a difference of approximately 6.5 times.
In the AppWorld task,JOLT without a student outcome reward was 51.92%，GRPO was 62.20%; after adding the student's final task reward,JOLT+ was 72.09%。

AI-assisted interpretation

The following is analysis, separate from reported facts. Verify important claims independently.

JOLT Have the same model play both student and teacher: the student solves problems according to its own process, while the teacher solves problems using additional information and provides step-by-step feedback. Training also limits the difference between their outputs to prevent the teacher's guidance from being too far beyond the student's current capabilities.

Why it matters to readers

The reported test results show that JOLT achieved higher accuracy than MATH in the test,OPSD and GRPO, and the estimated training Token required to reach the same accuracy was also lower; however, the AppWorld results also show that the teacher's step-by-step guidance cannot fully replace the final reward after the student actually completes the task.

Beginners should distinguish between types of feedback: the teacher's step-by-step Token process guidance and the final outcome reward after task completion. The AppWorld data in the evidence indicate that the may need to be used together.

Risks and unknowns

The provided evidence does not include the paper, code, or complete experimental setup, so the method details and test results cannot be independently verified.
JOLT The 1300000 training Token is estimated from the training curve, whereas the GRPO 8400000 is an actual measured value; the use different statistical methods.
The evidence only lists Qwen3-0. 6B of MATH testing and AppWorld tasks, so it cannot confirm the method's performance at other model scales or on other tasks.

Perplexity and NVIDIA proposed JOLT a self-distillation training method

Article

AI-assisted interpretation

Why it matters to readers

Risks and unknowns

Related Developments

Related concepts

Keep following

Security Center