RLVR lets the model try a task many times. A program checks each attempt (reward 1 if correct, 0 if not), and the model is trained towards its successful attempts.

Introduction

This is how today's reasoning and agent models are trained, for example GRPO in DeepSeek-R1. The same automatic checks are used to evaluate agents, see Agent Evaluation.

PieceMeaning
Reward for attempt , for example "did the order stay within the 50 dollar limit"
Grading on a curve: attempts better than the average of the group are reinforced, and worse ones are discouraged
The number of attempts in the group

Remembering tip

A teacher who grades on a curve. An attempt is rewarded for beating the other attempts of the same group.

Comparison with RLHF

PointRLHFRLVR
Reward comes fromA reward model trained on human picksA program that checks the result
Best forStyle and what people preferTasks with a checkable answer, such as code, maths and spend limits

Code

A verifiable reward for the shopping task, and the group advantage:

import numpy as np
 
def reward(order_total, limit=50.0):
    """1 if the order stayed within the limit, 0 otherwise."""
    return 1.0 if order_total <= limit else 0.0
 
totals = [47.85, 65.18, 47.85, 47.85, 65.18, 47.85]    # 6 attempts at the same task
r = np.array([reward(t) for t in totals])               # [1 0 1 1 0 1]
 
A = (r - r.mean()) / (r.std() + 1e-8)                   # grading on a curve
print(np.round(A, 2))                                   # [ 0.71 -1.41  0.71  0.71 -1.41  0.71]

Notes:

  • attempts above the group average get a positive advantage and are reinforced
  • 1e-8 avoids a division by zero when every attempt gets the same reward

After training, the model writes tool calls as JSON, see Tool Call Validation.