GRPOTrainer

class GRPOTrainer(cfg)

Bases: Trainer

Reinforcement-learning trainer implementing GRPO at the trainer level.

Replaces the single supervised optimization step with a rollout-based loop: each batch is sampled into a group of trajectories, then the policy is updated policy_updates_per_rollout times over the cached rollout (optionally split into trajectory microbatches to bound memory). The scheduler is sized to len(train_loader) * epoch * policy_updates_per_rollout accordingly, and scalar GRPO metrics are reduced across ranks with key-specific ops (min/max/mean). Inherits all model/loader/optimizer construction from Trainer. Registered as GRPOTrainer – select via train = dict(type="GRPOTrainer").

build_scheduler()

Build a scheduler sized to rollout count times policy updates.

run_step()

Sample rollouts, run policy updates, and publish GRPO metrics.