reinforcement learning from human feedback

RLHF aligns a model with human preferences in stages: train a reward model on ranked responses, then optimize the policy against that reward with reinforcement learning. PPO is a common optimizer but not required — preference methods like DPO reach similar goals without an explicit RL loop. In practice it shapes refusal style, helpfulness, and instruction adherence.