Safety Training — AI Glossary

Overview

Safety training encompasses: supervised fine-tuning on safe demonstrations, RLHF with human feedback penalizing harmful outputs, Constitutional AI (Anthropic's self-critique method), and red-teaming-informed data collection. Safety-capability trade-offs are actively researched; overly aggressive safety training causes refusal of benign requests ('over-refusal').