Safety-Prompts

Safety-Prompts is an open-source repository that provides a curated collection of prompts designed to evaluate and improve the safety behavior of large language models. The project focuses primarily on safety testing scenarios relevant to Chinese language models, though the concepts can be applied to other languages and systems. The prompts are structured to test whether models generate outputs that align with human values and safety guidelines when faced with potentially harmful or sensitive requests. Researchers and developers use the dataset to benchmark how well models avoid unsafe responses and follow alignment constraints. The repository also serves as a training resource for improving model alignment by providing examples of prompts that require safe reasoning and appropriate refusal behavior. In addition to evaluation prompts, the project references related tools and benchmarks for assessing model safety across different contexts.