Download
Safety-Prompts is an open-source repository that provides a curated collection of prompts designed to evaluate and improve the safety behavior of large language models. The project focuses primarily on safety testing scenarios relevant to Chinese language models, though the concepts can be applied to other languages and systems. The prompts are structured to test whether models generate outputs that align with human values and safety guidelines when faced with potentially harmful or sensitive requests. Researchers and developers use the dataset to benchmark how well models avoid unsafe responses and follow alignment constraints. The repository also serves as a training resource for improving model alignment by providing examples of prompts that require safe reasoning and appropriate refusal behavior. In addition to evaluation prompts, the project references related tools and benchmarks for assessing model safety across different contexts.
Features
- Collection of prompts designed for evaluating LLM safety behavior
- Focus on alignment with human values and responsible responses
- Dataset for testing Chinese language models and multilingual safety scenarios
- Examples useful for training or fine-tuning safer language models
- Integration with related safety evaluation frameworks and research tools
- Benchmarking resources for analyzing unsafe or harmful model outputs
Project Samples
