Download
The following-instructions-human-feedback repository contains the code and supplementary materials underpinning OpenAI’s work in training language models (InstructGPT models) that better follow user instructions through human feedback. The repo hosts the model card, sample automatic evaluation outputs, and labeling guidelines used in the process. It is explicitly tied to the “Training language models to follow instructions with human feedback” paper, and serves as a reference for how OpenAI collects annotation guidelines, runs preference comparisons, and evaluates model behaviors. The repository is not a full implementation of the entire RLHF pipeline, but rather an archival hub supporting the published research—providing transparency around evaluation and human labeling standards. It includes directories such as automatic-eval-samples (samples of model outputs on benchmark tasks) and a model-card.md that describes the InstructGPT models’ intended behavior, limitations, and biases.
Features
- Archive of evaluation sample outputs from InstructGPT experiments
- model-card.md describing model usage, limitations, and safety considerations
- Labeling guidelines / annotation instructions used for human evaluators
- Structured “automatic-eval-samples” folder showing baseline vs fine-tuned outputs
- Transparency around how OpenAI measured model preference ranking and alignment
- Links and references to the original research paper and documentation
Project Samples
