OpenAI ModerationOpenAI
|
ShieldstralMistral AI
|
|||||
Related Products
|
||||||
About
The OpenAI Moderation API provides developers with a dedicated endpoint to automatically evaluate whether text or images contain potentially harmful or policy-violating content, enabling safer AI applications through real-time filtering and classification. It works by analyzing inputs (and optionally outputs) and returning structured results that indicate whether the content is flagged, along with detailed category labels such as hate, harassment, self-harm, sexual content, or violence. It is designed to be integrated directly into application workflows, allowing developers to take immediate action, such as blocking, filtering, or escalating content, before it reaches end users. Moderation models like “omni-moderation-latest” are optimized for speed and accuracy, supporting scalable use across high-volume applications while maintaining consistent safety standards.
|
About
Shieldstral is a 3B open-weights, policy-adaptive multimodal safety classifier designed to evaluate text, images, and text-plus-image content using policies defined at inference time. Instead of relying on a fixed taxonomy of harm categories, it frames moderation as a binary question-answering task: users provide an instruction describing the evaluation context and strictness, a yes-or-no safety question, and the content to judge. The model reads the “yes” and “no” logits and converts them into a continuous, calibrated safety score, allowing applications to threshold or rank results by confidence rather than depend on a single discrete label. This formulation unifies prompt classification, response moderation, refusal detection, toxicity detection, and multimodal safety in one interface, while letting teams adapt policies without retraining the model. Shieldstral can evaluate prompts, responses, prompt-response pairs, images, and images with accompanying text.
|
|||||
Platforms Supported
Windows
Mac
Linux
Cloud
On-Premises
iPhone
iPad
Android
Chromebook
|
Platforms Supported
Windows
Mac
Linux
Cloud
On-Premises
iPhone
iPad
Android
Chromebook
|
|||||
Audience
Developers building AI applications who need to detect and manage harmful content to ensure safe and policy-compliant user interactions
|
Audience
AI platform teams that need customizable, multimodal content moderation without retraining a separate safety model for every policy
|
|||||
Support
Phone Support
24/7 Live Support
Online
|
Support
Phone Support
24/7 Live Support
Online
|
|||||
API
Offers API
|
API
Offers API
|
|||||
Screenshots and Videos |
Screenshots and Videos |
|||||
Pricing
Free
Free Version
Free Trial
|
Pricing
No information available.
Free Version
Free Trial
|
|||||
Reviews/
|
Reviews/
|
|||||
Training
Documentation
Webinars
Live Online
In Person
|
Training
Documentation
Webinars
Live Online
In Person
|
|||||
Company InformationOpenAI
Founded: 2015
United States
developers.openai.com/api/docs/guides/moderation
|
Company InformationMistral AI
Founded: 2023
France
mistral.ai/news/shieldstral/
|
|||||
Alternatives |
Alternatives |
|||||
|
|
||||||
|
|
|
|||||
Categories |
Categories |
|||||
|
|
|