OpenAI is enhancing its commitment to transparency by regularly publishing the results of its internal safety evaluations for AI models. On Wednesday, the company unveiled the Safety Evaluations Hub, a new platform where users can explore how its models perform on tests related to harmful content generation, jailbreaks, and hallucinations. OpenAI plans to update this hub periodically, especially following significant model updates.
In a recent blog post, OpenAI stated, “As the science of AI evaluation evolves, we aim to share our progress on developing more scalable ways to measure model capability and safety.” The intention behind the hub is to simplify the understanding of OpenAI systems’ safety performance while also promoting greater transparency across the AI community.
OpenAI has also indicated plans to expand the types of evaluations available in the hub over time. However, the company has faced criticism from ethicists recently for allegedly hurrying the safety testing of key models and not releasing comprehensive technical reports for others. There are allegations against OpenAI’s CEO, Sam Altman, claiming he misled executives regarding model safety reviews shortly before being temporarily ousted in November 2023.
Additionally, OpenAI had to retract an update for its ChatGPT default model, GPT-4o, after users reported issues with its overly agreeable responses. Social media channels were flooded with screenshots showing ChatGPT affirming several harmful ideas and decisions. In response, the company announced it would implement several fixes, including the introduction of an opt-in “alpha phase” that allows select users to test new models and provide feedback before they are launched.
Overall, OpenAI’s initiative to make safety metrics more accessible represents a step towards fostering trust and accountability in AI development, even as the company navigates the complexities and ethical challenges of its technology.
Fanpage:Â TechArena.au
Watch more about AI – Artificial Intelligence
