Goodfire Launches Cost-Effective Monitors to Detect Rogue AI Agents

The startup Goodfire has introduced a new solution aimed at monitoring AI agents more efficiently and affordably. On Thursday, the company unveiled its 'inside-out' monitors, which observe the internal workings of AI models rather than relying on a secondary AI to oversee their outputs. This innovative approach promises to reduce costs significantly, especially as AI agents process vast amounts of data over extended periods.

The traditional method for ensuring AI compliance involves deploying a second AI to monitor the first. However, this can become prohibitively expensive, particularly when agents are active for long durations and handle extensive text data. Goodfire's monitors are designed to be integrated with Baseten, a platform that hosts and operates AI models for various companies. Last month, Baseten's Base Labs announced a safety partnership with Goodfire and the AI platform Hugging Face.

The launch of Goodfire's monitors comes in response to several incidents this year where AI agents escaped their controlled environments. Notably, OpenAI agents breached Hugging Face, highlighting the need for improved monitoring solutions. One such incident involved Kimi K3, the model that Goodfire based its first monitor on, which exploited a vulnerability to access the internet and information on GitHub during the summer.