Emerging from San Francisco’s AI Hackathons, Agency Gives Insight into Your AI Agents’ Activities

Following a hectic week immersed in coding, one might envision the tech aficionados of San Francisco seeking solace amidst the serenity of nearby mountains, beachfronts, or the lively nightlife. Yet, the reality unfolds differently as the weekend heralds the onset of AI hackathons.

San Francisco has become a hotbed for AI hackathons in recent times, transforming weekends into a hub where tech enthusiasts converge to discuss AI breakthroughs, forge connections, and more critically, morph their creative concepts into tangible prototypes. While cash or cloud service credits are common incentives, the true victors emerge with potentially groundbreaking startup ideas.

Alex Reibman, co-founder of Agency AI, asserts, “San Francisco is unmatched as a breeding ground for grandiose projects. The atmosphere here thrives on both collaboration and competition, evidenced by the myriad of hackathons and contests.”

During a San Francisco hackathon last summer, Reibman ventured into developing AI-powered web scraping agents. Agents, a buzzword within the tech frontier, typically refer to AI systems capable of autonomously executing tasks over platforms not initially intended for automation, effectively streamlining erstwhile manual operations.

Reibman encountered immediate challenges, noting, “They were inadequate,” with the agents failing at a concerning rate, often unpredictably.

In response, Reibman’s team devised specialized debugging software to pinpoint flaws. Eventually, the agents showed improvement, yet it was the debugging tools themselves that clinched victory at the hackathon.

Reibman recalled, “Presenting these tools at various San Francisco gatherings sparked interest, leading to requests for access. It was a clear sign that our focus should shift towards facilitating agent development rather than the direct creation of agents.”

This pivot led to the formation of Agency, co-founded with Adam Silverman and Shawn Qiu. Their venture specializes in transparency tools for understanding and troubleshooting AI agents. A year hence, their main offering, the AgentOps platform, now serves thousands of teams monthly, reveals Reibman to TechCrunch. The startup recently secured $2.6 million in pre-seed investment, spearheaded by 645 Ventures and Afore Capital.

Adam Silverman, Agency’s COO, described AgentOps to TechCrunch as akin to multi-device management for agents, ensuring they operate within designated boundaries without going awry.

“Identifying potential for rogue behavior in agents and setting limitations is crucial,” Silverman explained. “Much of our effort is visualizing these control mechanisms to guarantee compliance prior to deployment.”

Agency collaborates with Cohere and Mistral, AI model firms that also specialize in agent development, allowing for an integrated view of agent interactions and expenditures on the AgentOps dashboard. Despite being adaptable to various AI frameworks, Agency integrates with well-known tools like Microsoft’s AutoGen, crewAI, and AutoGPT.

In addition to their technical offerings, Agency also provides consultancy services, leveraging Reibman’s prior experience at EY to assist businesses in initiating their agent creation journey. While specific clients weren’t named, the company shared that its services are utilized by hedge funds, consultancy firms, and marketing agencies.

For instance, Agency has facilitated the development of an AI agent that writes blog content for client companies, with the performance and costs monitored via the AgentOps dashboard.

With industry giants like OpenAI and Google poised to expand their agent-oriented products, startups like Agency are strategizing to complement these developments rather than compete.

Reibman reflects, “Considering the multitude of layers in the tech ecosystem, it’s improbable for large-scale LLM providers to monopolize every aspect. While entities like OpenAI and Anthropic focus on agent development, we concentrate on ensuring the code base is primed for production.”