Meredith Whittaker, the President of Signal, cautioned on Friday regarding the potential risks to user privacy posed by agentic AI.
During her speech at the SXSW conference in Austin, Texas, Whittaker — a proponent of secure communications — described the adoption of AI agents as akin to “putting your brain in a jar.” She emphasized that this emerging computing model, which allows AI to execute tasks on behalf of users, presents significant privacy and security challenges.
Whittaker elaborated on how AI agents are being promoted as tools to enhance everyday life by managing various online activities for users. These agents could handle tasks such as finding concert information, securing tickets, scheduling events on your calendar, and messaging friends to inform them about bookings.
“In essence, we could put our brains in a jar, as the AI would handle everything without our need to intervene, right?” Whittaker pondered.
She then highlighted the extensive access AI agents require to facilitate these tasks, which includes permissions to our web browsers, credit card information for ticket purchases, and access to our calendars and messaging applications to communicate with friends.
“They would need to operate with something akin to root permission, allowing access to numerous databases — likely unencrypted, as there isn’t a method presently to handle that in a secure manner,” Whittaker cautioned.
“If we’re discussing a sufficiently advanced AI model driving this process, it’s improbable that it would function on a local device. More likely, it would transfer data to a cloud server for processing and then return the results. Therefore, this creates serious security and privacy concerns surrounding the hype associated with AI agents, threatening to disrupt the boundary between application layers and operating systems by merging various services and complicating data management,” she concluded.
Whittaker warned that if a messaging app like Signal were to incorporate AI agents, it could jeopardize the privacy of users’ messages. The agent would have to access the app to send messages to contacts and retrieve information to summarize those conversations.
Her statements were a continuation of her earlier comments during the panel, where she discussed how the AI sector has largely been built upon a surveillance framework reliant on extensive data collection. She noted that the “bigger is better AI paradigm” – suggesting that more data equates to better outcomes – could lead to unfavorable repercussions.
Whittaker cautioned that with the rise of agentic AI, we risk further compromising privacy and security for the sake of a “magic genie bot that promises to manage life’s complexities,” she concluded.
Compiled by Techarena.au.
Fanpage: TechArena.au
Watch more about AI – Artificial Intelligence
