The European Union is well-known for its stringent privacy regulations. However, a legislative initiative aimed at tackling child exploitation, first introduced in May 2022, poses a significant risk to the privacy and safety of millions of users across messaging platforms in the region.
The European Commission, the legislative arm of the EU behind the proposed legislation, presents it as a protective measure for children’s rights online, focusing on the use of popular technological tools by child predators who allegedly leverage messaging apps to share child sexual abuse material (CSAM) and lure potential victims.
Possibly influenced by pressures from child safety technology advocates, the EU’s approach is decidedly techno-solutionist. The Commission’s proposal mandates that digital services, particularly messaging applications, adopt legal obligations to employ technology that scans users’ communications to identify and report unlawful activities.
For a number of years, leading messaging services have enjoyed a temporary exemption from the EU’s ePrivacy laws, which concern digital communication confidentiality. This exemption, extended until May 2025, allowed them to voluntarily monitor communications for CSAM under certain conditions.
In contrast, the proposed child abuse regulation would establish permanent protocols that obligate AI-driven content scanning across the EU.
Opponents of the proposal contend that it would obligate messaging services to deploy imperfect technologies for default user communication scanning—potentially resulting in severe privacy breaches. They express concerns that this legislation may conflict with strong encryption, as it would necessitate that end-to-end encrypted (E2EE) applications weaken their security measures to meet content monitoring requirements.
The apprehensions surrounding the proposal are so significant that the EU’s own data protection authority cautioned last year that it could mark a pivotal moment for democratic freedoms. A legal advisory group to the European Council has also indicated that it could contradict EU laws, according to a leaked assessment. Since EU law bars the imposition of universal monitoring obligations, if the legislation moves forward, it is likely to face legal opposition.
Currently, the EU’s co-legislators have not reached a consensus on how to proceed with the legislation. Nevertheless, the draft law remains active, as do the myriad risks it entails.
Broad CSAM Detection Mandates
The Commission’s initial proposal includes a stipulation that platforms, upon receiving a detection order, must scan user messages not only for known CSAM (i.e., previously identified abusive images with designated hashes) but also for unidentified CSAM (new images of abuse). This greatly increases the technical challenges involved in accurately detecting illicit content while minimizing false positives.
Additionally, the Commission’s framework specifies that platforms must detect grooming behavior in real-time. This implies that, besides image uploads for CSAM detection, apps would need to analyze user communications to ascertain when an adult might attempt to entice a minor into sexual activities.
Employing automated tools to identify behaviors that may indicate future abuse across general interactions among app users opens the door for significant misinterpretation of innocuous conversations. Together, the Commission’s expansive CSAM detection mandates could convert mainstream messaging services into instruments of mass surveillance, according to critics of the proposal.
The term “chat control” has emerged to encapsulate the anxieties regarding the EU potentially enacting a law that would enforce comprehensive scanning of private digital communications, extending to the scrutiny of text messages exchanged by users.
Concerns About End-to-End Encryption
The original proposal from the Commission does not exempt E2EE platforms from the CSAM screening requirements.
It’s evident that due to the nature of E2EE, where platforms cannot access user communications in a readable format—as they do not hold the encryption keys—secure messaging services would encounter compliance hurdles if they are mandated to decipher content they cannot view.
Critics of the EU’s initiative warn that this law could compel E2EE messaging platforms to compromise their top-notch security features by resorting to risky solutions such as client-side scanning for compliance.
The Commission’s proposal does not specify the technologies that should be employed for CSAM detection. Such decisions would be delegated to a newly established EU center for combating child sexual abuse mandated by the new law. Nevertheless, experts predict that it is likely to promote the usage of client-side scanning.
An additional consequence could see platforms that have embraced strong encryption opting to exit the EU market entirely; for instance, Signal Messenger has previously indicated it would withdraw from a market rather than be compelled to compromise user security by law. This scenario could leave EU residents without access to leading messaging apps that utilize state-of-the-art E2EE protocols, including Signal, Meta-owned WhatsApp, and Apple’s iMessage.
Opponents of the measures proposed by the EU argue that none of their intended actions will effectively prevent child abuse. Instead, they foresee severely adverse effects on app users, as the private conversations of millions of Europeans would be subject to imperfect scanning algorithms.
This poses the risk of generating numerous false positives; millions of innocent individuals might be wrongly implicated in illicit activities, overwhelming law enforcement with a flood of erroneous reports.
The system envisioned by the EU would routinely require citizens’ private messages to be exposed to third parties tasked with reviewing suspicious content reports sent to them by platforms’ detection systems. Consequently, even if a flagged piece of content isn’t ultimately forwarded to law enforcement for further investigation, it would nonetheless have been examined by individuals other than the original sender and their intended recipients. Thus, communications privacy would be severely compromised.
Safeguarding personal communications extracted from various platforms would continue to represent a security threat, as the risk of reported content being further released is prevalent if third parties do not adhere to sufficient security protocols.
Individuals opt for E2EE for specific reasons, and minimizing the number of intermediaries handling their data is a primary concern.
Where Does This Concerning Proposal Stand Now?
Generally, EU lawmaking involves a tripartite process, wherein the Commission introduces legislation while its co-legislators in the European Parliament and Council collaborate with the bloc’s executive to draft a mutually agreeable compromise.
However, regarding the child abuse regulation, EU institutions have to date held significantly different positions on the proposal.
A year ago, members of the European Parliament reached an agreement on their negotiating stance by proposing extensive amendments to the Commission’s original draft. Lawmakers from various political backgrounds endorsed substantial modifications intended to mitigate rights risks, including advocating for a full exemption for E2EE platforms from scanning mandates.
They also suggested narrowing the scanning requirements to specifically target the messages of individuals or groups suspected of child sexual abuse—rather than enforcing universal scanning on all users once a platform receives a detection order.
Another alteration supported by MEPs would confine the detection focus to known and unknown CSAM only, eliminating the requirement for platforms to detect grooming behaviors through text monitoring.
The parliament’s version of the legislation also advocated for additional measures, such as mandating platforms to enhance user privacy protections by defaulting profiles to non-public, thereby reducing the chances of minors being targeted by predatory adults.
Overall, the MEPs’ approach appears significantly more balanced than the Commission’s initial proposition. However, since then, EU elections have altered the composition of the parliament, and the stance of the newly elected MEPs is not yet clear.
Furthermore, the European Council, comprising representatives from member states’ governments, still has to establish a negotiating mandate on this matter, which explains the lack of progress in discussions with the parliament.
Those choosing privacy could be relegated to a rudimentary communication model comprising only basic text and audio functions. Yes, this is indeed what lawmakers in the region have been contemplating.
Despite MEPs’ requests last year to harmonize positions with their proposed compromises, the Council has instead appeared to support a stance that aligns more closely with the Commission’s initial “scan everything” premise. Yet divisions among member states regarding how to advance remain apparent, and enough countries have pushed back against compromise texts presented by the Council presidency to obstruct agreement on a mandate.
Leaked proposals from Council discussions indicate that member state governments continue to advocate for the ability to conduct blanket scanning of content. However, a compromise draft in May 2024 sought to reframe this mandate, referring to the legal obligation imposed on messaging platforms as “upload moderation.”
This prompted public criticism from Signal’s president, Meredith Whittaker, who accused EU lawmakers of engaging in “rhetorical manipulations” to garner support for mass scanning of citizens’ communications—a measure she cautioned would ultimately “fundamentally undermine encryption.”
According to leaked text, it was also proposed that messaging app users could be asked for consent regarding content scanning. However, those who declined would face restrictions on key app functionalities, preventing them from sending images or URLs.
Under this arrangement, users of messaging apps in the EU would essentially be compelled to make a choice between safeguarding their privacy or enjoying a contemporary app experience. Those opting for privacy may find themselves relegated to a fundamental, dumb-phone-style communication system limited to text and audio. Yes, this is indeed a consideration under discussion among regional legislators.
However, there are indications that support for widespread surveillance of citizens’ messaging may be waning within the Council. Recently, Netzpolitik highlighted a statement from the Dutch government, which indicated it would abstain from a newly revised compromise, citing concerns about the implications for E2EE and the security risks tied to client-side scanning.
Additionally, earlier this month, discussions on the regulation were withdrawn from another Council agenda, reportedly due to the absence of a qualified majority.
Nevertheless, a considerable number of EU nations continue to support the Commission’s initiative for comprehensive message scanning. The current Hungarian Council presidency remains dedicated to pursuing a compromise, ensuring that the risk of such legislation has not dissipated.
It is still possible for member states to craft a proposal version that satisfies sufficient governmental interests to initiate talks with MEPs, which would subsequently enable interactions within the EU’s closed-door trilogue discussion framework. As such, the stakes for the rights of European citizens—and the EU’s reputation as a protector of privacy—remain substantial.
Compiled by Techarena.au.
Fanpage: TechArena.au
Watch more about AI – Artificial Intelligence
