Google Can Terminate Account Based on CSAM Allegations–Baker v. Google

July 30, 2024 · by · in Content Regulation, Licensing/Contracts

The scenario: Google thought that a user uploaded CSAM and terminated her account. The user disagreed, appealed, and got nowhere. The user took the matter to court (pro se), where the lawsuit failed:

  • Contract Breach. “Plaintiff does not allege any facts indicating that Defendant was contractually prohibited from removing her Google account.”
  • Fraud. “Plaintiff offers nothing to support the elements of fraud.”
  • Constitutional Violations. “Defendant Google is a private business, not a state actor.”

This case is very similar to the uncited Deutsch v. Microsoft case from last year, which also failed.

This becomes yet another unsuccessful account termination/content removal lawsuit. Still, I don’t see this as an ordinary case. It’s a microcosm of the potential collateral damage from regulators’ relentless pressures on services to “do more” against CSAM. The services were already doing a lot, but if regulators don’t think that’s enough, the services tighten up their filters and standards and create more false positives. (We don’t know if this is a false positive, but obviously the user claims it is and cares enough to take it to court).

This case also highlights the stakes of the individualized explanations requirement in the TX and FL social media censorship laws if those survive constitutional scrutiny. Google in fact provided an individualized explanation and an appeal (apparently voluntarily), but will that explanation and appeal satisfy users who don’t get their desired outcome? Some of those users will complain to a court, questioning the accuracy of the explanation or the rigorousness of the appeal. This creates a lot of costly and distracting litigation, even if the courts ultimately reject the claims–another collateral cost of the regulatory intervention.

Note that determining if a content item is CSAM isn’t always a zero or one. The border cases leave Google caught between its legal obligations to remove and report what might be CSAM and a user’s view that the CSAM characterization was overly cautious. Google’s discretion should prevail in court, but each choice creates legal peril for Google.

Case Citation: Baker v. Google LLC, 2024 WL 3551878 (D.D.C. July 26, 2024). The complaint.

Some of the cases this situation brought to mind: