It is almost unheard of for an AI company to sue its own users, but Elon Musk's xAI has set a precedent. According to Reuters, xAI has taken legal action against a man from South Carolina, Terry Harwood, accusing him of using the Grok model as a tool for criminal activities.

Harwood was arrested in February this year on suspicion of exploiting minors. xAI alleges in its complaint that he abused Grok to generate child sexual abuse material. On Tuesday local time, xAI formally filed the complaint with a federal court in Texas, charging Harwood with violating the service terms. Before this, there have been very few cases where AI companies have sued users for producing pornographic content using their own models.

The timing of this lawsuit is rather delicate. xAI is currently under global scrutiny due to repeated failures in the content safety measures of Grok. The external accusations point to a sharp question: Grok allows users to create non-consensual pornographic deepfake content. Those fake videos, which look real, are quietly eroding the boundaries of real individuals.

In the complaint, xAI also took a defensive stance. The company stated that it takes measures such as suspending or terminating accounts that violate the rules, and reports suspected child sexual abuse materials to the National Center for Missing & Exploited Children to enforce platform policies. A set of figures was directly presented in court: since 2026, xAI has suspended 52,222 accounts, submitted 73,604 reports to the center, and facilitated the arrest of at least 244 people.

Regarding Harwood himself, according to IT Home, xAI accuses him of uploading ordinary photos of adults and minors to Grok, then attempting to generate pornographic deepfakes featuring these individuals. At the same time, he is accused of creating pornographic images without the consent of the adults involved. In other words, his hands have touched lines they should not have, and also violated the principle of informed consent for adults.

xAI has made clear demands to the court: to order Harwood to pay an unspecified amount of compensation and permanently prohibit him from using Grok. The wording in the complaint is unyielding - the defendant intentionally used the plaintiff's tool as a weapon for crime, causing serious and lasting harm to real victims, and exposing the plaintiff to significant legal risks and reputational losses.

When an AI company has to take the position of a plaintiff to protect those harmed by its own model, this incident itself serves as a reminder to the entire industry: the boundaries of model capabilities should never be defined by abusers.