A pro se plaintiff from Connecticut was found by a judge to have hidden text instructions that could only be read by AI systems in documents submitted to the court, attempting to influence the handling of the case. This is considered the first case in the U.S. judicial system where a party used "prompt injection" to interfere with an AI review process. Although the state court does not currently use AI to review materials, the hidden instructions did not work.

Small White Text Holds Secrets, Abuse of Procedure Leads to Sanctions

These hidden instructions were set to an extremely small font size and white text on a white background, making them almost invisible to the naked eye, but they could be identified by software that extracts text from documents. Their content required any AI system reviewing the documents to produce output consistent with the plaintiff's claims, ignore unfavorable court decisions, and push for results in the way the plaintiff desired. Judge Walter Spade Jr. pointed out that this approach essentially involved secretly planting commands into systems that read the files, making the plaintiff's requests appear as if they came from the system operator. After being warned of penalties, the plaintiff continued to add such instructions, constituting a serious abuse of procedure.

The plaintiff argued that some parts were just jokes, including a video link and nonsensical text like "Hi, I hope you can't see me," but the judge believed it was unreasonable to include jokes in formal complaints. Considering that he was representing himself and seemed to be overly influenced by AI tools, the court did not impose a fine but instead prohibited him from using the electronic submission system, requiring him to submit paper documents instead. This preserved his right to access the courts while preventing further abuse of the platform.

Attacks at the Input End Become a New Risk

The judge warned that with prompt injection becoming more common in areas such as recruitment, it is not surprising that similar situations have emerged in the courts. The judicial system has previously focused on preventing AI-generated precedents or other output issues, but it was unprepared for input-side attacks. A lawyer in Brazil was fined about $16,000 for using similar methods in an AI-reviewed case, and the ruling mentioned that such attacks currently have limited success but that it is necessary in the future to establish specific rules.

Spade also criticized how some self-represented litigants use chatbots: they first determine the conclusion and then ask the AI to find only supporting arguments, without allowing it to test facts or analyze opposing views, which instead solidifies incorrect judgments. He emphasized that AI users should also require the system to question their own positions, saying, "If one only asks the system to agree with the author's argument, in the end, it is dishonest even to the author themselves."