%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
Anthropic recently announced a new feature for some of its latest and largest AI models, allowing them to proactively end conversations when facing "rare, extreme, persistent harmful or abusive user interactions." Notably, the company explicitly stated that this measure is not intended to protect human users, but rather to protect the AI model itself.
Anthropic officially stated that its Claude AI model does not have the ability to perceive, nor does it claim that its interactions with users cause it harm. However, the company acknowledges "a high degree of uncertainty about the potential moral status of Claude and other large language models now or in the future." To address this, Anthropic recently launched a project called "Moral Welfare," aiming to take "precautionary" measures by implementing low-cost interventions to mitigate potential "moral welfare" risks.
This new feature is currently available only in Claude Opus 4 and 4.1 versions and will only be triggered in "extreme situations." For example, the AI model will activate this feature when users continuously request "sexual content involving minors, as well as attempts to obtain information that may trigger mass violence or terrorist acts."
Although these requests may bring legal or public relations issues to the company, Anthropic stated that during testing before deployment, Claude Opus 4 showed "strong opposition" and "clear signs of distress" when facing such harmful requests.
According to Anthropic, this feature is considered a "last resort" and will only be used after multiple redirection attempts have failed, the hope for effective interaction has been exhausted, or the user explicitly asks Claude to end the conversation. Additionally, the company has instructed Claude not to use this feature when there is an imminent risk of the user harming themselves or others.
Even if the conversation is terminated, users can start a new conversation from the same account or create a new chat branch by editing their response. Anthropic added that this feature is currently seen as an ongoing experiment, and the company will continue to improve its approach.
