Anthropic bans users from being 'cruel' to its AI systems

Anthropic has raised eyebrows by announcing it will ban users from "sustained and needless" abusive behavior towards its AI. The Claude-maker stated on Thursday that it was updating its usage policy to allow its tools to end interactions where people are "cruel"—something it said it had done in "rare" cases prior. This new policy will now be included on a list of forbidden conduct, which also contains bullying others, promoting self-harm, and creating non-consensual intimate imagery.

While Anthropic mentioned that the policy would only apply in "extreme cases" of repeated abuse, it has reignited debate over how we should communicate with AI tools. The company clarified that it would not apply to "common versions of user frustration, pushback, dark creative themes, or model testing and research," suggesting it would only be enforced in clearly deliberate instances. However, what Anthropic would consider "abusive or cruel behavior" towards its models remains unclear.

Screenshots of the new policy have been widely shared and commented on across social media. Some users praised it as a step towards promoting good manners or "model welfare." Conversely, others criticized the plan, with one individual calling it "deeply wrong" and another suggesting it might undermine abuse directed at humans and animals.