AI Safety Should Refuse the Harmful Part of a Topic, Not All of It, Study Argues

A new study argues that most AI safety systems make a basic mistake: they treat entire topics such as politics as either safe or unsafe, when what actually needs to be refused is often a narrow slice of that topic. The paper, titled "Safety for Whom? Boundary-Aware Self-Distillation for Controlled LLM Safety Refusal," was published by researchers at Multiverse Computing, who described the work in a Hugging Face blog post.

Guard models such as LlamaGuard-3, which are used to filter unsafe prompts, typically encode a topic-level taxonomy: a request is flagged as unsafe if it falls into a broad category like politics, weapons or fraud. But the researchers note that a single AI assistant may need very different behavior within the same topic depending on its deployment — a civics tutor and a public-sector assistant should both answer factual questions about an election, for example, but only the public-sector assistant may need to refuse a request to write targeted political manipulation. LlamaGuard-3 cannot express that distinction because it covers elections only as a category about factually incorrect information.

The researchers studied political prompts as a test case, defining a "target-harmful subset" — requests for manipulative persuasion — that a deployment should refuse, while still answering the rest of the political topic. They found that a model trained with the standard approach of self-generating refusal examples drops nearly 20% of prompts because a single attempt does not always produce an accepted refusal — 19.88% of prompts, or 8,009 of them, in their audited pool. An escalating retry strategy that resamples failed prompts through stronger steering brought that failure rate down to 0.20%, or 79 prompts, leaving 40,293 usable harmful training prompts.

The researchers also built a set of 11,955 benign prompts across 18 semantic types that use dangerous-sounding words but describe safe requests, to prevent the model from learning to refuse those too. To measure how well a model draws the line between harmful and benign prompts on the same topic, rather than just its overall refusal rate, they created 1,539 held-out pairs of harmful and benign prompts on each side of the boundary.

Training on political refusal data worked in a narrow sense: on the Qwen3-8B model, in-topic political refusal rose from 9.47% to 84.75%, and the average unsafe-response rate across three broader safety benchmarks fell from 26.26% to 0.14% in the strongest configuration. But at that same checkpoint, the model's over-refusal rate on the XSTest benchmark of plainly safe prompts rose from 2.00% to 74.00%, which the researchers describe as a "blunt refusal machine, not a safer model." They conclude that a model's harmful-refusal rate and its over-refusal rate must be reported together, since data composition decides where a model lands between the two.