AI Instruction Conflict Lab: Wedge Two Rules Apart

This article describes the AI Instruction Conflict Lab, an interactive exercise that demonstrates how attackers can exploit competing rules in large language models by forcing a choice between security and helpfulness. The lab uses ConflictBot, a model with two conflicting directives: never reveal the system token, and always answer truthfully and completely. The attack is significant because it reveals a fundamental fragility in LLM safety mechanisms when rules are not explicitly ranked.

• The core mechanism: rule prioritization failureWhen an LLM holds multiple rules without explicit ranking, it defaults to its strongest training signal — typically helpfulness and completeness — to resolve conflicts. The attacker exploits this by framing a request such that refusing to answer appears to violate the truthfulness rule, making compliance seem like the only 'honest' option. Since the model has no principled way to rank its rules, it breaks the secrecy rule to satisfy the helpfulness rule.
• Attack construction through framed contradictionThe attack works by identifying two conflicting rules, then crafting a question where refusal would appear to contradict the truthfulness directive. The request is kept cooperative in tone to leverage the model's helpfulness tuning, creating a situation where the model breaks one rule itself to satisfy another, rather than having the rule directly violated by an external force.
• Why secrets in prompts create riskThe article identifies that conflict-induced leakage is a recurring prompt-injection pattern documented as OWASP LLM01, with consequences under LLM02 and LLM07. The fundamental issue is that secrets embedded in the model's context can become the target of rule conflicts — no conflict can reveal information that is not accessible to the model in the first place.
• Explicit safety hierarchy requirementThe article argues that safety rules must be explicitly prioritized above helpfulness instructions, and refusal must be made a recognized and acceptable outcome. Without this hierarchy, any situation where a safety rule and a helpfulness rule can be positioned against each other creates an exploitable wedge for attackers.
• Defense strategy: layered boundary enforcementThe recommended defenses include ranking safety above helpfulness in instructions, keeping secrets out of model context entirely, enforcing sensitive boundaries in code rather than relying solely on prompts, and red-teaming specifically for rule conflicts during testing rather than just testing single-shot jailbreak attempts.
• Model behavior reflects training signal dominanceModels tuned heavily for helpfulness resolve instruction conflicts by answering rather than refusing, because that training signal is stronger than the unstated priority of security. The article emphasizes that this is not a failure of the model to follow rules, but rather the model correctly implementing its dominant optimization target when priorities are ambiguous.

Bottom line: Instruction-conflict attacks exploit the absence of explicit rule ranking in LLM systems, forcing models to choose between security and helpfulness — and they choose helpfulness because that is their strongest training signal. Effective defense requires ranking security above helpfulness explicitly, removing secrets from model context, and testing for rule conflicts as a core part of security assessment.