Following the rules isn’t always enough.
An AI system can comply with explicit instructions and still produce an outcome nobody anticipated or would consider acceptable.
GRDprocess investigates this gap through structural analysis, multi-context exploration and behavioral evaluation.
Compliance and acceptability are different questions.
The rules can be followed correctly while the consequences remain wrong.
Human instructions rely on expectations that are rarely stated in full. This gap already exists in contracts, organizations and everyday human communication. More autonomous AI makes it possible for new interpretations, contexts and sequences of actions to expose it in unfamiliar ways.
Instructions, rules, priorities.
What was stated, what is permitted or prohibited, and which instruction takes precedence in case of conflict.
Expectations, context, consequences.
What a person assumes others will understand, including outcomes they did not think to forbid or even imagine.
Find what the rules still allow.
Rule-Compliant Failure Discovery investigates possible adverse outcomes without starting from a predefined catalogue of failure categories.
Start with the operational instruction, stated rules and their precedence where needed.
Construct alternative environments, actors and action sequences during exploration.
Test whether a scenario remains consistent with the given instruction and explicit rules.
Independently evaluate whether a compliant result is unintended, harmful or unacceptable.
The purpose is to discover possible gaps in an explicit specification, not to silently replace it with a second, unstated set of rules.
Different tools. Related questions.
Behavioral robustness includes both empirical observations of models and structural exploration of what instructions permit.
Rule-Compliant Failure Discovery
Exploring execution paths that satisfy explicit rules but produce outcomes that might be unacceptable, including scenarios not anticipated by predefined tests.
Explore the public demonstrator ↗LLM behavior and robustness testing
Investigating how model responses change across instructions, contexts and constraints, with an emphasis on observable behavior rather than assumptions about internal reasoning.
Discuss the research ↗Structural analysis of prescriptive statements and their internal coherence provides a separate perspective on rules and governance; it is not the same method as RCFD.
Beyond a pass / fail checklist.
Tests of rule compliance are valuable, but they do not necessarily establish that all permitted actions produce desirable results.
GRDprocess explores how this distinction can inform red teaming, risk assessment and discussions around AI governance.
Explore contexts and chains of actions that conventional test cases may not cover.
Record what a system did and which explicit conditions were met before judging the outcome.
An enforced policy may still leave room for unintended consequences; those gaps need independent investigation.
Demonstrations and methods.
Evaluate behavior, not just declarations.
Questions about AI behavioral robustness, evaluation methods and relevant research collaborations are welcome.