Open Arsenal

Expose inconsistency at scale.

The origin of BreakSignal comes from a recurring pattern observed during the use of multiple enterprise-grade AI systems in real-world workflows: outputs that appeared internally confident, yet directly conflicted with externally verifiable reality. In several audit and testing scenarios, the same prompt would yield sharply different behaviors depending on the model—especially when interacting with web-linked content or structured data sources.

A particularly common pattern involved an AI displaying a raw URL and acknowledging its relevance, only to later assert that it could not access the same content. In some cases, the system would claim that a page was blocked due to robots.txt restrictions, even when parallel tests with other models or retrieval tools confirmed the content was fully accessible. These inconsistencies did not present as overt deception, but rather as a breakdown between perceived capability, policy constraints, and actual system behavior.

Similar experiences have been widely discussed in user communities and developer forums, where practitioners report contradictions such as an AI stating it cannot read a file that it has just referenced, or providing explanations for access limitations that do not align with observable network or retrieval results. Other users have described situations where identical documents were processed successfully by one model while another returned vague refusals or incomplete interpretations without clear technical justification. While these accounts vary in detail and context, the shared theme is inconsistency under comparable conditions.

BreakSignal was designed in response to these patterns—not to assign intent or fault, but to systematically measure and document when and how such inconsistencies occur. The goal is to replace anecdotal frustration with reproducible audit trails that can distinguish between true access limitations, model uncertainty, and behavior shaped by hidden constraints.