Open Arsenal

Follow the Trail. Discover the Strategy.

WebDetective is a multi-agent website research and intelligence specification designed to investigate websites from the ground up. It coordinates specialized agents that discover pages, analyze website architecture, examine search engine visibility, map internal and external links, extract keywords and entities, and preserve discovered information for future research. Rather than relying on a single crawler or search provider, WebDetective uses independent search engine modules, allowing each engine to maintain specialized knowledge of its own commands, operators, query syntax, filters, and capabilities.

WebDetective provides comprehensive website discovery and archival capabilities. It can discover pages through search engines, XML sitemaps, robots.txt, internal links, external references, feeds, structured data, redirects, and other publicly accessible sources. Discovered pages can be archived with their HTML, extracted content, metadata, links, headers, timestamps, hashes, and other research evidence. The system maintains relationships between pages and domains through a research graph, making it possible to reconstruct website architecture, identify orphan pages, trace redirects, analyze page depth, and compare website changes over time.

A major feature of WebDetective is its deep SEO intelligence. Specialized agents analyze keywords, keyword density, semantic relationships, metadata, headings, structured data, content clusters, internal linking, backlinks, search-result visibility, technical SEO, local SEO, entity optimization, and other observable search optimization techniques. Its SEO Strategy Analysis Agent goes further by identifying patterns across these individual techniques and reconstructing the observable strategy used throughout a website. The system can examine how keywords are assigned to pages, how internal authority is distributed, how backlinks are structured, how content is organized, and how technical and content decisions work together to influence search visibility.

WebDetective is designed as an extensible research framework rather than a single-purpose SEO tool. Additional agents and plugins can provide competitive research, historical analysis, AI search analysis, technology detection, geographic research, accessibility analysis, performance analysis, and other specialized capabilities. Every significant observation can retain its source, timestamp, agent, search query, evidence, archive reference, and confidence level, creating a reproducible chain from discovery to analysis. By combining search intelligence, website crawling, archival evidence, SEO analysis, backlink research, semantic analysis, and multi-agent investigation, WebDetective provides a comprehensive way to follow the trail through a website and discover the strategy behind it.