Shadowcrawl
by DevsHerocommunityv2.3.0
62
6
CortexScout is a self-hostable search and web extraction engine designed for AI agents, providing capabilities for token-efficient web retrieval and stateful browser automation.
From the registry: Rust MCP stealth scraper: anti-bot search/scrape with CDP fallback + HITL non-robot.
Install & configure
Prerequisites
- requiredCortexScout binaryDownload prebuilt binary from GitHub Releases or build from source (Rust + protoc)https://github.com/DevsHero/ShadowCrawl/releases
Install in your client
Paste into the AI you're chatting with — it will figure out the rest
Please install the `shadowcrawl` MCP server into my current AI client (that's you).
Required prerequisites (do these first if not already done):
- **CortexScout binary** — Download prebuilt binary from GitHub Releases or build from source (Rust + protoc) (https://github.com/DevsHero/ShadowCrawl/releases)
Canonical MCP server config (stdio transport):
- command: `<path-to>/cortex-scout-mcp`
- args: []
- required environment variables:
- `RUST_LOG`: Must be 'warn' for MCP stdio (example: `warn`)
- optional environment variables:
- `SEARCH_ENGINES`: Active search engines (comma-separated) (example: `google,bing,duckduckgo,brave`)
- `OPENAI_API_KEY`: API key for deep_research LLM synthesis (example: `<your-api-key>`)
Note: Same as shadow-crawl (duplicate slug, same repo: DevsHero/ShadowCrawl). Rust binary. Anti-bot web retrieval with CDP, HITL, browser automation, deep research with LLM synthesis.
Add this MCP server to my current client's config in the correct format for you. If you need secrets or credentials I haven't provided, ASK me — do not invent values or leave raw placeholders. After adding it, tell me how to verify the server is connected.Note: Same as shadow-crawl (duplicate slug, same repo: DevsHero/ShadowCrawl). Rust binary. Anti-bot web retrieval with CDP, HITL, browser automation, deep research with LLM synthesis.
Tools (7)
web_search
URL discovery or search with content.
web_fetch
Unified fetch family for single, batch, or crawl modes.
extract_fields
Primary structured extraction tool.
scout_browser_automate
Stateful omni-tool for browser automation.
visual_scout
Human-in-the-Loop tool for visual tasks.
memory_search
Research history backed by LanceDB.
deep_research
Multi-hop search, scrape, and synthesis.
Environment variables
OPENAI_API_KEYrequiredAPI key for OpenAI services.LANCEDB_URIrequiredPath to LanceDB for research history.Prerequisites
- • python3
Details
- Language
- Rust
- License
- MIT
- Released
- Feb 18, 2026
- Last commit
- Apr 10, 2026
- Registry name
- io.github.DevsHero/ShadowCrawl