CortexScout is a self-hostable search and web extraction engine designed for AI agents, providing capabilities for token-efficient web retrieval and stateful browser automation.
From the registry: Stealth scraping & search. Bypasses Cloudflare, DataDome & LinkedIn via Cyborg HITL approach.
Please install the `shadow-crawl` MCP server into my current AI client (that's you).
Required prerequisites (do these first if not already done):
- **CortexScout binary** — Download prebuilt binary from GitHub Releases or build from source (Rust + protoc) (https://github.com/DevsHero/ShadowCrawl/releases)
Canonical MCP server config (stdio transport):
- command: `<path-to>/cortex-scout-mcp`
- args: []
- required environment variables:
- `RUST_LOG`: Must be 'warn' for MCP stdio — 'info' floods stderr (example: `warn`)
- optional environment variables:
- `SEARCH_ENGINES`: Comma-separated search engines (example: `google,bing,duckduckgo,brave`)
- `OPENAI_API_KEY`: API key for deep_research LLM synthesis (OpenAI-compatible) (example: `<your-api-key>`)
Note: Rust binary. Anti-bot web retrieval with CDP fallback and HITL support. Tools: web_search, web_fetch (single/batch/crawl), extract_fields, browser_automate, deep_research, memory_search. Must download binary first.
Add this MCP server to my current client's config in the correct format for you. If you need secrets or credentials I haven't provided, ASK me — do not invent values or leave raw placeholders. After adding it, tell me how to verify the server is connected.OPENAI_API_KEYrequiredAPI key for OpenAI servicesLANCEDB_URIrequiredURI for LanceDB storageAI-powered browser automation — navigate, click, fill forms, and extract data from any website.