A Rust-based LLM-powered web crawler that turns plain-English goals into structured web extraction and browser actions. Its MCP server exposes browser tools and an autonomous goal-running agent to MCP-compatible clients.
From the registry: Autonomous web crawler. 17 browser tools + goal-driven run_goal agent. Single binary, stealth.
$ curl -fsSL https://raw.githubusercontent.com/Mingye-Lu/AgenticCrawler/main/install.sh | bashhttps://github.com/Mingye-Lu/AgenticCrawler$ acrawl auth anthropicPlease install the `acrawl` MCP server into my current AI client (that's you).
Required prerequisites (do these first if not already done):
- **Install acrawl binary** — Install the acrawl CLI using the platform install script so the `acrawl` command is available. Run: `curl -fsSL https://raw.githubusercontent.com/Mingye-Lu/AgenticCrawler/main/install.sh | bash` (https://github.com/Mingye-Lu/AgenticCrawler)
- **Install Node.js 20+** — Required for browser features; the installer sets up CloakBrowser for browser automation. (https://nodejs.org/)
Optional prerequisites:
- Configure an LLM provider for run_goal — The MCP server's `run_goal` tool requires acrawl credentials in `~/.acrawl/credentials.json`. Set up a provider interactively, such as Anthropic or OpenAI. Run: `acrawl auth anthropic`
Canonical MCP server config (stdio transport):
- command: `acrawl`
- args: ["mcp"]
- optional environment variables:
- `ACRAWL_CONFIG_HOME`: Override the directory where acrawl stores configuration and credentials instead of the default `~/.acrawl`. Useful if the MCP client runs in an environment with a different home directory. (example: `<your-acrawl-config-dir>`)
Note: acrawl's built-in MCP server is stdio-only in this release. It exposes direct browser tools without extra configuration. The `run_goal` autonomous tool additionally requires acrawl to have provider credentials configured in `credentials.json`.
Add this MCP server to my current client's config in the correct format for you. If you need secrets or credentials I haven't provided, ASK me — do not invent values or leave raw placeholders. After adding it, tell me how to verify the server is connected.ACRAWL_CONFIG_HOMErequiredOverrides the default directory used to store acrawl configuration and saved credentials.MCP server for Firecrawl web scraping, structured data extraction and web search integration.