Crawlforge Mcp Server
by mysleekdesignscommunityv4.6.3
1
CrawlForge MCP Server is an MCP-compatible server for web scraping, crawling, deep research, browser automation, and structured content extraction. It provides local no-key tools by default, plus premium metered tools that use CrawlForge infrastructure for search, deep crawling, and autonomous agents.
From the registry: 26-tool MCP server for web scraping, crawling, deep research & autonomous extraction
Install & configure
Prerequisites
- requiredInstall Node.js 18 or newerNode.js is required to run the MCP server via npm/npx.https://nodejs.org/
- optionalOptional: Get a CrawlForge API keyRequired only to use metered premium tools such as search_web, crawl_deep, stealth_mode, agent, and deep_research. Free local tools can run without an API key.
https://www.crawlforge.dev/signup$ npx crawlforge-setup - optionalOptional: Install Ollama for local LLM extractionNeeded only if you want to use extract_with_llm with the default local Ollama backend. Install Ollama and pull at least one model.
https://ollama.com$ ollama pull <your-model-name>
Install in your client
Paste into the AI you're chatting with — it will figure out the rest
Please install the `crawlforge-mcp-server` MCP server into my current AI client (that's you).
Required prerequisites (do these first if not already done):
- **Install Node.js 18 or newer** — Node.js is required to run the MCP server via npm/npx. (https://nodejs.org/)
Optional prerequisites:
- Optional: Get a CrawlForge API key — Required only to use metered premium tools such as search_web, crawl_deep, stealth_mode, agent, and deep_research. Free local tools can run without an API key. Run: `npx crawlforge-setup` (https://www.crawlforge.dev/signup)
- Optional: Install Ollama for local LLM extraction — Needed only if you want to use extract_with_llm with the default local Ollama backend. Install Ollama and pull at least one model. Run: `ollama pull <your-model-name>` (https://ollama.com)
Canonical MCP server config (stdio transport):
- command: `npx`
- args: ["-y","crawlforge-mcp-server"]
- optional environment variables:
- `CRAWLFORGE_API_KEY`: CrawlForge API key for metered premium tools. Not needed for the free local tools. (example: `<your-crawlforge-api-key>`)
- `CRAWLFORGE_API_URL`: Custom CrawlForge API endpoint for enterprise setups. Must be a supported CrawlForge backend host. (example: `<your-crawlforge-api-url>`)
- `OLLAMA_BASE_URL`: Base URL for your local Ollama server used by extract_with_llm when using the default local backend. (example: `<your-ollama-base-url>`)
- `OLLAMA_DEFAULT_MODEL`: Default local Ollama model name for extract_with_llm. (example: `<your-ollama-model-name>`)
- `OPENAI_API_KEY`: Required only when using extract_with_llm or related extraction with provider set to openai. (example: `<your-openai-api-key>`)
- `ANTHROPIC_API_KEY`: Required only when using extract_with_llm or related extraction with provider set to anthropic. (example: `<your-anthropic-api-key>`)
- `ALLOW_JAVASCRIPT_EXECUTION`: Enable executeJavaScript actions for scrape_with_actions. Disabled by default and not recommended in production. (example: `<true-or-false>`)
Note: Prefer the npx launch command for the most portable setup. The README also mentions globally installed binaries such as crawlforge-mcp and crawlforge, but npx -y crawlforge-mcp-server is the recommended canonical MCP config.
Add this MCP server to my current client's config in the correct format for you. If you need secrets or credentials I haven't provided, ASK me — do not invent values or leave raw placeholders. After adding it, tell me how to verify the server is connected.Note: Prefer the npx launch command for the most portable setup. The README also mentions globally installed binaries such as crawlforge-mcp and crawlforge, but npx -y crawlforge-mcp-server is the recommended canonical MCP config.
Tools (26)
fetch_url
Fetch content from any URL
extract_text
Extract clean text from web pages
extract_links
Get all links from a page
extract_metadata
Extract page metadata (title, OG tags, schema.org)
scrape
Unified single-fetch extraction that can return markdown, HTML, raw HTML, text, links, metadata, screenshots, or JSON-schema output
scrape_structured
Extract structured data with CSS selectors
scrape_template
Extract structured data from well-known sites without writing selectors
extract_content
Enhanced content extraction
summarize_content
Generate summaries of content
analyze_content
Analyze content comprehensively
extract_structured
LLM-powered schema-driven extraction
extract_with_llm
Natural-language extraction using local Ollama by default or cloud LLMs if configured
process_document
Process multi-format documents
Environment variables
CRAWLFORGE_API_KEYrequiredAPI key for enabling metered premium tools and CrawlForge API accessCRAWLFORGE_API_URLrequiredCustom API endpoint for enterprise deploymentsOPENAI_API_KEYrequiredAPI key for OpenAI cloud extraction when provider is set to openaiANTHROPIC_API_KEYrequiredAPI key for Anthropic cloud extraction when provider is set to anthropicOLLAMA_BASE_URLrequiredBase URL for the local Ollama server used by extract_with_llmOLLAMA_DEFAULT_MODELrequiredDefault local Ollama model name used by extract_with_llmPackages
crawlforge-mcp-server
npmv4.6.3transport: stdio
Details
- Language
- JavaScript
- License
- MIT
- Released
- Jun 7, 2026
- Last commit
- Jun 9, 2026
- Registry name
- io.github.mysleekdesigns/crawlforge-mcp-server