Repository avatar
AI Tools
v2.0.0
active

markudown-mcp

io.github.Scrape-Technology/markudown-mcp

Web scraping for AI agents: scrape, crawl, search and extract any site as clean Markdown or JSON.

Documentation

markudown-mcp

npm version License: MIT

MCP server for MarkUDown / Scrape Technology — gives AI agents (Claude, Cursor, Windsurf, n8n, LangGraph, etc.) access to web scraping, Google search, data extraction, and more.

No infrastructure needed. Just an API key — get one free at scrapetechnology.com/markudown/register (500 free credits, no credit card). Docs: scrapetechnology.com/markudown/docs.

MARKUDOWN_API_KEY=your-key npx -y markudown-mcp

How it works

AI Agent (Claude Desktop, Cursor, etc.)
        │  MCP protocol (stdio)
        ▼
  markudown-mcp  (this package)
        │  HTTPS + polling
        ▼
  api.scrapetechnology.com
        │  BullMQ jobs
        ▼
  MarkUDown Engine workers

Quick start

MARKUDOWN_API_KEY=your-key npx markudown-mcp

Get your API key at scrapetechnology.com/markudown/register.


Claude Desktop

Add to claude_desktop_config.json:

  • macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
  • Windows: %APPDATA%\Claude\claude_desktop_config.json
{
  "mcpServers": {
    "markudown": {
      "command": "npx",
      "args": ["markudown-mcp"],
      "env": {
        "MARKUDOWN_API_KEY": "your-api-key-here"
      }
    }
  }
}

Restart Claude Desktop after saving.


Cursor / Windsurf

Add to your editor's MCP config:

{
  "markudown": {
    "command": "npx",
    "args": ["markudown-mcp"],
    "env": {
      "MARKUDOWN_API_KEY": "your-api-key-here"
    }
  }
}

Environment variables

VariableRequiredDefaultDescription
MARKUDOWN_API_KEYYes—Your Scrape Technology API key
MARKUDOWN_API_URLNohttps://api.scrapetechnology.comOverride the API base URL

Note: a key the API does not recognise is not rejected — it is served as the anonymous free tier (20 requests/day), and a 429 "Free tier daily limit exceeded" follows once that is used up. If you see that message with a paid key, check the key in your dashboard. Only a missing key returns 401.


Available tools

scrape

Scrape a single URL and return its content as clean Markdown.

ParameterTypeDefaultDescription
urlstringrequiredURL to scrape
mainContentbooleantrueStrip navigation, ads, and sidebars
includeLinksbooleanfalseInclude hyperlinks in output
includeHtmlbooleanfalseAlso return raw HTML
excludeTagsstring[]["header","nav","footer"]HTML tags to strip
formats("markdown" | "summary")[]—Output formats to return
timeoutnumber60Timeout in seconds

Example prompt: "Scrape https://example.com and summarize the main content."


map

Discover all URLs on a website via sitemap parsing and link crawling.

ParameterTypeDefaultDescription
urlstringrequiredRoot URL of the site
maxUrlsnumber1000Maximum URLs to return
allowedWordsstring[][]Only include URLs containing these words
blockedWordsstring[][]Exclude URLs containing these words

Example prompt: "Map all URLs on docs.example.com."


crawl

Recursively crawl a website, following internal links. Returns Markdown for each page.

ParameterTypeDefaultDescription
urlstringrequiredStarting URL
maxDepthnumber2Maximum link depth
limitnumber10Maximum pages to crawl
mainContentbooleantrueExtract only main content
includeLinksbooleanfalseInclude hyperlinks
includeHtmlbooleanfalseInclude raw HTML
excludeTagsstring[]—HTML tags to strip
blockedWordsstring[]—Skip URLs with these words
includeOnlystring[]—Only crawl URLs with these words
timeoutnumber60Per-page timeout in seconds

Example prompt: "Crawl https://docs.example.com up to depth 3 and find all mentions of 'authentication'."


search

Run a Google search and optionally scrape the full content of each result page.

ParameterTypeDefaultDescription
querystringrequiredSearch query
limitnumber5Number of results
scrapeResultsbooleantrueScrape full content for each result
langstring"en"Language code
countrystring"us"Country code
timeoutnumber60Per-page timeout in seconds

Example prompt: "Search for 'best web scraping libraries 2025' and summarize the top 5 results."


extract

Scrape a URL and extract structured data using a natural language prompt.

ParameterTypeDefaultDescription
urlstringrequiredURL to extract from
promptstringrequiredWhat to extract, e.g. "All product names and prices"
timeoutnumber60Timeout in seconds

Example prompt: "Extract all job listings from https://jobs.example.com including title, location, and salary."


batch_scrape

Scrape multiple URLs in parallel. Returns Markdown for each.

ParameterTypeDefaultDescription
urlsstring[]requiredURLs to scrape (at least 1)
timeoutnumber60Per-page timeout in seconds

Example prompt: "Scrape these 5 competitor pricing pages and compare their plans."


screenshot

Take a full-page screenshot of a URL.

ParameterTypeDefaultDescription
urlstringrequiredURL to screenshot
timeoutnumber60Navigation timeout in seconds

Example prompt: "Take a screenshot of https://example.com."


deep_research

Scrape multiple URLs and synthesize their content into a comprehensive research report using an LLM.

ParameterTypeDefaultDescription
querystringrequiredResearch question or topic
urlsstring[]requiredSource URLs to scrape (at least 1)
maxTokensnumber4096Maximum tokens for the report
timeoutnumber180Timeout in seconds

Example prompt: "Research the pricing strategies of these 3 SaaS companies and write a comparative analysis."


agent

Launch an AI agent that autonomously navigates a website to answer a question or complete a task.

ParameterTypeDefaultDescription
urlstringrequiredStarting URL
promptstringrequiredTask or question for the agent
maxStepsnumber10Maximum navigation steps
maxPagesnumber5Maximum pages to visit
allowNavigationbooleantrueAllow following links
includeScreenshotsbooleanfalseScreenshot each step
timeoutnumber120Total timeout in seconds

Example prompt: "Go to https://shop.example.com and find the return policy."


change_detection

Detect whether a URL's content has changed since it was last checked.

ParameterTypeDefaultDescription
urlstringrequiredURL to monitor
mainContentbooleantrueCompare only main content
excludeTagsstring[]["header","nav","footer"]Tags to ignore when comparing (currently ignored by the API)
timeoutnumber60Timeout in seconds

Example prompt: "Check if the pricing page at https://example.com/pricing has changed."


Support & Community

ChannelLink
💬 Discorddiscord.gg/GBSKsC8DvS
📧 Emailjoao.sobhie@scrapetechnology.com
🌐 Docsscrapetechnology.com/markudown/docs

License

MIT

NPM
markudown-mcp
Install Command
npm install markudown-mcp
Runtime: npx