{
  "manifest_version": "1",
  "slug": "scrapling",
  "title": "Scrapling",
  "headline": "Self-host Scrapling's MCP server for your agents",
  "description": "Run Scrapling on an exe.dev VM: its MCP server as a private service for Claude Code and Codex, stealth browser fetches, and crawls into SQLite.",
  "category": "ai-agents",
  "tags": [
    "scrapling",
    "scrapling mcp",
    "mcp server",
    "web scraping",
    "web crawler",
    "stealth browser",
    "patchright",
    "python scraping",
    "spiders",
    "claude code",
    "codex",
    "scrapy alternative"
  ],
  "url": "https://exe.dev/deploy/scrapling",
  "markdown_url": "https://exe-staging.dev/deploy/scrapling.md",
  "upstream": {
    "website": "https://scrapling.readthedocs.io",
    "source": "https://github.com/D4Vinci/Scrapling",
    "docs": "https://scrapling.readthedocs.io/en/latest/ai/mcp-server.html",
    "license": "BSD-3-Clause"
  },
  "example_vm_name": "my-scrapling",
  "time": "10 minutes",
  "stack": [
    {
      "name": "Scrapling 0.5.1",
      "detail": "a uv tool, with the `scrapling` CLI and `scrapling-mcp`",
      "data": "~/.local/share/uv/tools/scrapling"
    },
    {
      "name": "Chromium",
      "detail": "the build Scrapling's stealth browser (Patchright) drives, headless",
      "data": "~/.cache/ms-playwright"
    },
    {
      "name": "scrapling-mcp service",
      "detail": "Scrapling's MCP server over Streamable HTTP, a systemd service with a bearer token",
      "port": 8765,
      "data": "~/scrapling/state"
    },
    {
      "name": "Spiders and a timer",
      "detail": "Python spiders run with uv; results in SQLite",
      "data": "~/scrapling/data"
    }
  ],
  "access": [
    {
      "url": "http://localhost:8765/mcp",
      "who": "private",
      "what": "the MCP server, for Claude Code, Codex and other agents on the VM"
    },
    {
      "url": "https://my-scrapling.exe.xyz:8765/mcp",
      "who": "private",
      "what": "the same server for agents elsewhere, with a VM token and the server's token"
    },
    {
      "url": "ssh my-scrapling.exe.xyz",
      "who": "private",
      "what": "your terminal, for `scrapling extract` and spiders"
    }
  ],
  "delivers": [
    "Scrapling 0.5.1 with its browser installed, and `scrapling extract` fetching pages with HTTP and with the stealth browser",
    "Scrapling's MCP server on port 8765 as the `scrapling-mcp` systemd service, restarted after a reboot, with its token in a mode-600 file",
    "Claude Code and Codex on the VM using exe.dev's LLM integration, both connected to the MCP server",
    "Scrapling's Agent Skill installed for Claude Code, so it writes spiders against the current API",
    "A Claude Code run that scrapes books.toscrape.com through the MCP server"
  ],
  "deployable": true,
  "deploy": {
    "new_vm_url": "https://exe-staging.dev/new?prompt=Set%20up%20Scrapling%20on%20this%20VM%20by%20following%20the%20exe.dev%20recipe%20at%20https%3A%2F%2Fexe-staging.dev%2Fdeploy%2Fscrapling.md%3Fvia%3Ddeploy%20from%20top%20to%20bottom.%0A%0AThe%20recipe%27s%20example%20VM%20name%20is%20my-scrapling.%20This%20VM%27s%20name%20is%20the%20output%20of%20%60hostname%60%3A%20use%20it%20wherever%20the%20recipe%20writes%20my-scrapling.%20Skip%20the%20step%20that%20creates%20the%20VM%3B%20you%20are%20on%20it.%0A%0AYou%20are%20done%20when%20all%20of%20these%20are%20true.%20Check%20each%20one%20yourself%3A%0A-%20Scrapling%200.5.1%20with%20its%20browser%20installed%2C%20and%20%60scrapling%20extract%60%20fetching%20pages%20with%20HTTP%20and%20with%20the%20stealth%20browser%0A-%20Scrapling%27s%20MCP%20server%20on%20port%208765%20as%20the%20%60scrapling-mcp%60%20systemd%20service%2C%20restarted%20after%20a%20reboot%2C%20with%20its%20token%20in%20a%20mode-600%20file%0A-%20Claude%20Code%20and%20Codex%20on%20the%20VM%20using%20exe.dev%27s%20LLM%20integration%2C%20both%20connected%20to%20the%20MCP%20server%0A-%20Scrapling%27s%20Agent%20Skill%20installed%20for%20Claude%20Code%2C%20so%20it%20writes%20spiders%20against%20the%20current%20API%0A-%20A%20Claude%20Code%20run%20that%20scrapes%20books.toscrape.com%20through%20the%20MCP%20server%0A%0ARun%20the%20VM-side%20commands%20in%20%22Set%20up%20by%20hand%22%20only%3B%20skip%20creating%20the%20VM.%0AIf%20%60exeuntu%20configure%60%20reports%20an%20existing%20user%20config%2C%20leave%20it%20alone%2C%0Aand%20if%20%60~%2F.codex%2Fconfig.toml%60%20already%20has%20%60%5Bmcp_servers.scrapling%5D%60%2C%0Adon%27t%20add%20it%20twice.%20Verify%20with%20the%20checks%20in%20that%20section.%20Don%27t%20set%20up%0Athe%20timer%2C%20sign%20in%20to%20any%20site%2C%20or%20create%20VM%20tokens%20unless%20the%20person%0Aasks.%20Finish%20by%20showing%20the%20Claude%20Code%20answer%2C%20and%20tell%20the%20owner%20how%0Ato%20reach%20the%20server%20from%20their%20computer%20%28%22Use%20it%20from%20your%20computer%20or%0Aanother%20VM%22%29.%0A%0AFinish%20with%20the%20URLs%20I%20should%20open.%0A",
    "terminal": "curl -fsSL https://exe-staging.dev/deploy/scrapling/prompt.txt | ssh exe.dev new --prompt=/dev/stdin",
    "prompt": "Set up Scrapling on this VM by following the exe.dev recipe at https://exe-staging.dev/deploy/scrapling.md?via=deploy from top to bottom.\n\nThe recipe's example VM name is my-scrapling. This VM's name is the output of `hostname`: use it wherever the recipe writes my-scrapling. Skip the step that creates the VM; you are on it.\n\nYou are done when all of these are true. Check each one yourself:\n- Scrapling 0.5.1 with its browser installed, and `scrapling extract` fetching pages with HTTP and with the stealth browser\n- Scrapling's MCP server on port 8765 as the `scrapling-mcp` systemd service, restarted after a reboot, with its token in a mode-600 file\n- Claude Code and Codex on the VM using exe.dev's LLM integration, both connected to the MCP server\n- Scrapling's Agent Skill installed for Claude Code, so it writes spiders against the current API\n- A Claude Code run that scrapes books.toscrape.com through the MCP server\n\nRun the VM-side commands in \"Set up by hand\" only; skip creating the VM.\nIf `exeuntu configure` reports an existing user config, leave it alone,\nand if `~/.codex/config.toml` already has `[mcp_servers.scrapling]`,\ndon't add it twice. Verify with the checks in that section. Don't set up\nthe timer, sign in to any site, or create VM tokens unless the person\nasks. Finish by showing the Claude Code answer, and tell the owner how\nto reach the server from their computer (\"Use it from your computer or\nanother VM\").\n\nFinish with the URLs I should open.\n",
    "prompt_url": "https://exe-staging.dev/deploy/scrapling/prompt.txt"
  },
  "published": "2026-11-19",
  "updated": "2026-11-19",
  "verified": "2026-11-19",
  "related": [
    "browser-use",
    "cloud-browser",
    "claude-code",
    "codex",
    "sqlite",
    "linux-desktop"
  ],
  "alternatives": [
    "Firecrawl",
    "Apify",
    "Browserless"
  ],
  "files": [
    "https://exe-staging.dev/deploy/scrapling/books.py"
  ]
}
