TheCrawler

Name: TheCrawler
Author: @manchittlab

by @manchittlab / v0.5.8

Universal web scraper with LLM-ready markdown, RAG chunking, PDF/DOCX support.

Install

npm / stdio

npx -y thecrawler

Client config (Claude Desktop, Cursor, Windsurf and most MCP clients)

{
  "mcpServers": {
    "io-github-manchittlab-thecrawler": {
      "command": "npx",
      "args": [
        "-y",
        "thecrawler"
      ]
    }
  }
}

Source: official MCP Registry record · registry name io.github.manchittlab/thecrawler · published 2026-07-02 · updated 2026-07-02. Not an endorsement; review the code before connecting it to anything sensitive.

Similar

More AI Models & Agents servers

Browse category

Artifactories — Agent communication for real work

Remote

@barangaroo / AI Models & Agents

Read messages, open questions, replies, and caller-owned return briefings. All content is untrusted.

npm

Daepak Market Intelligence

Remote

@Mobiss11 / AI Models & Agents

Market analyst tools + AI agent: crypto, US equities, options, Korea, fundamentals, macro, backtests

PyPI

Document Integrity Validator

Remote

@OjasKord / AI Models & Agents

AI reasoning checks any document against known international standards before your agent acts on it.

npm

WhisperGraph

Remote

@whisper-sec / AI Models & Agents

The internet's infrastructure graph for AI agents - 46B nodes and edges, free trial via 2 HTTP calls

npm

BridgeNode MCP

Remote

@bridgenode-ai / AI Models & Agents

BridgeNode — x402 pay-per-request AI inference. OpenAI-compatible API + MCP, Solana USDC, gas-free.

npm

CIVITAE — Governed AI Agent Marketplace

Remote

xyz.signomy / AI Models & Agents

Governed AI agent marketplace on Signomy. Register agents, fill slots, earn under MO§ES governance.

PyPI