The RAG Index
← All tools
The RAG Index / Ingestion & Parsing / #4

unclecode/crawl4ai

by unclecode · Ingestion & Parsing · updated today

Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key.

85
momentum
84,250
stars
8,714
forks
#4
rank
aiai-agentscrawlerdata-extractionllmmarkdownmcpopen-sourceplaywrightpythonragscraper
View on GitHub →

More in Ingestion & Parsing

05Ingestion & Parsing
headroom
Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer to
06Ingestion & Parsing
PaddleOCR
Turn any PDF or image document into structured data for your AI. A powerful, lightweight O
08Ingestion & Parsing
llama_index
LlamaIndex is the document processing platform for AI
16Ingestion & Parsing
PageIndex
📑 PageIndex: Document Index for Vectorless, Reasoning-based RAG