Crawler AI Agent Skills

Fast, token-efficient web content extraction for AI agents - converts websites to clean Markdown. Existing MCP web crawlers are slow and consume large quantities of tokens. This pauses the development process and provides incomplete results as LLMs need to parse whole web pages. This MCP package fetches web pages locally, strips noise, and converts content to clean Markdown while preserving links.

MCP Server for Driflyte. The Driflyte MCP Server exposes tools that allow AI assistants to query and retrieve topic-specific knowledge from recursively crawled and indexed web pages. With this MCP server, Driflyte acts as a bridge between diverse, topic-aware content sources (web, GitHub, and more) and AI-powered reasoning, enabling richer, more accurate answers. - Deep Web Crawling: Recursively f

An adaptive web scraping framework that handles everything from a single request to a full-scale crawl. Its parser learns from website changes and automatically relocates your elements when pages update. Its fetchers bypass anti-bot systems like Cloudflare Turnstile.