Turn URLs into clean markdown or structured JSON with one engine for search, scrape, map, crawl, and extract. Run it locally as a small Rust binary or use the managed API. Get 1000 free credits → · Install · No credit card. Continue with GitHub. Runs local and free, no account needed. To use the Cloud, paste your key into the same command and it installs the binary, connects the key, and registers
npx mdskills install us/crwRelated
@us? Sign in with GitHub to claim this listing.1<p align="center">2 <img src="docs/logo-animation.gif" alt="fastCRW" width="180" />3</p>45<h1 align="center">fastCRW</h1>67<p align="center">8 Turn URLs into clean <strong>markdown</strong> or structured9 <strong>JSON</strong> with one engine for search, scrape, map, crawl, and extract.10</p>1112<p align="center">13 Run it locally as a small Rust binary or use the managed API.14</p>1516<p align="center">17 <a href="https://fastcrw.com/register?ref=gh-hero"><strong>Get 1000 free credits →</strong></a> ·18 <a href="#one-command-install"><strong>Install</strong></a> ·19 <a href="https://docs.fastcrw.com"><strong>Docs</strong></a>20</p>2122<p align="center">23 <sub>No credit card. Continue with GitHub.</sub>24</p>2526<p align="center">27 <a href="https://crates.io/crates/crw-server"><img src="https://img.shields.io/crates/v/crw-server.svg" alt="crates.io"></a>28 <a href="https://pypi.org/project/crw/"><img src="https://img.shields.io/pypi/v/crw.svg?label=pypi" alt="PyPI"></a>29 <a href="https://www.npmjs.com/package/crw-mcp"><img src="https://img.shields.io/npm/v/crw-mcp.svg?label=npm%20mcp" alt="npm crw-mcp"></a>30 <a href="https://github.com/us/crw/actions/workflows/ci.yml"><img src="https://github.com/us/crw/actions/workflows/ci.yml/badge.svg?branch=main&event=push" alt="CI"></a>31 <a href="LICENSE"><img src="https://img.shields.io/badge/license-AGPL--3.0-blue.svg" alt="License"></a>32 <a href="https://github.com/us/crw/stargazers"><img src="https://img.shields.io/github/stars/us/crw?style=social" alt="GitHub Stars"></a>33</p>3435## One-command install3637```bash38curl -fsSL https://fastcrw.com/install | sh39```4041Runs local and free, no account needed. To use the Cloud, paste your key into42the same command and it installs the binary, connects the key, and registers the43MCP server with the AI coding tools you already have:4445```bash46curl -fsSL https://fastcrw.com/install | CRW_API_KEY=crw_live_... sh47```4849```bash50crw search "rust tutorials"51```5253Claude Code, Cursor, Codex, Gemini CLI, OpenCode and Windsurf are picked up54automatically when they are already set up; nothing else is touched, and your55key stays in `~/.config/crw/config.toml` rather than being copied into each56tool. Add `CRW_NO_AGENTS=1` to skip that step, or run `crw setup` on its own to57choose interactively.5859**1000 free credits, no credit card.** Managed proxies, JS rendering and search,60with nothing to run or keep up to date.61[Get my free key →](https://fastcrw.com/register?ref=gh-quickstart)6263macOS and Linux, Intel and ARM.64[More install options →](https://docs.fastcrw.com/installation/)6566## What it does6768| Operation | Outcome |69|---|---|70| **Scrape** | One URL to markdown, HTML, links, screenshots, or schema JSON |71| **Crawl** | Follow a bounded site crawl and collect its pages |72| **Map** | Discover URLs without scraping every page |73| **Search** | Search the web and optionally scrape selected results |74| **Extract** | Produce structured fields from one or many URLs |7576[See the full API →](https://docs.fastcrw.com/#rest-api)7778## Why fastCRW7980On Firecrawl's own public 1,000-URL dataset, fastCRW recovered more truth than81Crawl4AI and Firecrawl, matched the fastest median latency, and idled at82**~14 MB RAM**.8384<p align="center">85 <a href="BENCHMARKS.md">86 <img src=".github/benchmarks/bench-radar.svg" alt="fastCRW compared with Crawl4AI and Firecrawl on truth-recall, unique recoveries, median latency, download size, and recall depth" width="100%">87 </a>88</p>8990<p align="center"><sub><a href="BENCHMARKS.md">Methodology, full numbers, and how to reproduce it</a></sub></p>9192On a different benchmark entirely, answer accuracy rather than scrape recall,93fastCRW answers **90.0%** of the 600 AA-Omniscience questions correctly. Every94product listed on the Artificial Analysis Search Index sits below it.9596<p align="center">97 <a href="https://fastcrw.com/benchmarks/aa-omniscience">98 <img src=".github/benchmarks/bench-omniscience.svg" alt="AA-Omniscience answer accuracy. fastCRW 90.0 percent, ahead of every product on the Artificial Analysis Search Index: Firecrawl 73, Exa 70, You.com 69, Parallel 68, Tavily 64, and 38 with no search at all." width="100%">99 </a>100</p>101102<p align="center"><sub><a href="https://fastcrw.com/benchmarks/aa-omniscience">The full 14-product comparison, the control run, and how to reproduce it</a></sub></p>103104## Choose how you use it105106### CLI107108```bash109crw https://example.com # scrape, works right after install110crw search "rust async runtime" # search, after `crw setup`111```112113### Python SDK114115Using Cloud? [Get an API key](https://fastcrw.com/register?ref=gh-sdk), then export it once:116117```bash118export CRW_API_KEY="crw_live_..."119pip install crw120```121122```python123from crw import CrwClient124125client = CrwClient()126page = client.scrape("https://example.com", formats=["markdown"])127128print(page["markdown"])129```130131<details>132<summary><strong>Node.js</strong></summary>133134```bash135npm install crw-sdk136```137138```javascript139import { CrwClient } from "crw-sdk";140141const client = new CrwClient();142143const page = await client.scrape("https://example.com", {144 formats: ["markdown"],145});146147console.log(page.markdown);148```149150</details>151152[Local mode and more SDK examples →](https://docs.fastcrw.com/sdk-examples/) ·153[REST API →](https://docs.fastcrw.com/#rest-api)154155### MCP for AI agents156157```bash158npx -y crw-mcp@latest install159```160161Installs the CRW skill and MCP server in your detected AI tools. `crw setup` can162also do this step, so either path is enough.163[Manual setup →](https://docs.fastcrw.com/mcp-clients/)164165## Choose where it runs166167| | Managed API | Local / self-hosted |168|---|---|---|169| Best for | Zero infrastructure and managed scaling | Data control, private networks, or custom infrastructure |170| Start | [Create an API key](https://fastcrw.com/register?ref=gh-table), then `crw setup` | Install and run `crw <URL>` |171| Operations | Managed proxies, billing, and hosted capabilities | You choose renderers, search, auth, proxies, and capacity |172173Capabilities and response shapes can differ by deployment:174[`/v1/capabilities`](https://docs.fastcrw.com/capabilities/) · [response shapes](https://docs.fastcrw.com/response-shapes/)175176[Self-hosting guide →](https://docs.fastcrw.com/self-hosting/)177178## Learn more179180- [Quickstart](https://docs.fastcrw.com/quick-start/)181- [API reference](https://docs.fastcrw.com/#rest-api)182- [Benchmarks](BENCHMARKS.md)183- [Firecrawl migration](https://docs.fastcrw.com/migrate-from-firecrawl/)184- [Self-hosting](https://docs.fastcrw.com/self-hosting/)185186## Contributing187188The workspace requires Rust 1.85 or newer:189190```bash191git clone https://github.com/us/crw192cd crw193make check-fast194```195196[Read the contributor guide →](CONTRIBUTING.md)197198Engine and MCP server: [AGPL-3.0](LICENSE). Python and TypeScript SDKs: MIT.199Embedding license: hello@fastcrw.com.200201## Star History202203<a href="https://www.star-history.com/?repos=us%2Fcrw&type=date&legend=top-left">204 <picture>205 <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=us/crw&type=date&theme=dark&legend=top-left&sealed_token=Pe6pRWL7lqTM-St9eo-Cmpk5kYNyuyun0krw9eVZQFIrm3g_R2h46IW6wfNalPXquMsWSNCgKqiar1YVo9MGy2IZmN5Lz6rjZcjBCw6bCcRHORKORFRi9A" />206 <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=us/crw&type=date&legend=top-left&sealed_token=Pe6pRWL7lqTM-St9eo-Cmpk5kYNyuyun0krw9eVZQFIrm3g_R2h46IW6wfNalPXquMsWSNCgKqiar1YVo9MGy2IZmN5Lz6rjZcjBCw6bCcRHORKORFRi9A" />207 <img alt="Star History Chart" src="https://api.star-history.com/chart?repos=us/crw&type=date&legend=top-left&sealed_token=Pe6pRWL7lqTM-St9eo-Cmpk5kYNyuyun0krw9eVZQFIrm3g_R2h46IW6wfNalPXquMsWSNCgKqiar1YVo9MGy2IZmN5Lz6rjZcjBCw6bCcRHORKORFRi9A" />208 </picture>209</a>210211<details>212<summary>Contributors</summary>213214<!-- contributors:start -->215<p align="center">216 <a href="https://github.com/us" title="us"><img src="https://github.com/us.png?size=96" width="48" height="48" alt="us"/></a>217 <a href="https://github.com/santhreal" title="santhreal"><img src="https://github.com/santhreal.png?size=96" width="48" height="48" alt="santhreal"/></a>218 <a href="https://github.com/AsheTheWings" title="AsheTheWings"><img src="https://github.com/AsheTheWings.png?size=96" width="48" height="48" alt="AsheTheWings"/></a>219 <a href="https://github.com/adambenhassen" title="adambenhassen"><img src="https://github.com/adambenhassen.png?size=96" width="48" height="48" alt="adambenhassen"/></a>220 <a href="https://github.com/paoloantinori" title="paoloantinori"><img src="https://github.com/paoloantinori.png?size=96" width="48" height="48" alt="paoloantinori"/></a>221 <a href="https://github.com/VIVAAN-DHAWAN" title="VIVAAN-DHAWAN"><img src="https://github.com/VIVAAN-DHAWAN.png?size=96" width="48" height="48" alt="VIVAAN-DHAWAN"/></a>222 <a href="https://github.com/mj520" title="mj520"><img src="https://github.com/mj520.png?size=96" width="48" height="48" alt="mj520"/></a>223</p>224<!-- contributors:end -->225226</details>227228<sub>Please respect website policies. Crawl and map follow `robots.txt` by default.</sub>229
Full transparency — inspect the skill content before installing.