Open-WebSearch MCP Server

1<div align="center">
2 
3# Open-WebSearch MCP Server
4 
5[![ModelScope](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/Aas-ee/3af09e0f4c7821fb2e9acb96483a5ff0/raw/badge.json&color=%23de5a16)](https://www.modelscope.cn/mcp/servers/Aasee1/open-webSearch)
6[![Trust Score](https://archestra.ai/mcp-catalog/api/badge/quality/Aas-ee/open-webSearch)](https://archestra.ai/mcp-catalog/aas-ee__open-websearch)
7[![smithery badge](https://smithery.ai/badge/@Aas-ee/open-websearch)](https://smithery.ai/server/@Aas-ee/open-websearch)
8![Version](https://img.shields.io/github/v/release/Aas-ee/open-websearch)
9![License](https://img.shields.io/github/license/Aas-ee/open-websearch)
10![Issues](https://img.shields.io/github/issues/Aas-ee/open-websearch)
11 
12**[🇨🇳 中文](./README-zh.md) | 🇺🇸 English**
13 
14</div>
15 
16A Model Context Protocol (MCP) server based on multi-engine search results, supporting free web search without API keys.
17 
18## Features
19 
20- Web search using multi-engine results
21    - bing
22    - baidu
23    - ~~linux.do~~ temporarily unsupported
24    - csdn
25    - duckduckgo
26    - exa
27    - brave
28    - juejin
29- HTTP proxy configuration support for accessing restricted resources
30- No API keys or authentication required
31- Returns structured results with titles, URLs, and descriptions
32- Configurable number of results per search
33- Customizable default search engine
34- Support for fetching individual article content
35    - csdn
36    - github (README files)
37 
38## TODO
39- Support for ~~Bing~~ (already supported), ~~DuckDuckGo~~ (already supported), ~~Exa~~ (already supported), ~~Brave~~ (already supported), Google and other search engines
40- Support for more blogs, forums, and social platforms
41- Optimize article content extraction, add support for more sites
42- ~~Support for GitHub README fetching~~ (already supported)
43 
44## Installation Guide
45 
46### NPX Quick Start (Recommended)
47 
48The fastest way to get started:
49 
50```bash
51# Basic usage
52npx open-websearch@latest
53 
54# With environment variables (Linux/macOS)
55DEFAULT_SEARCH_ENGINE=duckduckgo ENABLE_CORS=true npx open-websearch@latest
56 
57# Windows PowerShell
58$env:DEFAULT_SEARCH_ENGINE="duckduckgo"; $env:ENABLE_CORS="true"; npx open-websearch@latest
59 
60# Windows CMD
61set MODE=stdio && set DEFAULT_SEARCH_ENGINE=duckduckgo && npx open-websearch@latest
62 
63# Cross-platform (requires cross-env, Used for local development)
64npm install -g open-websearch
65npx cross-env DEFAULT_SEARCH_ENGINE=duckduckgo ENABLE_CORS=true open-websearch
66```
67 
68**Environment Variables:**
69 
70| Variable | Default                 | Options | Description |
71|----------|-------------------------|---------|-------------|
72| `ENABLE_CORS` | `false`                 | `true`, `false` | Enable CORS |
73| `CORS_ORIGIN` | `*`                     | Any valid origin | CORS origin configuration |
74| `DEFAULT_SEARCH_ENGINE` | `bing`                  | `bing`, `duckduckgo`, `exa`, `brave`, `baidu`, `csdn`, `juejin` | Default search engine |
75| `USE_PROXY` | `false`                 | `true`, `false` | Enable HTTP proxy |
76| `PROXY_URL` | `http://127.0.0.1:7890` | Any valid URL | Proxy server URL |
77| `MODE` | `both`                  | `both`, `http`, `stdio` | Server mode: both HTTP+STDIO, HTTP only, or STDIO only |
78| `PORT` | `3000`                  | 1-65535 | Server port |
79| `ALLOWED_SEARCH_ENGINES` | empty (all available) | Comma-separated engine names | Limit which search engines can be used; if the default engine is not in this list, the first allowed engine becomes the default |
80| `MCP_TOOL_SEARCH_NAME` | `search` | Valid MCP tool name | Custom name for the search tool |
81| `MCP_TOOL_FETCH_LINUXDO_NAME` | `fetchLinuxDoArticle` | Valid MCP tool name | Custom name for the Linux.do article fetch tool |
82| `MCP_TOOL_FETCH_CSDN_NAME` | `fetchCsdnArticle` | Valid MCP tool name | Custom name for the CSDN article fetch tool |
83| `MCP_TOOL_FETCH_GITHUB_NAME` | `fetchGithubReadme` | Valid MCP tool name | Custom name for the GitHub README fetch tool |
84| `MCP_TOOL_FETCH_JUEJIN_NAME` | `fetchJuejinArticle` | Valid MCP tool name | Custom name for the Juejin article fetch tool |
85 
86**Common configurations:**
87```bash
88# Enable proxy for restricted regions
89USE_PROXY=true PROXY_URL=http://127.0.0.1:7890 npx open-websearch@latest
90 
91# Full configuration
92DEFAULT_SEARCH_ENGINE=duckduckgo ENABLE_CORS=true USE_PROXY=true PROXY_URL=http://127.0.0.1:7890 PORT=8080 npx open-websearch@latest
93```
94 
95### Local Installation
96 
971. Clone or download this repository
982. Install dependencies:
99```bash
100npm install
101```
1023. Build the server:
103```bash
104npm run build
105```
1064. Add the server to your MCP configuration:
107 
108**Cherry Studio:**
109```json
110{
111  "mcpServers": {
112    "web-search": {
113      "name": "Web Search MCP",
114      "type": "streamableHttp",
115      "description": "Multi-engine web search with article fetching",
116      "isActive": true,
117      "baseUrl": "http://localhost:3000/mcp"
118    }
119  }
120}
121```
122 
123**VSCode (Claude Dev Extension):**
124```json
125{
126  "mcpServers": {
127    "web-search": {
128      "transport": {
129        "type": "streamableHttp",
130        "url": "http://localhost:3000/mcp"
131      }
132    },
133    "web-search-sse": {
134      "transport": {
135        "type": "sse",
136        "url": "http://localhost:3000/sse"
137      }
138    }
139  }
140}
141```
142 
143**Claude Desktop:**
144```json
145{
146  "mcpServers": {
147    "web-search": {
148      "transport": {
149        "type": "streamableHttp",
150        "url": "http://localhost:3000/mcp"
151      }
152    },
153    "web-search-sse": {
154      "transport": {
155        "type": "sse",
156        "url": "http://localhost:3000/sse"
157      }
158    }
159  }
160}
161```
162 
163**NPX Command Line Configuration:**
164```json
165{
166  "mcpServers": {
167    "web-search": {
168      "args": [
169        "open-websearch@latest"
170      ],
171      "command": "npx",
172      "env": {
173        "MODE": "stdio",
174        "DEFAULT_SEARCH_ENGINE": "duckduckgo",
175        "ALLOWED_SEARCH_ENGINES": "duckduckgo,bing,exa"
176      }
177    }
178  }
179}
180```
181 
182**Local STDIO Configuration for Cherry Studio (Windows):**
183```json
184{
185  "mcpServers": {
186    "open-websearch-local": {
187      "command": "node",
188      "args": ["C:/path/to/your/project/build/index.js"],
189      "env": {
190        "MODE": "stdio",
191        "DEFAULT_SEARCH_ENGINE": "duckduckgo",
192        "ALLOWED_SEARCH_ENGINES": "duckduckgo,bing,exa"
193      }
194    }
195  }
196}
197```
198 
199### Docker Deployment
200 
201Quick deployment using Docker Compose:
202 
203```bash
204docker-compose up -d
205```
206 
207Or use Docker directly:
208```bash
209docker run -d --name web-search -p 3000:3000 -e ENABLE_CORS=true -e CORS_ORIGIN=* ghcr.io/aas-ee/open-web-search:latest
210```
211 
212Environment variable configuration:
213 
214| Variable | Default                 | Options | Description |
215|----------|-------------------------|---------|-------------|
216| `ENABLE_CORS` | `false`                 | `true`, `false` | Enable CORS |
217| `CORS_ORIGIN` | `*`                     | Any valid origin | CORS origin configuration |
218| `DEFAULT_SEARCH_ENGINE` | `bing`                  | `bing`, `duckduckgo`, `exa`, `brave` | Default search engine |
219| `USE_PROXY` | `false`                 | `true`, `false` | Enable HTTP proxy |
220| `PROXY_URL` | `http://127.0.0.1:7890` | Any valid URL | Proxy server URL |
221| `PORT` | `3000`                  | 1-65535 | Server port |
222 
223Then configure in your MCP client:
224```json
225{
226  "mcpServers": {
227    "web-search": {
228      "name": "Web Search MCP",
229      "type": "streamableHttp",
230      "description": "Multi-engine web search with article fetching",
231      "isActive": true,
232      "baseUrl": "http://localhost:3000/mcp"
233    },
234    "web-search-sse": {
235      "transport": {
236        "name": "Web Search MCP",
237        "type": "sse",
238        "description": "Multi-engine web search with article fetching",
239        "isActive": true,
240        "url": "http://localhost:3000/sse"
241      }
242    }
243  }
244}
245```
246 
247## Usage Guide
248 
249The server provides four tools: `search`, `fetchLinuxDoArticle`, `fetchCsdnArticle`, and `fetchGithubReadme`.
250 
251### search Tool Usage
252 
253```typescript
254{
255  "query": string,        // Search query
256  "limit": number,        // Optional: Number of results to return (default: 10)
257  "engines": string[]     // Optional: Engines to use (bing,baidu,linuxdo,csdn,duckduckgo,exa,brave,juejin) default bing
258}
259```
260 
261Usage example:
262```typescript
263use_mcp_tool({
264  server_name: "web-search",
265  tool_name: "search",
266  arguments: {
267    query: "search content",
268    limit: 3,  // Optional parameter
269    engines: ["bing", "csdn", "duckduckgo", "exa", "brave", "juejin"] // Optional parameter, supports multi-engine combined search
270  }
271})
272```
273 
274Response example:
275```json
276[
277  {
278    "title": "Example Search Result",
279    "url": "https://example.com",
280    "description": "Description text of the search result...",
281    "source": "Source",
282    "engine": "Engine used"
283  }
284]
285```
286 
287### fetchCsdnArticle Tool Usage
288 
289Used to fetch complete content of CSDN blog articles.
290 
291```typescript
292{
293  "url": string    // URL from CSDN search results using the search tool
294}
295```
296 
297Usage example:
298```typescript
299use_mcp_tool({
300  server_name: "web-search",
301  tool_name: "fetchCsdnArticle",
302  arguments: {
303    url: "https://blog.csdn.net/xxx/article/details/xxx"
304  }
305})
306```
307 
308Response example:
309```json
310[
311  {
312    "content": "Example search result"
313  }
314]
315```
316 
317### fetchLinuxDoArticle Tool Usage
318 
319Used to fetch complete content of Linux.do forum articles.
320 
321```typescript
322{
323  "url": string    // URL from linuxdo search results using the search tool
324}
325```
326 
327Usage example:
328```typescript
329use_mcp_tool({
330  server_name: "web-search",
331  tool_name: "fetchLinuxDoArticle",
332  arguments: {
333    url: "https://xxxx.json"
334  }
335})
336```
337 
338Response example:
339```json
340[
341  {
342    "content": "Example search result"
343  }
344]
345```
346 
347### fetchGithubReadme Tool Usage
348 
349Used to fetch README content from GitHub repositories.
350 
351```typescript
352{
353  "url": string    // GitHub repository URL (supports HTTPS, SSH formats)
354}
355```
356 
357Usage example:
358```typescript
359use_mcp_tool({
360  server_name: "web-search",
361  tool_name: "fetchGithubReadme",
362  arguments: {
363    url: "https://github.com/Aas-ee/open-webSearch"
364  }
365})
366```
367 
368Supported URL formats:
369- HTTPS: `https://github.com/owner/repo`
370- HTTPS with .git: `https://github.com/owner/repo.git`
371- SSH: `git@github.com:owner/repo.git`
372- URLs with parameters: `https://github.com/owner/repo?tab=readme`
373 
374Response example:
375```json
376[
377  {
378    "content": "<div align=\"center\">\n\n# Open-WebSearch MCP Server..."
379  }
380]
381```
382 
383### fetchJuejinArticle Tool Usage
384 
385Used to fetch complete content of Juejin articles.
386 
387```typescript
388{
389  "url": string    // Juejin article URL from search results
390}
391```
392 
393Usage example:
394```typescript
395use_mcp_tool({
396  server_name: "web-search",
397  tool_name: "fetchJuejinArticle",
398  arguments: {
399    url: "https://juejin.cn/post/7520959840199360563"
400  }
401})
402```
403 
404Supported URL format:
405- `https://juejin.cn/post/{article_id}`
406 
407Response example:
408```json
409[
410  {
411    "content": "🚀 开源 AI 联网搜索工具：Open-WebSearch MCP 全新升级，支持多引擎 + 流式响应..."
412  }
413]
414```
415 
416## Usage Limitations
417 
418Since this tool works by scraping multi-engine search results, please note the following important limitations:
419 
4201. **Rate Limiting**:
421    - Too many searches in a short time may cause the used engines to temporarily block requests
422    - Recommendations:
423        - Maintain reasonable search frequency
424        - Use the limit parameter judiciously
425        - Add delays between searches when necessary
426 
4272. **Result Accuracy**:
428    - Depends on the HTML structure of corresponding engines, may fail when engines update
429    - Some results may lack metadata like descriptions
430    - Complex search operators may not work as expected
431 
4323. **Legal Terms**:
433    - This tool is for personal use only
434    - Please comply with the terms of service of corresponding engines
435    - Implement appropriate rate limiting based on your actual use case
436 
4374. **Search Engine Configuration**:
438   - Default search engine can be set via the `DEFAULT_SEARCH_ENGINE` environment variable
439   - Supported engines: bing, duckduckgo, exa, brave
440   - The default engine is used when searching specific websites
441 
4425. **Proxy Configuration**:
443   - HTTP proxy can be configured when certain search engines are unavailable in specific regions
444   - Enable proxy with environment variable `USE_PROXY=true`
445   - Configure proxy server address with `PROXY_URL`
446 
447## Contributing
448 
449Welcome to submit issue reports and feature improvement suggestions!
450 
451### Contributor Guide
452 
453If you want to fork this repository and publish your own Docker image, you need to make the following configurations:
454 
455#### GitHub Secrets Configuration
456 
457To enable automatic Docker image building and publishing, please add the following secrets in your GitHub repository settings (Settings → Secrets and variables → Actions):
458 
459**Required Secrets:**
460- `GITHUB_TOKEN`: Automatically provided by GitHub (no setup needed)
461 
462**Optional Secrets (for Alibaba Cloud ACR):**
463- `ACR_REGISTRY`: Your Alibaba Cloud Container Registry URL (e.g., `registry.cn-hangzhou.aliyuncs.com`)
464- `ACR_USERNAME`: Your Alibaba Cloud ACR username
465- `ACR_PASSWORD`: Your Alibaba Cloud ACR password
466- `ACR_IMAGE_NAME`: Your image name in ACR (e.g., `your-namespace/open-web-search`)
467 
468#### CI/CD Workflow
469 
470The repository includes a GitHub Actions workflow (`.github/workflows/docker.yml`) that automatically:
471 
4721. **Trigger Conditions**:
473    - Push to `main` branch
474    - Push version tags (`v*`)
475    - Manual workflow trigger
476 
4772. **Build and Push to**:
478    - GitHub Container Registry (ghcr.io) - always enabled
479    - Alibaba Cloud Container Registry - only enabled when ACR secrets are configured
480 
4813. **Image Tags**:
482    - `ghcr.io/your-username/open-web-search:latest`
483    - `your-acr-address/your-image-name:latest` (if ACR is configured)
484 
485#### Fork and Publish Steps:
486 
4871. **Fork the repository** to your GitHub account
4882. **Configure secrets** (if you need ACR publishing):
489    - Go to Settings → Secrets and variables → Actions in your forked repository
490    - Add the ACR-related secrets listed above
4913. **Push changes** to the `main` branch or create version tags
4924. **GitHub Actions will automatically build and push** your Docker image
4935. **Use your image**, update the Docker command:
494   ```bash
495   docker run -d --name web-search -p 3000:3000 -e ENABLE_CORS=true -e CORS_ORIGIN=* ghcr.io/your-username/open-web-search:latest
496   ```
497 
498#### Notes:
499- If you don't configure ACR secrets, the workflow will only publish to GitHub Container Registry
500- Make sure your GitHub repository has Actions enabled
501- The workflow will use your GitHub username (converted to lowercase) as the GHCR image name
502 
503<div align="center">
504 
505## Star History
506If you find this project helpful, please consider giving it a ⭐ Star!
507 
508[![Star History Chart](https://api.star-history.com/svg?repos=Aas-ee/open-webSearch&type=Date)](https://www.star-history.com/#Aas-ee/open-webSearch&Date)
509 
510</div>
511