Pixelle MCP - Omnimodal Agent Framework

1<h1 align="center">🎨 Pixelle MCP - Omnimodal Agent Framework</h1>
2 
3<p align="center"><b>English</b> | <a href="README_CN.md">中文</a></p>
4 
5<p align="center">✨ An AIGC solution based on the MCP protocol, supporting both local ComfyUI and cloud ComfyUI (RunningHub) modes, seamlessly converting workflows into MCP tools with zero code.</p>
6 
7![](docs/readme-1.png)
8 
9https://github.com/user-attachments/assets/65422cef-96f9-44fe-a82b-6a124674c417
10 
11 
12## 📋 Recent Updates
13 
14- ✅ **2025-09-29**: Added RunningHub cloud ComfyUI support, enabling workflow execution without local GPU and ComfyUI environment
15- ✅ **2025-09-03**: Architecture refactoring from three services to unified application; added CLI tool support; published to [PyPI](https://pypi.org/project/pixelle/)
16- ✅ **2025-08-12**: Integrated the LiteLLM framework, adding multi-model support for Gemini, DeepSeek, Claude, Qwen, and more
17 
18 
19## 🚀 Features
20 
21- ✅ 🔄 **Full-modal Support**: Supports TISV (Text, Image, Sound/Speech, Video) full-modal conversion and generation
22- ✅ 🚀 **Dual Execution Modes**: Local ComfyUI self-hosted environment + RunningHub cloud ComfyUI service, users can flexibly choose based on their needs
23- ✅ 🧩 **ComfyUI Ecosystem**: Built on [ComfyUI](https://github.com/comfyanonymous/ComfyUI), inheriting all capabilities from the open ComfyUI ecosystem
24- ✅ 🔧 **Zero-code Development**: Defines and implements the Workflow-as-MCP Tool solution, enabling zero-code development and dynamic addition of new MCP Tools
25- ✅ 🗄️ **MCP Server**: Based on the [MCP](https://modelcontextprotocol.io/introduction) protocol, supporting integration with any MCP client (including but not limited to Cursor, Claude Desktop, etc.)
26- ✅ 🌐 **Web Interface**: Developed based on the [Chainlit](https://github.com/Chainlit/chainlit) framework, inheriting Chainlit's UI controls and supporting integration with more MCP Servers
27- ✅ 📦 **One-click Deployment**: Supports PyPI installation, CLI commands, Docker and other deployment methods, ready to use out of the box
28- ✅ ⚙️ **Simplified Configuration**: Uses environment variable configuration scheme, simple and intuitive configuration
29- ✅ 🤖 **Multi-LLM Support**: Supports multiple mainstream LLMs, including OpenAI, Ollama, Gemini, DeepSeek, Claude, Qwen, and more
30 
31 
32## 📁 Project Architecture
33 
34Pixelle MCP adopts a **unified architecture design**, integrating MCP server, web interface, and file services into one application, providing:
35 
36- 🌐 **Web Interface**: Chainlit-based chat interface supporting multimodal interaction
37- 🔌 **MCP Endpoint**: For external MCP clients (such as Cursor, Claude Desktop) to connect
38- 📁 **File Service**: Handles file upload, download, and storage
39- 🛠️ **Workflow Engine**: Supports both local ComfyUI and cloud ComfyUI (RunningHub) workflows, automatically converts workflows into MCP tools
40 
41![](docs/%20mcp_structure.png)
42 
43<div id="tutorial-start" />
44 
45## 🏃‍♂️ Quick Start
46 
47Choose the deployment method that best suits your needs, from simple to complex:
48 
49### 🎯 Method 1: One-click Experience
50 
51> 💡 **Zero configuration startup, perfect for quick experience and testing**
52 
53#### 🚀 Temporary Run
54 
55```bash
56# First you need to install the uv environment
57# Start with one command, no system installation required
58uvx pixelle@latest
59```
60 
61📚 **[View uvx CLI Reference →](docs/CLI.md#uvx-method)**
62 
63#### 📦 Persistent Installation
64 
65```bash
66# Here you need to install it in the python3.11 environment
67# Install to system
68pip install -U pixelle
69 
70# Start service
71pixelle
72```
73 
74📚 **[View pip CLI Reference →](docs/CLI.md#pip-install-method)**
75 
76After startup, it will automatically enter the **configuration wizard** to guide you through execution engine selection (ComfyUI/RunningHub) and LLM configuration.
77 
78### 🛠️ Method 2: Local Development Deployment
79 
80> 💡 **Supports custom workflows and secondary development**
81 
82#### 📥 1. Get Source Code
83 
84```bash
85git clone https://github.com/AIDC-AI/Pixelle-MCP.git
86cd Pixelle-MCP
87```
88 
89#### 🚀 2. Start Service
90 
91```bash
92# Interactive mode (recommended)
93uv run pixelle
94```
95 
96📚 **[View Complete CLI Reference →](docs/CLI.md#uv-run-method)**
97 
98#### 🔧 3. Add Custom Workflows (Optional)
99 
100```bash
101# Copy example workflows to data directory (run this in your desired project directory)
102cp -r workflows/* ./data/custom_workflows/
103```
104 
105**⚠️ Important**: Make sure to test workflows in ComfyUI first to ensure they run properly, otherwise execution will fail.
106 
107### 🐳 Method 3: Docker Deployment
108 
109> 💡 **Suitable for production environments and containerized deployment**
110 
111#### 📋 1. Prepare Configuration
112 
113```bash
114git clone https://github.com/AIDC-AI/Pixelle-MCP.git
115cd Pixelle-MCP
116 
117# Create environment configuration file
118cp .env.example .env
119# Edit .env file to configure your ComfyUI address and LLM settings
120```
121 
122#### 🚀 2. Start Container
123 
124```bash
125# Start all services in background
126docker compose up -d
127 
128# View logs
129docker compose logs -f
130```
131 
132### 🌐 Access Services
133 
134Regardless of which method you use, after startup you can access via:
135 
136- **🌐 Web Interface**: http://localhost:9004  
137  *Default username and password are both `dev`, can be modified after startup*
138- **🔌 MCP Endpoint**: http://localhost:9004/pixelle/mcp  
139  *For MCP clients like Cursor, Claude Desktop to connect*
140 
141**💡 Port Configuration**: Default port is 9004, can be customized via environment variable `PORT=your_port`.
142 
143### ⚙️ Initial Configuration
144 
145On first startup, the system will automatically detect configuration status:
146 
1471. **🚀 Execution Engine Selection**: Choose between local ComfyUI or RunningHub cloud service
1482. **🤖 LLM Configuration**: Configure at least one LLM provider (OpenAI, Ollama, etc.)
1493. **📁 Workflow Directory**: System will automatically create necessary directory structure
150 
151### 🌐 RunningHub Cloud Mode Advantages
152- ✅ **Zero Hardware Requirements**: No need for local GPU or high-performance hardware
153- ✅ **No Environment Setup**: No need to install and configure ComfyUI locally
154- ✅ **Ready to Use**: Register and get API key to start immediately
155- ✅ **Stable Performance**: Professional cloud infrastructure ensures stable execution
156- ✅ **Auto Scaling**: Automatically handles concurrent requests and resource allocation
157 
158### 🏠 Local ComfyUI Mode Advantages
159- ✅ **Full Control**: Complete control over execution environment and model versions
160- ✅ **Privacy Protection**: All data processing happens locally, ensuring data privacy
161- ✅ **Custom Models**: Support for custom models and nodes not available in cloud
162- ✅ **No Network Dependency**: Can work offline without internet connection
163- ✅ **Cost Control**: No cloud service fees for high-frequency usage
164 
165**🆘 Need Help?** Join community groups for support (see Community section below)
166 
167## 🛠️ Add Your Own MCP Tool
168 
169⚡ One workflow = One MCP Tool, supports two addition methods:
170 
171📋 **Method 1: Local ComfyUI Workflow** - Export API format workflow files
172📋 **Method 2: RunningHub Workflow ID** - Use cloud workflow IDs directly
173 
174![](docs/workflow_to_mcp_tool.png)
175 
176### 🎯 1. Add the Simplest MCP Tool
177 
178* 📝 Build a workflow in ComfyUI for image Gaussian blur ([Get it here](docs/i_blur_ui.json)), then set the `LoadImage` node's title to `$image.image!` as shown below:
179![](docs/easy-workflow.png)
180 
181* 📤 Export it as an API format file and rename it to `i_blur.json`. You can export it yourself or use our pre-exported version ([Get it here](docs/i_blur.json))
182 
183* 📋 Copy the exported API workflow file (must be API format), input it on the web page, and let the LLM add this Tool
184 
185  ![](docs/ready_to_send_en.png)
186 
187* ✨ After sending, the LLM will automatically convert this workflow into an MCP Tool
188 
189  ![](docs/added_mcp_en.png)
190 
191* 🎨 Now, refresh the page and send any image to perform Gaussian blur processing via LLM
192 
193  ![](docs/use_mcp_tool_en.png)
194 
195### 🔌 2. Add a Complex MCP Tool
196 
197The steps are the same as above, only the workflow part differs (Download workflow: [UI format](docs/t2i_by_flux_turbo_ui.json) and [API format](docs/t2i_by_flux_turbo.json))
198 
199> **Note:** When using RunningHub, you only need to input the corresponding workflow ID, no need to download and upload workflow files.
200 
201![](docs/t2i_by_flux_turbo.png)
202 
203 
204## 🔧 ComfyUI Workflow Custom Specification
205 
206### 🎨 Workflow Format
207The system supports ComfyUI workflows. Just design your workflow in the canvas and export it as API format. Use special syntax in node titles to define parameters and outputs.
208 
209### 📝 Parameter Definition Specification
210 
211In the ComfyUI canvas, double-click the node title to edit, and use the following DSL syntax to define parameters:
212 
213```
214$<param_name>.[~]<field_name>[!][:<description>]
215```
216 
217#### 🔍 Syntax Explanation:
218- `param_name`: The parameter name for the generated MCP tool function
219- `~`: Optional, indicates URL parameter upload processing, returns relative path
220- `field_name`: The corresponding input field in the node
221- `!`: Indicates this parameter is required
222- `description`: Description of the parameter
223 
224#### 💡 Example:
225 
226**Required parameter example:**
227 
228- Set LoadImage node title to: `$image.image!:Input image URL`
229- Meaning: Creates a required parameter named `image`, mapped to the node's `image` field
230 
231**URL upload processing example:**
232 
233- Set any node title to: `$image.~image!:Input image URL`
234- Meaning: Creates a required parameter named `image`, system will automatically download URL and upload to ComfyUI, returns relative path
235 
236> 📝 Note: `LoadImage`, `VHS_LoadAudioUpload`, `VHS_LoadVideo` and other nodes have built-in functionality, no need to add `~` marker
237 
238 
239### 🎯 Type Inference Rules
240 
241The system automatically infers parameter types based on the current value of the node field:
242- 🔢 `int`: Integer values (e.g. 512, 1024)
243- 📊 `float`: Floating-point values (e.g. 1.5, 3.14)
244- ✅ `bool`: Boolean values (e.g. true, false)
245- 📝 `str`: String values (default type)
246 
247### 📤 Output Definition Specification
248 
249#### 🤖 Method 1: Auto-detect Output Nodes
250The system will automatically detect the following common output nodes:
251- 🖼️ `SaveImage` - Image save node
252- 🎬 `SaveVideo` - Video save node
253- 🔊 `SaveAudio` - Audio save node
254- 📹 `VHS_SaveVideo` - VHS video save node
255- 🎵 `VHS_SaveAudio` - VHS audio save node
256 
257#### 🎯 Method 2: Manual Output Marking
258> Usually used for multiple outputs
259Use `$output.var_name` in any node title to mark output:
260- Set node title to: `$output.result`
261- The system will use this node's output as the tool's return value
262 
263 
264### 📄 Tool Description Configuration (Optional)
265 
266You can add a node titled `MCP` in the workflow to provide a tool description:
267 
2681. Add a `String (Multiline)` or similar text node (must have a single string property, and the node field should be one of: value, text, string)
2692. Set the node title to: `MCP`
2703. Enter a detailed tool description in the value field
271 
272 
273### ⚠️ Important Notes
274 
2751. **🔒 Parameter Validation**: Optional parameters (without !) must have default values set in the node
2762. **🔗 Node Connections**: Fields already connected to other nodes will not be parsed as parameters
2773. **🏷️ Tool Naming**: Exported file name will be used as the tool name, use meaningful English names
2784. **📋 Detailed Descriptions**: Provide detailed parameter descriptions for better user experience
2795. **🎯 Export Format**: Must export as API format, do not export as UI format
280 
281<div id="tutorial-end" />
282 
283## 💬 Community
284 
285Scan the QR codes below to join our communities for latest updates and technical support:
286 
287|                      Discord Community                       |                         WeChat Group                         |
288| :----------------------------------------------------------: | :----------------------------------------------------------: |
289| <img src="docs/discord.png" alt="Discord Community" width="250" /> | <img src="docs/wechat.png" alt="WeChat Group" width="250" /> |
290 
291## 🤝 How to Contribute
292 
293We welcome all forms of contribution! Whether you're a developer, designer, or user, you can participate in the project in the following ways:
294 
295### 🐛 Report Issues
296* 📋 Submit bug reports on the [Issues](https://github.com/AIDC-AI/Pixelle-MCP/issues) page
297* 🔍 Please search for similar issues before submitting
298* 📝 Describe the reproduction steps and environment in detail
299 
300### 💡 Feature Suggestions
301* 🚀 Submit feature requests in [Issues](https://github.com/AIDC-AI/Pixelle-MCP/issues)
302* 💭 Describe the feature you want and its use case
303* 🎯 Explain how it improves user experience
304 
305### 🔧 Code Contributions
306 
307#### 📋 Contribution Process
3081. 🍴 Fork this repo to your GitHub account
3092. 🌿 Create a feature branch: `git checkout -b feature/your-feature-name`
3103. 💻 Develop and add corresponding tests
3114. 📝 Commit changes: `git commit -m "feat: add your feature"`
3125. 📤 Push to your repo: `git push origin feature/your-feature-name`
3136. 🔄 Create a Pull Request to the main repo
314 
315#### 🎨 Code Style
316* 🐍 Python code follows [PEP 8](https://pep8.org/) style guide
317* 📖 Add appropriate documentation and comments for new features
318 
319### 🧩 Contribute Workflows
320* 📦 Share your ComfyUI workflows with the community
321* 🛠️ Submit tested workflow files
322* 📚 Add usage instructions and examples for workflows
323 
324## 🙏 Acknowledgements
325 
326❤️ Sincere thanks to the following organizations, projects, and teams for supporting the development and implementation of this project.
327 
328* 🧩 [ComfyUI](https://github.com/comfyanonymous/ComfyUI)
329* 💬 [Chainlit](https://github.com/Chainlit/chainlit)
330 
331* 🔌 [MCP](https://modelcontextprotocol.io/introduction)
332* 🎬 [WanVideo](https://github.com/Wan-Video/Wan2.1)
333* ⚡ [Flux](https://github.com/black-forest-labs/flux)
334* 🤖 [LiteLLM](https://github.com/BerriAI/litellm)
335 
336## License
337This project is released under the MIT License ([LICENSE](LICENSE), SPDX-License-identifier: MIT).
338 
339## ⭐ Star History
340 
341[![Star History Chart](https://api.star-history.com/svg?repos=AIDC-AI/Pixelle-MCP&type=Date)](https://star-history.com/#AIDC-AI/Pixelle-MCP&Date)
342