ScreenPilot

1[![MseeP.ai Security Assessment Badge](https://mseep.net/pr/mtehabsim-screenpilot-badge.png)](https://mseep.ai/app/mtehabsim-screenpilot)
2 
3[![Trust Score](https://archestra.ai/mcp-catalog/api/badge/quality/Mtehabsim/ScreenPilot)](https://archestra.ai/mcp-catalog/mtehabsim__screenpilot)
4 
5# ScreenPilot
6 
7MCP server to let LLM take full control on your device by providing screen automation toolkit for controlling and interacting with graphical user interfaces. Good for automation, education and having fun.
8 
9 
10## Main Features
11 
12- 📷 Screen capture and analysis
13- 🖱️ Mouse control (clicking, positioning)
14- ⌨️ Keyboard input (typing, key presses, hotkeys)
15 
16## watch demo
17 
18 
19https://github.com/user-attachments/assets/c18380c0-b3dd-4b7c-925d-28ef205ca11f
20 
21 
22 
23## Installation
240. Install python 3.12
251. Clone the repository:
26   ```bash
27   git clone https://github.com/Mtehabsim/ScreenPilot.git
28   ```
292. create virtiual environment
30```bash
31 
32python -m venv venv
33```
343. activate the env
35```bash
36venv\Scripts\activate
37```
384. Install the required packages:
39   ```bash
40   pip install -r requirements.txt
41   ```
425. Open Claude AI desktop
436. file -> settings -> developer -> edit config
447. open config file and paste this
45```bash
46{
47    "mcpServers": {
48        "device-controll": {
49            "command": "pathToEnv\\venv\\Scripts\\python.exe",
50            "args": [
51                "pathToProject\\ScreenPilot\\main.py"
52            ]
53        }
54    }
55}
56 
57```
588. Replace
59    "pathToEnv\\venv\\Scripts\\python.exe" → with the full path to your python.exe
60    "pathToProject\\ScreenPilot\\main.py" → with the full path to your main.py file
61 
629. Save the config file.
63 
6410. Open Claude AI Desktop.
65 
6611. Go to File → Exit
67 
6812. You can now open Claude AI Desktop and enjoy ScreenPilot.
69 
70 
71### Available Tools
72 
73- **Screen Capture**: Take screenshots and get screen information
74- **Mouse Control**: Move the mouse and perform clicks
75- **Keyboard Actions**: Type text, press keys, and use hotkey combinations
76- **Scrolling**: Scroll in different directions and to specific positions
77- **Element Detection**: Check if elements exist on screen and wait for them to appear
78- **Action Sequences**: Perform multiple actions in sequence
79 
80## Contributing
81 
82Contributions are welcome! Please feel free to submit a Pull Request.
83 
84