Autonomous Agent Patterns

1---
2name: autonomous-agent-patterns
3description: "Design patterns for building autonomous coding agents. Covers tool integration, permission systems, browser automation, and human-in-the-loop workflows. Use when building AI agents, designing tool APIs, implementing permission systems, or creating autonomous coding assistants."
4---
5 
6# 🕹️ Autonomous Agent Patterns
7 
8> Design patterns for building autonomous coding agents, inspired by [Cline](https://github.com/cline/cline) and [OpenAI Codex](https://github.com/openai/codex).
9 
10## When to Use This Skill
11 
12Use this skill when:
13 
14- Building autonomous AI agents
15- Designing tool/function calling APIs
16- Implementing permission and approval systems
17- Creating browser automation for agents
18- Designing human-in-the-loop workflows
19 
20---
21 
22## 1. Core Agent Architecture
23 
24### 1.1 Agent Loop
25 
26```
27┌─────────────────────────────────────────────────────────────┐
28│                     AGENT LOOP                               │
29│                                                              │
30│  ┌──────────┐    ┌──────────┐    ┌──────────┐              │
31│  │  Think   │───▶│  Decide  │───▶│   Act    │              │
32│  │ (Reason) │    │ (Plan)   │    │ (Execute)│              │
33│  └──────────┘    └──────────┘    └──────────┘              │
34│       ▲                               │                     │
35│       │         ┌──────────┐          │                     │
36│       └─────────│ Observe  │◀─────────┘                     │
37│                 │ (Result) │                                │
38│                 └──────────┘                                │
39└─────────────────────────────────────────────────────────────┘
40```
41 
42```python
43class AgentLoop:
44    def __init__(self, llm, tools, max_iterations=50):
45        self.llm = llm
46        self.tools = {t.name: t for t in tools}
47        self.max_iterations = max_iterations
48        self.history = []
49 
50    def run(self, task: str) -> str:
51        self.history.append({"role": "user", "content": task})
52 
53        for i in range(self.max_iterations):
54            # Think: Get LLM response with tool options
55            response = self.llm.chat(
56                messages=self.history,
57                tools=self._format_tools(),
58                tool_choice="auto"
59            )
60 
61            # Decide: Check if agent wants to use a tool
62            if response.tool_calls:
63                for tool_call in response.tool_calls:
64                    # Act: Execute the tool
65                    result = self._execute_tool(tool_call)
66 
67                    # Observe: Add result to history
68                    self.history.append({
69                        "role": "tool",
70                        "tool_call_id": tool_call.id,
71                        "content": str(result)
72                    })
73            else:
74                # No more tool calls = task complete
75                return response.content
76 
77        return "Max iterations reached"
78 
79    def _execute_tool(self, tool_call) -> Any:
80        tool = self.tools[tool_call.name]
81        args = json.loads(tool_call.arguments)
82        return tool.execute(**args)
83```
84 
85### 1.2 Multi-Model Architecture
86 
87```python
88class MultiModelAgent:
89    """
90    Use different models for different purposes:
91    - Fast model for planning
92    - Powerful model for complex reasoning
93    - Specialized model for code generation
94    """
95 
96    def __init__(self):
97        self.models = {
98            "fast": "gpt-3.5-turbo",      # Quick decisions
99            "smart": "gpt-4-turbo",        # Complex reasoning
100            "code": "claude-3-sonnet",     # Code generation
101        }
102 
103    def select_model(self, task_type: str) -> str:
104        if task_type == "planning":
105            return self.models["fast"]
106        elif task_type == "analysis":
107            return self.models["smart"]
108        elif task_type == "code":
109            return self.models["code"]
110        return self.models["smart"]
111```
112 
113---
114 
115## 2. Tool Design Patterns
116 
117### 2.1 Tool Schema
118 
119```python
120class Tool:
121    """Base class for agent tools"""
122 
123    @property
124    def schema(self) -> dict:
125        """JSON Schema for the tool"""
126        return {
127            "name": self.name,
128            "description": self.description,
129            "parameters": {
130                "type": "object",
131                "properties": self._get_parameters(),
132                "required": self._get_required()
133            }
134        }
135 
136    def execute(self, **kwargs) -> ToolResult:
137        """Execute the tool and return result"""
138        raise NotImplementedError
139 
140class ReadFileTool(Tool):
141    name = "read_file"
142    description = "Read the contents of a file from the filesystem"
143 
144    def _get_parameters(self):
145        return {
146            "path": {
147                "type": "string",
148                "description": "Absolute path to the file"
149            },
150            "start_line": {
151                "type": "integer",
152                "description": "Line to start reading from (1-indexed)"
153            },
154            "end_line": {
155                "type": "integer",
156                "description": "Line to stop reading at (inclusive)"
157            }
158        }
159 
160    def _get_required(self):
161        return ["path"]
162 
163    def execute(self, path: str, start_line: int = None, end_line: int = None) -> ToolResult:
164        try:
165            with open(path, 'r') as f:
166                lines = f.readlines()
167 
168            if start_line and end_line:
169                lines = lines[start_line-1:end_line]
170 
171            return ToolResult(
172                success=True,
173                output="".join(lines)
174            )
175        except FileNotFoundError:
176            return ToolResult(
177                success=False,
178                error=f"File not found: {path}"
179            )
180```
181 
182### 2.2 Essential Agent Tools
183 
184```python
185CODING_AGENT_TOOLS = {
186    # File operations
187    "read_file": "Read file contents",
188    "write_file": "Create or overwrite a file",
189    "edit_file": "Make targeted edits to a file",
190    "list_directory": "List files and folders",
191    "search_files": "Search for files by pattern",
192 
193    # Code understanding
194    "search_code": "Search for code patterns (grep)",
195    "get_definition": "Find function/class definition",
196    "get_references": "Find all references to a symbol",
197 
198    # Terminal
199    "run_command": "Execute a shell command",
200    "read_output": "Read command output",
201    "send_input": "Send input to running command",
202 
203    # Browser (optional)
204    "open_browser": "Open URL in browser",
205    "click_element": "Click on page element",
206    "type_text": "Type text into input",
207    "screenshot": "Capture screenshot",
208 
209    # Context
210    "ask_user": "Ask the user a question",
211    "search_web": "Search the web for information"
212}
213```
214 
215### 2.3 Edit Tool Design
216 
217```python
218class EditFileTool(Tool):
219    """
220    Precise file editing with conflict detection.
221    Uses search/replace pattern for reliable edits.
222    """
223 
224    name = "edit_file"
225    description = "Edit a file by replacing specific content"
226 
227    def execute(
228        self,
229        path: str,
230        search: str,
231        replace: str,
232        expected_occurrences: int = 1
233    ) -> ToolResult:
234        """
235        Args:
236            path: File to edit
237            search: Exact text to find (must match exactly, including whitespace)
238            replace: Text to replace with
239            expected_occurrences: How many times search should appear (validation)
240        """
241        with open(path, 'r') as f:
242            content = f.read()
243 
244        # Validate
245        actual_occurrences = content.count(search)
246        if actual_occurrences != expected_occurrences:
247            return ToolResult(
248                success=False,
249                error=f"Expected {expected_occurrences} occurrences, found {actual_occurrences}"
250            )
251 
252        if actual_occurrences == 0:
253            return ToolResult(
254                success=False,
255                error="Search text not found in file"
256            )
257 
258        # Apply edit
259        new_content = content.replace(search, replace)
260 
261        with open(path, 'w') as f:
262            f.write(new_content)
263 
264        return ToolResult(
265            success=True,
266            output=f"Replaced {actual_occurrences} occurrence(s)"
267        )
268```
269 
270---
271 
272## 3. Permission & Safety Patterns
273 
274### 3.1 Permission Levels
275 
276```python
277class PermissionLevel(Enum):
278    # Fully automatic - no user approval needed
279    AUTO = "auto"
280 
281    # Ask once per session
282    ASK_ONCE = "ask_once"
283 
284    # Ask every time
285    ASK_EACH = "ask_each"
286 
287    # Never allow
288    NEVER = "never"
289 
290PERMISSION_CONFIG = {
291    # Low risk - can auto-approve
292    "read_file": PermissionLevel.AUTO,
293    "list_directory": PermissionLevel.AUTO,
294    "search_code": PermissionLevel.AUTO,
295 
296    # Medium risk - ask once
297    "write_file": PermissionLevel.ASK_ONCE,
298    "edit_file": PermissionLevel.ASK_ONCE,
299 
300    # High risk - ask each time
301    "run_command": PermissionLevel.ASK_EACH,
302    "delete_file": PermissionLevel.ASK_EACH,
303 
304    # Dangerous - never auto-approve
305    "sudo_command": PermissionLevel.NEVER,
306    "format_disk": PermissionLevel.NEVER
307}
308```
309 
310### 3.2 Approval UI Pattern
311 
312```python
313class ApprovalManager:
314    def __init__(self, ui, config):
315        self.ui = ui
316        self.config = config
317        self.session_approvals = {}
318 
319    def request_approval(self, tool_name: str, args: dict) -> bool:
320        level = self.config.get(tool_name, PermissionLevel.ASK_EACH)
321 
322        if level == PermissionLevel.AUTO:
323            return True
324 
325        if level == PermissionLevel.NEVER:
326            self.ui.show_error(f"Tool '{tool_name}' is not allowed")
327            return False
328 
329        if level == PermissionLevel.ASK_ONCE:
330            if tool_name in self.session_approvals:
331                return self.session_approvals[tool_name]
332 
333        # Show approval dialog
334        approved = self.ui.show_approval_dialog(
335            tool=tool_name,
336            args=args,
337            risk_level=self._assess_risk(tool_name, args)
338        )
339 
340        if level == PermissionLevel.ASK_ONCE:
341            self.session_approvals[tool_name] = approved
342 
343        return approved
344 
345    def _assess_risk(self, tool_name: str, args: dict) -> str:
346        """Analyze specific call for risk level"""
347        if tool_name == "run_command":
348            cmd = args.get("command", "")
349            if any(danger in cmd for danger in ["rm -rf", "sudo", "chmod"]):
350                return "HIGH"
351        return "MEDIUM"
352```
353 
354### 3.3 Sandboxing
355 
356```python
357class SandboxedExecution:
358    """
359    Execute code/commands in isolated environment
360    """
361 
362    def __init__(self, workspace_dir: str):
363        self.workspace = workspace_dir
364        self.allowed_commands = ["npm", "python", "node", "git", "ls", "cat"]
365        self.blocked_paths = ["/etc", "/usr", "/bin", os.path.expanduser("~")]
366 
367    def validate_path(self, path: str) -> bool:
368        """Ensure path is within workspace"""
369        real_path = os.path.realpath(path)
370        workspace_real = os.path.realpath(self.workspace)
371        return real_path.startswith(workspace_real)
372 
373    def validate_command(self, command: str) -> bool:
374        """Check if command is allowed"""
375        cmd_parts = shlex.split(command)
376        if not cmd_parts:
377            return False
378 
379        base_cmd = cmd_parts[0]
380        return base_cmd in self.allowed_commands
381 
382    def execute_sandboxed(self, command: str) -> ToolResult:
383        if not self.validate_command(command):
384            return ToolResult(
385                success=False,
386                error=f"Command not allowed: {command}"
387            )
388 
389        # Execute in isolated environment
390        result = subprocess.run(
391            command,
392            shell=True,
393            cwd=self.workspace,
394            capture_output=True,
395            timeout=30,
396            env={
397                **os.environ,
398                "HOME": self.workspace,  # Isolate home directory
399            }
400        )
401 
402        return ToolResult(
403            success=result.returncode == 0,
404            output=result.stdout.decode(),
405            error=result.stderr.decode() if result.returncode != 0 else None
406        )
407```
408 
409---
410 
411## 4. Browser Automation
412 
413### 4.1 Browser Tool Pattern
414 
415```python
416class BrowserTool:
417    """
418    Browser automation for agents using Playwright/Puppeteer.
419    Enables visual debugging and web testing.
420    """
421 
422    def __init__(self, headless: bool = True):
423        self.browser = None
424        self.page = None
425        self.headless = headless
426 
427    async def open_url(self, url: str) -> ToolResult:
428        """Navigate to URL and return page info"""
429        if not self.browser:
430            self.browser = await playwright.chromium.launch(headless=self.headless)
431            self.page = await self.browser.new_page()
432 
433        await self.page.goto(url)
434 
435        # Capture state
436        screenshot = await self.page.screenshot(type='png')
437        title = await self.page.title()
438 
439        return ToolResult(
440            success=True,
441            output=f"Loaded: {title}",
442            metadata={
443                "screenshot": base64.b64encode(screenshot).decode(),
444                "url": self.page.url
445            }
446        )
447 
448    async def click(self, selector: str) -> ToolResult:
449        """Click on an element"""
450        try:
451            await self.page.click(selector, timeout=5000)
452            await self.page.wait_for_load_state("networkidle")
453 
454            screenshot = await self.page.screenshot()
455            return ToolResult(
456                success=True,
457                output=f"Clicked: {selector}",
458                metadata={"screenshot": base64.b64encode(screenshot).decode()}
459            )
460        except TimeoutError:
461            return ToolResult(
462                success=False,
463                error=f"Element not found: {selector}"
464            )
465 
466    async def type_text(self, selector: str, text: str) -> ToolResult:
467        """Type text into an input"""
468        await self.page.fill(selector, text)
469        return ToolResult(success=True, output=f"Typed into {selector}")
470 
471    async def get_page_content(self) -> ToolResult:
472        """Get accessible text content of the page"""
473        content = await self.page.evaluate("""
474            () => {
475                // Get visible text
476                const walker = document.createTreeWalker(
477                    document.body,
478                    NodeFilter.SHOW_TEXT,
479                    null,
480                    false
481                );
482 
483                let text = '';
484                while (walker.nextNode()) {
485                    const node = walker.currentNode;
486                    if (node.textContent.trim()) {
487                        text += node.textContent.trim() + '\\n';
488                    }
489                }
490                return text;
491            }
492        """)
493        return ToolResult(success=True, output=content)
494```
495 
496### 4.2 Visual Agent Pattern
497 
498```python
499class VisualAgent:
500    """
501    Agent that uses screenshots to understand web pages.
502    Can identify elements visually without selectors.
503    """
504 
505    def __init__(self, llm, browser):
506        self.llm = llm
507        self.browser = browser
508 
509    async def describe_page(self) -> str:
510        """Use vision model to describe current page"""
511        screenshot = await self.browser.screenshot()
512 
513        response = self.llm.chat([
514            {
515                "role": "user",
516                "content": [
517                    {"type": "text", "text": "Describe this webpage. List all interactive elements you see."},
518                    {"type": "image", "data": screenshot}
519                ]
520            }
521        ])
522 
523        return response.content
524 
525    async def find_and_click(self, description: str) -> ToolResult:
526        """Find element by visual description and click it"""
527        screenshot = await self.browser.screenshot()
528 
529        # Ask vision model to find element
530        response = self.llm.chat([
531            {
532                "role": "user",
533                "content": [
534                    {
535                        "type": "text",
536                        "text": f"""
537                        Find the element matching: "{description}"
538                        Return the approximate coordinates as JSON: {{"x": number, "y": number}}
539                        """
540                    },
541                    {"type": "image", "data": screenshot}
542                ]
543            }
544        ])
545 
546        coords = json.loads(response.content)
547        await self.browser.page.mouse.click(coords["x"], coords["y"])
548 
549        return ToolResult(success=True, output=f"Clicked at ({coords['x']}, {coords['y']})")
550```
551 
552---
553 
554## 5. Context Management
555 
556### 5.1 Context Injection Patterns
557 
558````python
559class ContextManager:
560    """
561    Manage context provided to the agent.
562    Inspired by Cline's @-mention patterns.
563    """
564 
565    def __init__(self, workspace: str):
566        self.workspace = workspace
567        self.context = []
568 
569    def add_file(self, path: str) -> None:
570        """@file - Add file contents to context"""
571        with open(path, 'r') as f:
572            content = f.read()
573 
574        self.context.append({
575            "type": "file",
576            "path": path,
577            "content": content
578        })
579 
580    def add_folder(self, path: str, max_files: int = 20) -> None:
581        """@folder - Add all files in folder"""
582        for root, dirs, files in os.walk(path):
583            for file in files[:max_files]:
584                file_path = os.path.join(root, file)
585                self.add_file(file_path)
586 
587    def add_url(self, url: str) -> None:
588        """@url - Fetch and add URL content"""
589        response = requests.get(url)
590        content = html_to_markdown(response.text)
591 
592        self.context.append({
593            "type": "url",
594            "url": url,
595            "content": content
596        })
597 
598    def add_problems(self, diagnostics: list) -> None:
599        """@problems - Add IDE diagnostics"""
600        self.context.append({
601            "type": "diagnostics",
602            "problems": diagnostics
603        })
604 
605    def format_for_prompt(self) -> str:
606        """Format all context for LLM prompt"""
607        parts = []
608        for item in self.context:
609            if item["type"] == "file":
610                parts.append(f"## File: {item['path']}\n```\n{item['content']}\n```")
611            elif item["type"] == "url":
612                parts.append(f"## URL: {item['url']}\n{item['content']}")
613            elif item["type"] == "diagnostics":
614                parts.append(f"## Problems:\n{json.dumps(item['problems'], indent=2)}")
615 
616        return "\n\n".join(parts)
617````
618 
619### 5.2 Checkpoint/Resume
620 
621```python
622class CheckpointManager:
623    """
624    Save and restore agent state for long-running tasks.
625    """
626 
627    def __init__(self, storage_dir: str):
628        self.storage_dir = storage_dir
629        os.makedirs(storage_dir, exist_ok=True)
630 
631    def save_checkpoint(self, session_id: str, state: dict) -> str:
632        """Save current agent state"""
633        checkpoint = {
634            "timestamp": datetime.now().isoformat(),
635            "session_id": session_id,
636            "history": state["history"],
637            "context": state["context"],
638            "workspace_state": self._capture_workspace(state["workspace"]),
639            "metadata": state.get("metadata", {})
640        }
641 
642        path = os.path.join(self.storage_dir, f"{session_id}.json")
643        with open(path, 'w') as f:
644            json.dump(checkpoint, f, indent=2)
645 
646        return path
647 
648    def restore_checkpoint(self, checkpoint_path: str) -> dict:
649        """Restore agent state from checkpoint"""
650        with open(checkpoint_path, 'r') as f:
651            checkpoint = json.load(f)
652 
653        return {
654            "history": checkpoint["history"],
655            "context": checkpoint["context"],
656            "workspace": self._restore_workspace(checkpoint["workspace_state"]),
657            "metadata": checkpoint["metadata"]
658        }
659 
660    def _capture_workspace(self, workspace: str) -> dict:
661        """Capture relevant workspace state"""
662        # Git status, file hashes, etc.
663        return {
664            "git_ref": subprocess.getoutput(f"cd {workspace} && git rev-parse HEAD"),
665            "git_dirty": subprocess.getoutput(f"cd {workspace} && git status --porcelain")
666        }
667```
668 
669---
670 
671## 6. MCP (Model Context Protocol) Integration
672 
673### 6.1 MCP Server Pattern
674 
675```python
676from mcp import Server, Tool
677 
678class MCPAgent:
679    """
680    Agent that can dynamically discover and use MCP tools.
681    'Add a tool that...' pattern from Cline.
682    """
683 
684    def __init__(self, llm):
685        self.llm = llm
686        self.mcp_servers = {}
687        self.available_tools = {}
688 
689    def connect_server(self, name: str, config: dict) -> None:
690        """Connect to an MCP server"""
691        server = Server(config)
692        self.mcp_servers[name] = server
693 
694        # Discover tools
695        tools = server.list_tools()
696        for tool in tools:
697            self.available_tools[tool.name] = {
698                "server": name,
699                "schema": tool.schema
700            }
701 
702    async def create_tool(self, description: str) -> str:
703        """
704        Create a new MCP server based on user description.
705        'Add a tool that fetches Jira tickets'
706        """
707        # Generate MCP server code
708        code = self.llm.generate(f"""
709        Create a Python MCP server with a tool that does:
710        {description}
711 
712        Use the FastMCP framework. Include proper error handling.
713        Return only the Python code.
714        """)
715 
716        # Save and install
717        server_name = self._extract_name(description)
718        path = f"./mcp_servers/{server_name}/server.py"
719 
720        with open(path, 'w') as f:
721            f.write(code)
722 
723        # Hot-reload
724        self.connect_server(server_name, {"path": path})
725 
726        return f"Created tool: {server_name}"
727```
728 
729---
730 
731## Best Practices Checklist
732 
733### Agent Design
734 
735- [ ] Clear task decomposition
736- [ ] Appropriate tool granularity
737- [ ] Error handling at each step
738- [ ] Progress visibility to user
739 
740### Safety
741 
742- [ ] Permission system implemented
743- [ ] Dangerous operations blocked
744- [ ] Sandbox for untrusted code
745- [ ] Audit logging enabled
746 
747### UX
748 
749- [ ] Approval UI is clear
750- [ ] Progress updates provided
751- [ ] Undo/rollback available
752- [ ] Explanation of actions
753 
754---
755 
756## Resources
757 
758- [Cline](https://github.com/cline/cline)
759- [OpenAI Codex](https://github.com/openai/codex)
760- [Model Context Protocol](https://modelcontextprotocol.io/)
761- [Anthropic Tool Use](https://docs.anthropic.com/claude/docs/tool-use)
762