AI Distiller (aid)

1# AI Distiller (`aid`)
2 
3> **Note:** This is the very first version of this tool. We would be very grateful for any feedback in the form of a discussion or by creating an issue on [GitHub](https://github.com/janreges/ai-distiller/issues). Thank you\!
4 
5🚀 **MCP Server Available**: Install the Model Context Protocol server for AI Distiller from NPM: [`@janreges/ai-distiller-mcp`](https://www.npmjs.com/package/@janreges/ai-distiller-mcp) - seamlessly integrate with Claude, Cursor, and other MCP-compatible AI tools!
6 
7<p align="center">
8  <img src="docs/assets/aid-mascot-300.png" alt="AI Distiller (aid) Mascot" width="200">
9</p>
10 
11<p align="center">
12  <img src="https://img.shields.io/badge/Languages-12+-blue" alt="12+ Languages">
13  <img src="https://img.shields.io/badge/Performance-5k+_files/sec-green" alt="Performance">
14  <img src="https://img.shields.io/badge/Compression-90%25+-orange" alt="Compression">
15  <img src="https://img.shields.io/badge/Tests-1211_passed-purple" alt="Tests">
16</p
17 
18 
19## **🤔 Why AI Distiller?**
20 
21Do you work with large-scale projects that have thousands of files and functions? Do you struggle with AI tools like Claude Code, Gemini, Copilot, or Cursor frequently "hallucinating" and generating code that looks correct at first glance but is actually incompatible with your project?
22 
23**The problem is context.** AI models have a limited context window and cannot comprehend your entire codebase. Instead, AI agents search files, "grep" for keywords, look at a few lines before and after the found term, and try (often, but not always) to guess the interface of your classes and functions. The result? Code full of errors that guesses parameters, returns incorrect data types, and ignores the existing architecture. If you are a sophisticated user of AI agents (vibe coder), you know that you can help yourself by instructing the AI ​​agent to consistently write and run tests, using static code analysis, pre-commit hooks, etc. - the AI ​​agent will usually fix the code itself, but in the meantime it will take 20 steps and 5 minutes. On the other hand, it must be admitted that if you pay for each AI request (and large context is an expensive factor) and are not "playing for time", you may not mind this limited context approach.
24 
25**AI Distiller (or `aid` for short) helps solve this problem.** Its main function is code "distillation" – a process where it extracts only the most essential information from the entire project (ideally from the main source folder, or a specific module subdirectory for extremely large projects) that the AI needs to write code correctly on the first try. This distillation usually generates a context that is only 5-20% of the original source code volume, allowing AI tools to include it in their context. As a result, the AI uses the existing code exactly as it was designed, not by trial and error.
26 
27Very simply, it can be said that `aid`, within the distillation process, will leave only the public parts of the interface, input and output data types, but in the default state it will discard method implementations and non-public structures. But everything is configurable via [CLI Options](#-complete-cli-reference).
28 
29## Table of Contents
30 
31- [🤔 Why AI Distiller?](#-why-ai-distiller)
32- [✨ Key Features](#-key-features)
33- [🎯 How It Works](#-how-it-works)
34- [🔗 Dependency-Aware Distillation](#-dependency-aware-distillation)
35- [🚀 Quick Start](#-quick-start)
36  - [One-Line Installation](#one-line-installation)
37  - [Basic Usage](#basic-usage)
38- [📖 Example Output](#-example-output)
39- [📖 Guides & Examples](#-guides--examples)
40  - [Deep Code Analysis Prompt Generation](#deep-code-analysis-prompt-generation)
41  - [🤖 Use with Claude Desktop (MCP)](#-use-with-claude-desktop-mcp)
42- [📖 Complete CLI Reference](#-complete-cli-reference)
43- [🛠️ Advanced Usage](#-advanced-usage)
44  - [🚫 Ignoring Files with .aidignore](#-ignoring-files-with-aidignore)
45  - [🎯 Git History Analysis Mode](#-git-history-analysis-mode)
46- [⚠️ Limitations](#-limitations)
47- [🔒 Security Considerations](#-security-considerations)
48- [❓ FAQ](#-faq)
49- [🤝 Contributing](#-contributing)
50- [📄 License](#-license)
51- [🙏 Acknowledgments](#-acknowledgments)
52 
53## **✨ Key Features**
54 
55| Feature | Description | 
56| ------- | ----------- |
57| 🚀 Extreme Speed | Processes tens of megabytes of code in hundreds of milliseconds. By default, it uses 80% of available CPU cores, but can be configured, e.g., with `--workers=1` to use only a single CPU core. |  
58| 🧠 Intelligent Distillation | Understands 12+ programming languages and extracts only public APIs (methods, properties, types). |  
59| ⚙️ High Configurability | Allows including private, protected, and internal members, implementation, or comments. |  
60| 🤖 AI Prompt Generation | Generates ready-to-use prompts with distilled code for AI analysis. The tool creates files with prompts that AI agents can then execute for security audits, refactoring, etc. See `--ai-action` switch. |  
61| 📋 Analysis Automation | Creates a complete checklist and directory structure for AI agents, who can then systematically analyze the entire project. See the flow-for-\* actions for the `--ai-action` switch. |  
62| 📜 Git Analysis | Processes commit history and prepares data for in-depth analysis of development quality and team dynamics. |  
63| 💻 Multi-platform | A single binary file with no dependencies for Windows, Linux, and macOS (x64 & ARM). |  
64| 🔌 Integration via MCP | Can be integrated into tools like Claude Code, VS Code, Cursor, Windsurf and others thanks to the included MCP server. |
65 
66### 🎯 Intelligent Filtering
67 
68Control exactly what to include with our new granular flag system:
69 
70**Visibility Control**:
71- `--public=1` (default) - Include public members
72- `--protected=0` (default) - Exclude protected members
73- `--internal=0` (default) - Exclude internal/package-private
74- `--private=0` (default) - Exclude private members
75 
76**Content Control**:
77- `--comments=0` (default) - Exclude comments
78- `--docstrings=1` (default) - Include documentation
79- `--implementation=0` (default) - Exclude function/methods bodies
80- `--imports=1` (default) - Include import/use statements
81 
82**Default behavior**: Shows only public API signatures with basic documentation - perfect for AI understanding while maintaining maximum compression.
83 
84### 🤖 AI-Powered Analysis Prompt Generation
85 
86AI Distiller generates specialized prompts combined with distilled code for AI-driven analysis:
87 
88- **`--ai-action=flow-for-deep-file-to-file-analysis`** - Generates task lists and prompts for systematic file-by-file analysis
89- **`--ai-action=flow-for-multi-file-docs`** - Creates documentation workflow prompts with code structure
90- **Output to files** - Prompts are saved to `.aid/` directory (or use `--stdout` for small codebases)
91- **Ready for AI execution** - Generated files contain both the analysis prompt and distilled code
92- **AI agent instructions** - Output includes guidance for AI agents to read and process the generated files
93- **Gemini advantage** - 1M token context window perfect for larger codebase analysis
94 
95**Note**: AI Distiller doesn't perform the analysis itself - it prepares optimized prompts that AI agents (Claude, Gemini, ChatGPT) then execute. Users often need to explicitly ask their AI agent to process the generated file or copy its contents to web-based AI tools.
96 
97### 📝 Multiple Output Formats
98- **Text** (`--format text`) - Ultra-compact for AI consumption (default)
99- **Markdown** (`--format md`) - Clean, structured Markdown
100- **JSON Structured** (`--format json-structured`) - Rich semantic data for tools
101- **JSONL** (`--format jsonl`) - Streaming format
102- **XML** (`--format xml`) - Legacy system compatible
103 
104### 📊 Smart Summary Output
105 
106After each distillation, AI Distiller displays a summary showing compression efficiency and processing speed:
107 
108```bash
109# Default: Visual progress bar for interactive terminals (green dots = saved, red dots = remaining)
110✨ Distilled 970 files [░░░░░░░░░░░░░░░] 98% (10M → 256K) in 231ms 💰 ~2.4M tokens saved (~64k remaining)
111 
112# Choose your preferred format with --summary-type
113aid ./src --summary-type=stock-ticker
114📊 AID 97.6% ▲ │ SIZE: 10M→256K │ TIME: 231ms │ EST: ~2.4M tokens saved
115 
116# JSON output
117aid ./src --summary-type=json
118 
119{
120  "original_bytes": 70020,
121  "distilled_bytes": 8244,
122  "savings_pct": 88.22622107969151,
123  "duration_ms": 6,
124  "tokens_before": 17505,
125  "tokens_after": 2061,
126  "tokens_saved": 15444,
127  "token_savings_pct": 88.22622107969151,
128  "file_count": 9,
129  "output_path": "/home/user/project/.aid/aid.processor.txt",
130  "tokenizer": "cl100k_base"
131}
132```
133 
134**Available formats:**
135- `visual-progress-bar` (default) - Shows compression as a progress bar
136- `stock-ticker` - Compact stock market style display
137- `speedometer-dashboard` - Multi-line dashboard with metrics
138- `minimalist-sparkline` - Single line with all essential info
139- `ci-friendly` - Clean format for CI/CD pipelines
140- `json` - Machine-readable JSON output
141- `off` - Disable summary output
142 
143Use `--no-emoji` to remove emojis from any format.
144 
145### 📁 Smart Project Root Detection
146 
147AI Distiller automatically detects your project root and centralizes all outputs in a `.aid/` directory:
148 
149- **Automatic detection**: Searches upward for `.aidrc`, `go.mod`, `package.json`, `.git`, etc.
150- **Consistent location**: All outputs go to `<project-root>/.aid/` regardless of where you run `aid`
151- **Cache management**: MCP cache stored in `.aid/cache/` for better organization
152- **Easy cleanup**: Add `.aid/` to `.gitignore` to keep outputs out of version control
153 
154**Detection priority:**
1551. **`.aidrc` file** - Create this empty file to explicitly mark your project root
1562. **Language markers** - `go.mod`, `package.json`, `pyproject.toml`, etc.
1573. **Version control** - `.git` directory
1584. **Environment variable** - `AID_PROJECT_ROOT` (fallback if no markers found)
1595. **Current directory** - Final fallback with warning
160 
161```bash
162# Mark a specific directory as project root (recommended)
163touch /my/project/.aidrc
164 
165# Run from anywhere in your project - outputs always go to project root
166cd deep/nested/directory
167aid ../../../src  # Output: <project-root>/.aid/aid.src.txt
168 
169# Use environment variable as fallback (useful for CI/CD)
170AID_PROJECT_ROOT=/build/workspace aid src/
171```
172 
173### 🌍 Language Support
174Currently supports 12 languages via tree-sitter:
175- **Full Support**: Python, Go, JavaScript, PHP, Ruby
176- **Beta**: TypeScript, Java, C#, Rust, Kotlin, Swift, C++
177- **Coming Soon**: Zig, Scala, Clojure
178 
179#### Language-Specific Documentation:
180- [C++](docs/lang/cpp.md) - C++11/14/17/20 support with templates, namespaces, modern features
181- [C#](docs/lang/csharp.md) - Complete C# 12 support with records, nullable reference types, pattern matching
182- [Go](docs/lang/go.md) - Full Go support with interfaces, goroutines, generics (1.18+)
183- [Java](docs/lang/java.md) - Java 8-21 support with records, sealed classes, pattern matching
184- [JavaScript](docs/lang/javascript.md) - ES6+ support with classes, modules, async/await
185- [Kotlin](docs/lang/kotlin.md) - Kotlin 1.x support with coroutines, data classes, sealed classes
186- [PHP](docs/lang/php.md) - PHP 7.4+ with PHP 8.x features (attributes, union types, enums)
187- [Python](docs/lang/python.md) - Full Python 3.x support with type hints, async/await, decorators
188- [Ruby](docs/lang/ruby.md) - Ruby 2.x/3.x support with blocks, modules, metaprogramming
189- [Rust](docs/lang/rust.md) - Rust 2018/2021 editions with traits, lifetimes, async
190- [Swift](docs/lang/swift.md) - Swift 5.x support with protocols, extensions, property wrappers
191- [TypeScript](docs/lang/typescript.md) - TypeScript 4.x/5.x with generics, decorators, type system
192 
193## 🎯 How It Works
194 
1951. **Scans** your codebase recursively for supported file types (10+ languages)
1962. **Parses** each file using language-specific tree-sitter parsers (all bundled, no dependencies)
1973. **Extracts** only what you need: public APIs, type signatures, class hierarchies
1984. **Outputs** in your preferred format: compact text, markdown, or structured JSON
199 
200All tree-sitter grammars are compiled into the `aid` binary - zero external dependencies!
201 
202## 🔗 Dependency-Aware Distillation
203 
204**Advanced Feature**: AI Distiller includes dependency-aware distillation that analyzes call graphs across files and includes only the code that is actually used from your codebase. This creates focused distillations for deep code analysis by following function/method calls across multiple files.
205 
206> 💡 **New to dependency analysis?** This feature traces which functions actually call each other in your code, creating a minimal context that includes only the relevant parts. Perfect for AI tools that need to understand code relationships without processing entire files.
207 
208### 🎯 How Dependency-Aware Analysis Works
209 
210Instead of including entire files, dependency-aware distillation:
2111. **Identifies entry points** (main functions, exported APIs)
2122. **Traces function calls** across file boundaries
2133. **Builds call graphs** to understand dependencies
2144. **Includes only used code** - functions that are actually called
2155. **Filters out unused code** - dead code elimination for AI context
216 
217```bash
218# Basic dependency analysis
219aid main.py --dependency-aware
220 
221# Control analysis depth
222aid main.py --dependency-aware --max-depth=2
223 
224# Include implementations for deeper analysis
225aid main.py --dependency-aware --implementation=1 --max-depth=3
226```
227 
228### 📊 Language Support Quality
229 
230We've worked extensively to make dependency-aware distillation as reliable as possible across different programming languages. However, the complexity varies significantly between languages, and we want to be transparent about the current state:
231 
232| Language | Support Level | Cross-File Analysis | Intra-File Calls | Performance | Notes |
233|----------|---------------|--------------------|--------------------|-------------|-------|
234| **Python** | 🟢 **Very Good** | ✅ Full | ✅ Complete | ~37ms | Package imports, all call patterns |
235| **JavaScript** | 🟢 **Very Good** | ✅ Full | ✅ Complete | ~38ms | CommonJS & ES6 modules |
236| **Go** | 🟢 **Very Good** | ✅ Full | ✅ Complete | ~37ms | Package system integration |
237| **Rust** | 🟢 **Very Good** | ✅ Full | ✅ Complete | ~36ms | Crate system, proper filtering |
238| **Java** | 🟢 **Very Good** | ✅ Full | ✅ Complete | ~41ms | Package imports, static methods |
239| **Swift** | 🟢 **Very Good** | ✅ Full | ✅ Complete | ~37ms | Class and static method detection |
240| **PHP** | 🟢 **Very Good** | ✅ Full | ✅ Complete | ~37ms | Include/require resolution |
241| **Ruby** | 🟢 **Very Good** | ✅ Full | ✅ Complete | ~40ms | Module system, all call patterns |
242| **TypeScript** | 🟡 **Limited** | ❌ Issues | ❌ Issues | N/A | Language processor limitations |
243| **C#** | 🟡 **Limited** | ❌ Issues | ❌ Issues | N/A | Language processor limitations |
244| **C++** | 🟡 **Limited** | ❌ Issues | ❌ Issues | N/A | Language processor limitations |
245| **Kotlin** | 🟠 **Good** | ✅ Partial | ⚠️ Basic | ~45ms | Companion objects, some edge cases |
246 
247**Legend:**
248- 🟢 **Very Good**: Production-ready, handles complex scenarios reliably
249- 🟠 **Good**: Solid functionality with minor limitations
250- 🟡 **Limited**: Basic functionality, parsing capabilities may be limited
251- ✅ **Full**: Complete cross-file dependency tracing
252- ⚠️ **Basic**: Simple scenarios work well, complex patterns may be limited
253- ❌ **Issues**: Functionality is significantly limited
254 
255### 🚀 Performance Characteristics
256 
257**Very Good Performance** (8 languages):
258- **Processing Time**: 36-41ms consistently
259- **Compression**: 65-72% size reduction
260- **Scalability**: Handles projects up to 10 files efficiently
261- **Memory Usage**: Minimal, no hanging or timeout issues
262 
263**Areas for Enhancement**:
264- **Large Projects**: Performance may be limited with 50+ files
265- **Language Processors**: C#, C++, TypeScript have fundamental limitations  
266- **Complex Call Patterns**: Advanced metaprogramming patterns may be limited
267 
268### 💡 When to Use Dependency-Aware Analysis
269 
270**Perfect for:**
271- 🎯 **Impact Analysis** - Understand what code is affected by changes
272- 🔍 **Code Navigation** - Follow execution flows across multiple files
273- 🎪 **Focused Context** - Get only relevant code for AI assistants
274- 📚 **Legacy Understanding** - Trace through complex codebases systematically
275- 🔧 **API Analysis** - See which methods are actually called vs. just defined
276 
277**Best Practices:**
278```bash
279# Start with small depth for quick overview
280aid main.py --dependency-aware --max-depth=1
281 
282# Increase depth for comprehensive analysis
283aid main.py --dependency-aware --max-depth=2 --implementation=1
284 
285# Use with specific languages known to work well
286aid src/ --dependency-aware --include="*.py,*.js,*.go"
287```
288 
289### 🚀 Transform Massive Codebases Into AI-Friendly Context
290 
291> **The Problem**: Modern codebases contain thousands of files with millions of lines. But for AI to understand your code architecture, suggest improvements, or help with development, it doesn't need to see every implementation detail - it needs the **structure and public interfaces**.
292 
293> **The Solution**: AI Distiller extracts only what matters - public APIs, types, and signatures - reducing codebase size by 90-98% while preserving all essential information for AI comprehension.
294 
295<table>
296<tr>
297<th align="left">Project</th>
298<th align="center">Files</th>
299<th align="right">Original Tokens</th>
300<th align="right">Distilled Tokens</th>
301<th align="center">Fits in Context<sup>1</sup></th>
302<th align="center">Speed<sup>2</sup></th>
303</tr>
304<tr>
305<td>⚛️ <code>react</code></td>
306<td align="center">1,781</td>
307<td align="right">~5.5M</td>
308<td align="right"><strong>250K</strong> (-95%)</td>
309<td align="center">✅ Gemini<sup>3</sup></td>
310<td align="center">2,875 files/s</td>
311</tr>
312<tr>
313<td>🎨 <code>vscode</code></td>
314<td align="center">4,768</td>
315<td align="right">~22.5M</td>
316<td align="right"><strong>2M</strong> (-91%)</td>
317<td align="center">⚠️ Needs chunking</td>
318<td align="center">5,072 files/s</td>
319</tr>
320<tr>
321<td>🐍 <code>django</code></td>
322<td align="center">970</td>
323<td align="right">~10M</td>
324<td align="right"><strong>256K</strong> (-97%)</td>
325<td align="center">✅ Gemini<sup>3</sup></td>
326<td align="center">4,199 files/s</td>
327</tr>
328<tr>
329<td>📦 <code>prometheus</code></td>
330<td align="center">685</td>
331<td align="right">~8.5M</td>
332<td align="right"><strong>154K</strong> (-98%)</td>
333<td align="center">✅ Claude/Gemini</td>
334<td align="center">3,071 files/s</td>
335</tr>
336<tr>
337<td>🦀 <code>rust-analyzer</code></td>
338<td align="center">1,275</td>
339<td align="right">~5.5M</td>
340<td align="right"><strong>172K</strong> (-97%)</td>
341<td align="center">✅ Claude/Gemini</td>
342<td align="center">10,451 files/s</td>
343</tr>
344<tr>
345<td>🚀 <code>astro</code></td>
346<td align="center">1,058</td>
347<td align="right">~10.5M</td>
348<td align="right"><strong>149K</strong> (-99%)</td>
349<td align="center">✅ Claude/Gemini</td>
350<td align="center">5,212 files/s</td>
351</tr>
352<tr>
353<td>💎 <code>rails</code></td>
354<td align="center">394</td>
355<td align="right">~1M</td>
356<td align="right"><strong>104K</strong> (-90%)</td>
357<td align="center">✅ ChatGPT-4o</td>
358<td align="center">4,864 files/s</td>
359</tr>
360<tr>
361<td>🐘 <code>laravel</code></td>
362<td align="center">1,443</td>
363<td align="right">~3M</td>
364<td align="right"><strong>238K</strong> (-92%)</td>
365<td align="center">✅ Gemini<sup>3</sup></td>
366<td align="center">4,613 files/s</td>
367</tr>
368<tr>
369<td>⚡ <code>nestjs</code></td>
370<td align="center">802</td>
371<td align="right">~1.5M</td>
372<td align="right"><strong>107K</strong> (-93%)</td>
373<td align="center">✅ ChatGPT-4o</td>
374<td align="center">8,813 files/s</td>
375</tr>
376<tr>
377<td>👻 <code>ghost</code></td>
378<td align="center">2,184</td>
379<td align="right">~8M</td>
380<td align="right"><strong>235K</strong> (-97%)</td>
381<td align="center">✅ Gemini<sup>3</sup></td>
382<td align="center">4,719 files/s</td>
383</tr>
384</table>
385 
386<sub><sup>1</sup> Context windows: ChatGPT-4o (128K), Claude (200K), Gemini (1M). ✅ = fits completely, ⚠️ = needs splitting</sub><br>
387<sub><sup>2</sup> Processing speed with 12 parallel workers on AMD Ryzen 7945HX. Use `-w 1` for serial mode or `-w N` for custom workers.</sub><br>
388<sub><sup>3</sup> These frameworks exceed 200K tokens and work only with Gemini due to its larger 1M token context window.</sub>
389 
390### 🎯 Why This Matters for AI-Assisted Development
391 
392**Large codebases are overwhelming for AI models.** A typical web framework like Django has ~10 million tokens of source code. Even with Claude's 200K context window, you'd need to split it into 50+ chunks, losing coherence and relationships between components.
393 
394**But here's the good news**: Most real-world projects that teams have invested hundreds to thousands of hours developing are much smaller. Thanks to AI Distiller, the vast majority of typical business applications, SaaS products, and internal tools can fit entirely within AI context windows, enabling unprecedented AI assistance quality.
395 
396### ⚠️ The Hidden Problem with AI Coding Tools
397 
398**Most AI agents and IDEs are "context misers"** - they try to save tokens at the expense of actual codebase knowledge. They rely on:
399- 🔍 **Grep/search** to find relevant code snippets
400- 📄 **Limited context** showing only 10-50 lines around matches  
401- 🎲 **Guessing interfaces** based on partial information
402 
403**This is why AI-generated code often fails on first attempts** - the AI is literally guessing method signatures, parameter types, and return values because it can't see the full picture.
404 
405**AI Distiller changes the game** by giving AI complete knowledge of:
406- ✅ **Exact interfaces** of all classes, methods, and functions
407- ✅ **All parameter types** and their expected values
408- ✅ **Return types** and data structures
409- ✅ **Full inheritance hierarchies** and relationships
410 
411Instead of playing "code roulette", AI can now write correct code from the start.
412 
413**Result**: Django's 10M tokens compress to just 256K tokens - suddenly the **entire framework fits in a single AI conversation**, leading to:
414- 🎯 **More accurate suggestions** - AI sees all available APIs at once
415- 🚀 **Less hallucination** - No more inventing methods that don't exist
416- 💡 **Better architecture advice** - AI understands the full system design
417- ⚡ **Faster development** - Especially for "vibe coding" with AI assistance
418- 💰 **40x cost reduction** - Pay for 256K tokens instead of 10M on API calls
419 
420### 🔧 Flexible for Different Use Cases
421 
422```bash
423# Process entire codebase (default: public APIs only)
424aid ./my-project
425 
426# Process specific directory or module
427aid ./my-project/src/auth
428aid ./my-project/src/api
429 
430# Process a directory
431aid ./my-project/core/
432 
433# Process individual file
434aid src/main.py
435 
436# Include protected/private for deeper analysis
437aid ./my-project --private=1 --protected=1
438 
439# Include implementations for small projects
440aid ./my-small-lib --implementation=1
441 
442# Everything for complete understanding
443aid ./micro-service --private=1 --protected=1 --implementation=1
444```
445 
446**Granular Control**: Process your entire codebase, specific modules, directories, or even individual files. Perfect for focusing AI on the exact context it needs - whether that's understanding the whole system architecture or diving deep into a specific authentication module.
447 
448📈 **[Full benchmark details](benchmark/BENCHMARK_RESULTS.md)** | 🧪 **[Reproduce these results](benchmark/test_benchmark.sh)**
449 
450## 🚀 Quick Start
451 
452### One-Line Installation
453 
454**macOS / Linux / WSL:**
455```bash
456# Install to ~/.aid/bin (recommended, no sudo required)
457curl -sSL https://raw.githubusercontent.com/janreges/ai-distiller/main/install.sh | bash
458 
459# Install to /usr/local/bin (requires sudo)
460curl -sSL https://raw.githubusercontent.com/janreges/ai-distiller/main/install.sh | bash -s -- --sudo
461```
462 
463**Windows PowerShell:**
464```powershell
465iwr https://raw.githubusercontent.com/janreges/ai-distiller/main/install.ps1 -useb | iex
466```
467 
468The installer will:
469- Detect your OS and architecture automatically
470- Download the appropriate pre-built binary
471- Install to `~/.aid/bin/aid` by default (no sudo required)
472- Or to `/usr/local/bin/aid` with `--sudo` flag
473- Guide you through PATH configuration if needed
474 
475### Basic Usage
476 
477```bash
478# Basic usage
479aid .                                   # Current directory, output is saved to file in ./aid
480aid . --stdout                          # Current directory, output is printed to STDOUT
481aid src/                                # Specific directory
482aid main.py                             # Specific file
483 
484```
485 
486## 📖 Example Output
487 
488<details>
489<summary>Python Class Example</summary>
490 
491**Input** (`car.py`):
492```python
493class Car:
494    """A car with basic attributes and methods."""
495    
496    def __init__(self, make: str, model: str):
497        self.make = make
498        self.model = model
499        self._mileage = 0  # Private
500    
501    def drive(self, distance: int) -> None:
502        """Drive the car."""
503        if distance > 0:
504            self._mileage += distance
505```
506 
507**Output** (`aid car.py --format text --implementation=0`):
508```
509<file path="car.py">
510class Car:
511    +def __init__(self, make: str, model: str)
512    +def drive(self, distance: int) -> None
513</file>
514```
515 
516</details>
517 
518<details>
519<summary>TypeScript Interface Example</summary>
520 
521**Input** (`api.ts`):
522```typescript
523export interface User {
524  id: number;
525  name: string;
526  email?: string;
527}
528 
529export class UserService {
530  private cache = new Map<number, User>();
531  
532  async getUser(id: number): Promise<User | null> {
533    return this.cache.get(id) || null;
534  }
535}
536```
537 
538**Output** (`aid api.ts --format text --implementation=0`):
539```
540<file path="api.ts">
541export interface User {
542  id: number;
543  name: string;
544  email?: string;
545}
546 
547export class UserService {
548  +async getUser(id: number): Promise<User | null>
549}
550</file>
551```
552 
553</details>
554 
555## 📖 Guides & Examples
556 
557### Deep Code Analysis Prompt Generation
558 
559AI Distiller generates sophisticated analysis prompts that AI assistants can execute for comprehensive codebase understanding:
560 
561```bash
562aid internal \
563   --private=1 --protected=1 --implementation=1 \
564   --ai-action=flow-for-deep-file-to-file-analysis
565 
566✅ AI Analysis Task List generated successfully!
567📋 Task List: .aid/ANALYSIS-TASK-LIST.internal.2025-06-20.md
568📊 Summary File: .aid/ANALYSIS-SUMMARY.internal.2025-06-20.md
569📁 Analysis Reports Directory: .aid/analysis.internal/2025-06-20
570🤖 Ready for AI-driven analysis workflow!
571📂 Files to analyze: 158
572 
573💡 If you are an AI agent, please read the Task List above and carefully follow all instructions to systematically analyze each file.
574```
575 
576**What AI Distiller generates**:
577- 📋 **Task list prompt** - A structured checklist for AI to follow (`.aid/ANALYSIS-TASK-LIST.PROJECT.DATE.md`)
578- 🎯 **Analysis instructions** - Detailed prompts guiding AI through security, performance, and quality checks
579- 📊 **Code structure** - Distilled code included in the prompt files for AI to analyze
580- 📁 **Directory structure** - Pre-created folders where AI agents can save their analysis results
581 
582**How to use the generated prompts**:
5831. **For AI agents**: Direct the agent to read the generated task list file and follow instructions
5842. **For web AI tools**: Copy the content of generated files and paste into Gemini (best for large codebases due to 1M context)
5853. **For small codebases**: Use `--stdout` to get prompt directly without saving to file
586 
587**Note**: The analysis dimensions (Security, Performance, Maintainability, Readability) are part of the prompts that guide the AI - AI Distiller itself doesn't perform any analysis.
588 
589### 🤖 Use with Claude Code/Desktop (MCP)
590 
591AI Distiller now integrates seamlessly with Claude Code/Desktop through the Model Context Protocol (MCP), enabling AI agents to analyze and understand codebases directly within conversations.
592 
593```bash
594# One-line installation
595claude mcp add aid -- npx -y @janreges/ai-distiller-mcp
596```
597 
598📦 **NPM Package**: [`@janreges/ai-distiller-mcp`](https://www.npmjs.com/package/@janreges/ai-distiller-mcp) - Full documentation and examples available
599 
600#### Available MCP Tools
601 
602**🔍 Code Structure Tools:**
603- `distill_file` - Extract structure from a single file
604- `distill_directory` - Extract structure from entire directories
605- `list_files` - Browse directories with file statistics
606- `get_capabilities` - Get info about AI Distiller capabilities
607 
608**🎯 Specialized AI Analysis Tools:**
609- `aid_hunt_bugs` - Generate bug-hunting prompts with distilled code
610- `aid_suggest_refactoring` - Create refactoring analysis prompts
611- `aid_generate_diagram` - Produce diagram generation prompts (Mermaid)
612- `aid_analyze_security` - Generate security audit prompts (OWASP Top 10)
613- `aid_generate_docs` - Create documentation generation prompts
614- `aid_deep_file_analysis` - Systematic file-by-file analysis workflow
615- `aid_multi_file_docs` - Multi-file documentation workflow
616- `aid_complex_analysis` - Enterprise-grade analysis prompts
617- `aid_performance_analysis` - Performance optimization prompts
618- `aid_best_practices` - Code quality and best practices prompts
619 
620**🔧 Core Analysis Engine:**
621- `aid_analyze` - Direct access to all AI actions for custom workflows
622 
623**Important**: AI Distiller **generates analysis prompts** with distilled code - it does NOT perform the actual analysis! The output is a specialized prompt + distilled code that AI agents (like Claude) then execute. For large codebases, you can copy the output to tools like Gemini 2.0 with 1M context window.
624 
625**Smart Context Management**: AI agents can analyze your entire project for understanding the big picture, then zoom into specific modules (auth, API, database) for detailed work. No more overwhelming AI with irrelevant code!
626 
627## 📖 Complete CLI Reference
628 
629### Command Synopsis
630```bash
631aid <path> [OPTIONS]
632```
633 
634### Core Arguments and Options
635 
636#### 🎯 Primary Arguments
637 
638| Argument | Type | Default | Description |
639|----------|------|---------|-------------|
640| `<path>` | String | *(required)* | Path to source file or directory to analyze. Use `.git` for git history mode, `-` (or empty) for stdin input |
641 
642#### 📁 Output Options
643 
644| Option | Type | Default | Description |
645|--------|------|---------|-------------|
646| `-o, --output` | String | `.aid/<dirname>.[options].txt` | Output file path. Auto-generated based on input directory basename and options if not specified |
647| `--stdout` | Flag | `false` | Print output to stdout in addition to file. When used alone, no file is created |
648| `--format` | String | `text` | Output format: `text` (ultra-compact), `md` (clean Markdown), `jsonl` (one JSON per file), `json-structured` (rich semantic data), `xml` (structured XML) |
649 
650#### 🤖 AI Actions
651 
652| Option | Type | Default | Description |
653|--------|------|---------|-------------|
654| `--ai-action` | String | *(none)* | Generate pre-configured prompts with distilled code for AI analysis. See [AI Actions](#ai-actions-detailed) section below |
655| `--ai-output` | String | `./.aid/<action>.<timestamp>.<dirname>.md` | Custom output path for generated AI prompt files |
656 
657#### 👁️ Visibility Filtering
658 
659| Option | Type | Default | Description |
660|--------|------|---------|-------------|
661| `--public` | 0\|1 | `1` | Include public members (methods, functions, classes) |
662| `--protected` | 0\|1 | `0` | Include protected members |
663| `--internal` | 0\|1 | `0` | Include internal/package-private members |
664| `--private` | 0\|1 | `0` | Include private members |
665 
666#### 📝 Content Filtering
667 
668| Option | Type | Default | Description |
669|--------|------|---------|-------------|
670| `--comments` | 0\|1 | `0` | Include inline and block comments |
671| `--docstrings` | 0\|1 | `1` | Include documentation comments (docstrings, JSDoc, etc.) |
672| `--implementation` | 0\|1 | `0` | Include function/method bodies (implementation details) |
673| `--imports` | 0\|1 | `1` | Include import/require statements |
674| `--annotations` | 0\|1 | `1` | Include decorators and annotations |
675| `--fields` | 0\|1 | `1` | Include class fields and properties |
676| `--methods` | 0\|1 | `1` | Include methods and functions |
677 
678#### 🎛️ Alternative Filtering Syntax
679 
680| Option | Type | Default | Description |
681|--------|------|---------|-------------|
682| `--include-only` | String | *(none)* | Include ONLY these categories (comma-separated: `public,protected,imports`) |
683| `--exclude-items` | String | *(none)* | Exclude these categories (comma-separated: `private,comments,implementation`) |
684 
685#### 📂 File Selection
686 
687| Option | Type | Default | Description |
688|--------|------|---------|-------------|
689| `--include` | String | *(all files)* | Include file patterns (comma-separated: `*.go,*.py` or multiple: `--include "*.go" --include "*.py"`) |
690| `--exclude` | String | *(none)* | Exclude file patterns (comma-separated: `*test*,*.json` or multiple: `--exclude "*test*" --exclude "vendor/**"`) |
691| `-r, --recursive` | 0\|1 | `1` | Process directories recursively. Set to 0 to process only immediate directory contents |
692 
693#### 🔧 Processing Options
694 
695| Option | Type | Default | Description |
696|--------|------|---------|-------------|
697| `--raw` | Flag | `false` | Process all text files without language parsing. Overrides all content filters |
698| `--lang` | String | `auto` | Force language detection: `auto`, `python`, `typescript`, `javascript`, `go`, `rust`, `java`, `csharp`, `kotlin`, `cpp`, `php`, `ruby`, `swift` |
699 
700#### 📍 Path Control
701 
702| Option | Type | Default | Description |
703|--------|------|---------|-------------|
704| `--file-path-type` | String | `relative` | Path format in output: `relative` or `absolute` |
705| `--relative-path-prefix` | String | *(empty)* | Custom prefix for relative paths (e.g., `module/` → `module/src/file.go`) |
706 
707#### ⚡ Performance Options
708 
709| Option | Type | Default | Description |
710|--------|------|---------|-------------|
711| `-w, --workers` | Integer | `0` | Number of parallel workers. `0` = auto (80% of CPU cores), `1` = serial processing, `2+` = specific worker count |
712 
713#### 📊 Summary Output Options
714 
715| Option | Type | Default | Description |
716|--------|------|---------|-------------|
717| `--summary-type` | String | `visual-progress-bar` | Summary format after processing. See [Summary Types](#summary-types) below |
718| `--no-emoji` | Flag | `false` | Disable emojis in summary output for plain text terminals |
719 
720#### 📜 Git Mode Options (when path is `.git`)
721 
722| Option | Type | Default | Description |
723|--------|------|---------|-------------|
724| `--git-limit` | Integer | `200` | Number of commits to analyze. Use `0` for all commits |
725| `--with-analysis-prompt` | Flag | `false` | Add comprehensive AI analysis prompt for commit quality, patterns, and insights |
726 
727#### 🐛 Diagnostic Options
728 
729| Option | Type | Default | Description |
730|--------|------|---------|-------------|
731| `-v, --verbose` | Count | `0` | Verbose output. Use `-vv` for detailed info, `-vvv` for full trace with data dumps |
732| `--version` | Flag | `false` | Show version information and exit |
733| `--help` | Flag | `false` | Show help message |
734| `--help-extended` | Flag | `false` | Show complete documentation (man page style) |
735| `--cheat` | Flag | `false` | Show quick reference card |
736 
737### AI Actions Detailed
738 
739AI actions generate pre-configured prompts combined with distilled code that AI agents can then execute for specific analysis tasks:
740 
741| Action | Generated Prompt Type | AI Agent Will |
742|--------|-------------|----------|
743| `prompt-for-refactoring-suggestion` | Refactoring analysis prompt with distilled code | Analyze code for improvements, technical debt, effort sizing |
744| `prompt-for-complex-codebase-analysis` | Enterprise-grade analysis prompt with full codebase | Generate architecture diagrams, compliance checks, findings |
745| `prompt-for-security-analysis` | Security audit prompt with OWASP Top 10 guidelines | Detect vulnerabilities, suggest remediation steps |
746| `prompt-for-performance-analysis` | Performance optimization prompt with complexity focus | Identify bottlenecks, analyze scalability issues |
747| `prompt-for-best-practices-analysis` | Code quality prompt with industry standards | Assess code quality, suggest improvements |
748| `prompt-for-bug-hunting` | Bug detection prompt with pattern analysis | Find bugs, analyze quality metrics |
749| `prompt-for-single-file-docs` | Documentation generation prompt for single file | Create comprehensive API documentation |
750| `prompt-for-diagrams` | Diagram generation prompt with Mermaid syntax | Generate 10+ architecture and process diagrams |
751| `flow-for-deep-file-to-file-analysis` | Systematic analysis task list with directory structure | Perform file-by-file deep analysis |
752| `flow-for-multi-file-docs` | Documentation workflow with file relationships | Create interconnected documentation |
753 
754### Summary Types
755 
756| Type | Description | Example Output |
757|------|-------------|----------------|
758| `visual-progress-bar` | Default. Shows compression progress bar with colors | `✅ Distilled 150 files [████████░░] 85% (5MB → 750KB)` |
759| `stock-ticker` | Compact stock market style | `📊 AID 97.5% ▲ \| 5MB→128KB \| ~1.2M tokens saved` |
760| `speedometer-dashboard` | Multi-line dashboard with detailed metrics | Shows files, size, tokens, processing time in box format |
761| `minimalist-sparkline` | Single line with sparkline visualization | `▁▃▅▇█ 150 files → 97.5% reduction (750KB) ✓` |
762| `ci-friendly` | Clean format for CI/CD pipelines | `[aid] ✓ 85.9% saved \| 21 kB → 2.9 kB \| 4ms` |
763| `json` | Machine-readable JSON output | `{"original_bytes":5242880,"distilled_bytes":131072,...}` |
764| `off` | Disable summary output | No summary displayed |
765 
766### Exit Codes
767 
768| Code | Meaning |
769|------|---------|
770| `0` | Success |
771| `1` | General error (file not found, parse error, etc.) |
772| `2` | Invalid arguments or conflicting options |
773 
774### Examples
775 
776```bash
777# Basic usage - distill with default settings (public APIs only)
778aid ./src
779 
780# Include all visibility levels and implementation
781aid ./src --private=1 --protected=1 --internal=1 --implementation=1
782 
783# Generate security analysis prompt (AI agent will execute the analysis)
784aid --ai-action prompt-for-security-analysis ./api --private=1
785 
786# Process only Python and Go files, exclude tests
787aid --include "*.py,*.go" --exclude "*test*,*spec*" ./
788 
789# Git history analysis with AI insights
790aid .git --with-analysis-prompt --git-limit=500
791 
792# Raw text processing for documentation
793aid ./docs --raw
794 
795# Force single-threaded processing for debugging (-v, -vv, -vvv)
796aid ./complex-code -w 1 -vv
797 
798# Custom output with absolute paths
799aid ./lib --output=/tmp/analysis.txt --file-path-type=absolute
800 
801# CI/CD integration with clean output
802aid ./internal --summary-type=ci-friendly --no-emoji
803```
804 
805## ⚠️ Limitations
806 
807- **Syntax Errors**: Files with syntax errors may be skipped or partially processed
808- **Dynamic Features**: Runtime-determined types/interfaces in dynamic languages are not resolved
809- **Macro Expansion**: Complex macros (Rust, C++) show pre-expansion source
810- **Generated Code**: Consider using `.aidignore` to skip generated files
811 
812## 🔒 Security Considerations
813 
814**⚠️ Important**: AI Distiller extracts code structure which may include:
815- Function and variable names that could reveal business logic (e.g., `processPayment`, `calculateTaxEvasion`)
816- API endpoints and internal routes (e.g., `/api/v1/internal/user-data`)
817- Type information and data structures
818- Comments and docstrings (unless stripped)
819- File paths revealing project structure or codenames
820 
821**Recommendations**:
8221. Always review output before sending to external services
8232. Use `--comments=0` to remove potentially sensitive documentation
8243. Consider running a secrets scanner on your codebase first
8254. For maximum security, run AI Distiller in an isolated environment
8265. Future: We're exploring an `--obfuscate` flag to anonymize sensitive identifiers
827 
828 
829## 🛠️ Advanced Usage
830 
831### ⚡ Parallel Processing
832 
833AI Distiller now supports parallel processing for significantly faster analysis of large codebases:
834 
835```bash
836# Use default parallel processing (80% of CPU cores)
837aid ./src
838 
839# Force serial processing (original behavior)
840aid ./src -w 1
841 
842# Use specific number of workers
843aid ./src -w 16
844 
845# Check performance with verbose output
846aid ./src -v  # Shows: "Using 25 parallel workers (32 CPU cores available)"
847```
848 
849**Performance Benefits**:
850- React packages: 3.5s → 0.5s (7x faster)
851- Large codebases: Near-linear speedup with CPU cores
852- Maintains identical output order as serial processing
853 
854### Processing from stdin
855 
856AI Distiller can process code directly from stdin, perfect for:
857- Quick code snippet analysis
858- Pipeline integration
859- Testing without creating files
860- Dynamic code generation workflows
861 
862```bash
863# Auto-detect language from stdin
864echo 'class User { getName() { return this.name; } }' | aid --format text
865 
866# Explicit language specification
867cat mycode.php | aid --lang php --private=0 --protected=0
868 
869# Use "-" to explicitly read from stdin
870aid - --lang python < snippet.py
871 
872# Pipeline example: extract structure from generated code
873generate-code.sh | aid --lang typescript --format json
874```
875 
876**Language Detection**: When using stdin without `--lang`, AI Distiller automatically detects the language based on syntax patterns. Supported languages for auto-detection: python, typescript, javascript, go, ruby, swift, rust, java, c#, kotlin, c++, php.
877 
878### Integration with AI Tools
879 
880```bash
881# Create a context file for Claude or GPT
882aid ./src --format text --implementation=0 > context.txt
883 
884# Generate a codebase summary for RAG systems
885aid . --format json-structured | jq -r '.files[].symbols[].name' > symbols.txt
886 
887# Extract API surface for documentation
888aid ./api --comments=0 --implementation=0 --format md > api-ref.md
889 
890# Extract only method signatures (no fields/properties) - great for large codebases
891aid ./src --fields=0 --implementation=0 > methods-only.txt
892 
893# Extract only data structures (no method noise)  
894aid ./models --methods=0 > data-structures.txt
895 
896# Focus on public API methods only
897aid ./services --fields=0 --private=0 --protected=0 --internal=0
898```
899 
900### 🚫 Ignoring Files with .aidignore
901 
902AI Distiller respects `.aidignore` files for excluding files and directories from processing. The syntax is similar to `.gitignore`.
903 
904#### Important: What AI Distiller Processes
905 
906AI Distiller only processes source code files with these extensions:
907- **Python**: `.py`, `.pyw`, `.pyi`
908- **JavaScript**: `.js`, `.mjs`, `.cjs`, `.jsx`
909- **TypeScript**: `.ts`, `.tsx`, `.d.ts`
910- **Go**: `.go`
911- **Rust**: `.rs`
912- **Ruby**: `.rb`, `.rake`, `.gemspec`
913- **Java**: `.java`
914- **C#**: `.cs`
915- **Kotlin**: `.kt`, `.kts`
916- **C++**: `.cpp`, `.cc`, `.cxx`, `.c++`, `.h`, `.hpp`, `.hh`, `.hxx`, `.h++`
917- **PHP**: `.php`, `.phtml`, `.php3`, `.php4`, `.php5`, `.php7`, `.phps`, `.inc`
918- **Swift**: `.swift`
919 
920**Note**: Files like `.log`, `.txt`, `.md`, images, PDFs, and other non-source files are automatically ignored by AI Distiller, so you don't need to add them to `.aidignore`.
921 
922#### Default Ignored Directories
923 
924AI Distiller automatically ignores these common dependency and build directories:
925- `node_modules/` - npm packages
926- `vendor/` - Go and PHP dependencies
927- `target/` - Rust build output
928- `build/`, `dist/` - Common build directories
929- `__pycache__/`, `.pytest_cache/`, `venv/`, `.venv/`, `env/`, `.env/` - Python
930- `.gradle/`, `gradle/` - Java/Kotlin
931- `Pods/` - Swift/iOS dependencies
932- `.bundle/` - Ruby bundler
933- `bin/`, `obj/` - Compiled binaries
934- `.vs/`, `.idea/`, `.vscode/` - IDE directories
935- `coverage/`, `.nyc_output/` - Test coverage
936- `bower_components/` - Legacy JavaScript
937- `.terraform/` - Terraform
938- `.git/`, `.svn/`, `.hg/` - Version control
939 
940You can override these defaults using `!` patterns in `.aidignore` (see Advanced Usage below).
941 
942#### Basic Syntax
943 
944Create a `.aidignore` file in your project root or any subdirectory:
945 
946```bash
947# Comments start with hash
948*.test.js          # Ignore test files
949*.spec.ts          # Ignore spec files
950temp/              # Ignore temp directory
951build/             # Ignore build directory
952/secrets.py        # Ignore secrets.py only in root
953node_modules/      # Ignore node_modules everywhere
954**/*.bak           # Ignore .bak files in any directory
955src/test_*         # Ignore test_* files in src/
956!important.test.js # Don't ignore important.test.js (negation)
957```
958 
959#### How It Works
960 
961- `.aidignore` files work recursively - place them in any directory
962- Patterns are relative to the directory containing the `.aidignore` file
963- Use `/` prefix for patterns relative to the `.aidignore` location
964- Use `**` for recursive matching
965- Directory patterns should end with `/`
966- Use `!` prefix to negate a pattern (re-include previously ignored files)
967 
968#### Examples
969 
970```bash
971# .aidignore in project root
972node_modules/       # Excludes all node_modules directories
973*.test.js          # Excludes all test files
974*.spec.ts          # Excludes all spec files
975dist/              # Excludes dist directory
976.env.py            # Excludes environment config files
977vendor/            # Excludes vendor directory
978 
979# More specific patterns
980src/**/test_*.py   # Test files in src subdirectories
981!src/test_utils.py # But include this specific test file
982/config/*.local.py # Local config files in root config dir
983**/*_generated.go  # Generated Go files anywhere
984```
985 
986#### Advanced Usage: Including Normally Ignored Content
987 
988##### Include Default-Ignored Directories
989 
990Use `!` patterns to include directories that are ignored by default:
991 
992```bash
993# Include vendor directory for analysis
994!vendor/
995 
996# Include specific node_modules package
997!node_modules/my-local-package/
998 
999# Include Python virtual environment
1000!venv/
1001```
1002 
1003##### Include Non-Source Files
1004 
1005You can also include files that AI Distiller normally doesn't process:
1006 
1007```bash
1008# Include all markdown files
1009!*.md
1010!**/*.md
1011 
1012# Include configuration files
1013!*.yaml
1014!*.json
1015!.env
1016 
1017# Include specific documentation
1018!docs/**/*.txt
1019!README.md
1020!CHANGELOG.md
1021```
1022 
1023When you include non-source files with `!` patterns, AI Distiller will include their raw content in the output.
1024 
1025#### Nested .aidignore Files
1026 
1027You can place `.aidignore` files in subdirectories for more specific control:
1028 
1029```bash
1030# project/.aidignore
1031*.test.py
1032!vendor/            # Include vendor in this project
1033 
1034# project/src/.aidignore
1035test_*.go
1036*.mock.ts
1037!test_helpers.ts   # Exception: include test_helpers.ts
1038```
1039 
1040### 🎯 Git History Analysis Mode
1041 
1042AI Distiller includes a special mode for analyzing git repositories. When you pass a `.git` directory, it switches to git log mode:
1043 
1044```bash
1045# View formatted git history
1046aid .git
1047 
1048# Limit to recent commits (default is 200)
1049aid .git --git-limit=500
1050 
1051# Include AI analysis prompt for comprehensive insights
1052aid .git --git-limit=1000 --with-analysis-prompt
1053```
1054 
1055The `--with-analysis-prompt` flag adds a sophisticated prompt combined with git history that AI agents can use to generate:
1056- **Contributor statistics** with expertise areas and collaboration patterns
1057- **Timeline analysis** with development phases and activity visualization
1058- **Functional categorization** of commits (features, fixes, refactoring)
1059- **Codebase evolution insights** including technology shifts
1060- **Actionable recommendations** based on discovered patterns
1061 
1062The output file contains both the analysis prompt and formatted git history, ready for AI agents to process. Perfect for understanding project history, identifying knowledge silos, or generating impressive development reports.
1063 
1064## ❓ FAQ
1065 
1066<details>
1067<summary><strong>How accurate are the token counts?</strong></summary>
1068 
1069Token counts are estimated using OpenAI's cl100k_base tokenizer (1 token ≈ 4 characters). Actual token usage varies by model - Claude and GPT-4 use similar tokenizers, while others may differ by ±10%.
1070</details>
1071 
1072<details>
1073<summary><strong>Can AI Distiller handle very large repositories?</strong></summary>
1074 
1075Yes! We've tested on repositories with 50,000+ files. The parallel processing mode (`-w` flag) scales linearly with CPU cores. Memory usage is bounded - large files are processed in streaming chunks.
1076</details>
1077 
1078<details>
1079<summary><strong>What about generated code and vendor directories?</strong></summary>
1080 
1081Create a `.aidignore` file (same syntax as `.gitignore`) to exclude generated files, vendor directories, or any paths you don't want processed.
1082</details>
1083 
1084<details>
1085<summary><strong>What happens with unsupported file types?</strong></summary>
1086 
1087Files with unknown or unsupported extensions are automatically skipped - no errors, no interruption. AI Distiller only processes files it has parsers for, ensuring clean and relevant output. This means you can safely run it on mixed repositories containing documentation, images, configs, etc.
1088</details>
1089 
1090<details>
1091<summary><strong>Is my code sent anywhere?</strong></summary>
1092 
1093No! AI Distiller runs 100% locally. It only extracts and formats your code structure - you decide what to do with the output. The tool itself makes no network connections.
1094</details>
1095 
1096<details>
1097<summary><strong>Which programming languages are supported?</strong></summary>
1098 
1099Currently 12+ languages via tree-sitter: Python, TypeScript, JavaScript, Go, Java, C#, Rust, Ruby, Swift, Kotlin, PHP, C++. All parsers are bundled in the binary - no external dependencies needed.
1100</details>
1101 
1102## 🤝 Contributing
1103 
1104We welcome contributions! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
1105 
1106### Development Setup
1107 
1108```bash
1109# Clone and setup
1110git clone https://github.com/janreges/ai-distiller
1111cd ai-distiller
1112make dev-init    # Initialize development environment
1113 
1114# Run tests
1115make test         # Unit tests
1116make test-integration  # Integration tests
1117 
1118# Build binary
1119make build        # Build for current platform
1120```
1121 
1122### Building Release Binaries
1123 
1124AI Distiller requires CGO for full language support via tree-sitter parsers. To build release binaries for all supported platforms:
1125 
1126#### Prerequisites
1127 
1128**Ubuntu/Debian:**
1129```bash
1130# Install cross-compilation toolchains
1131sudo apt-get update
1132sudo apt-get install -y gcc-aarch64-linux-gnu gcc-mingw-w64-x86-64
1133 
1134# For macOS cross-compilation, you need osxcross:
1135# 1. Clone osxcross: git clone https://github.com/tpoechtrager/osxcross tools/osxcross
1136# 2. Obtain macOS SDK (see https://github.com/tpoechtrager/osxcross#packaging-the-sdk)
1137# 3. Place SDK in tools/osxcross/tarballs/
1138# 4. Build osxcross: cd tools/osxcross && ./build.sh
1139```
1140 
1141#### Build All Platforms
1142 
1143```bash
1144# Build release archives for all platforms
1145./scripts/build-releases.sh
1146 
1147# This creates:
1148# - aid-linux-amd64.tar.gz    (Linux 64-bit)
1149# - aid-linux-arm64.tar.gz    (Linux ARM64)
1150# - aid-darwin-amd64.tar.gz   (macOS Intel)
1151# - aid-darwin-arm64.tar.gz   (macOS Apple Silicon)
1152# - aid-windows-amd64.zip     (Windows 64-bit)
1153```
1154 
1155The script automatically detects available toolchains and builds for all possible platforms. Each archive contains the `aid` binary (or `aid.exe` for Windows) with full language support.
1156 
1157**Note**: Without proper toolchains, only the native platform will be built.
1158 
1159## 📄 License
1160 
1161MIT License - see [LICENSE](LICENSE) for details.
1162 
1163## 🙏 Acknowledgments
1164 
1165- Built on [tree-sitter](https://tree-sitter.github.io/) for accurate parsing
1166- Inspired by the need for better AI-code interaction
1167- Created with ❤️ by Claude Code & Ján Regeš from [SiteOne](https://www.siteone.io/) (Czech Republic).
1168