Token Optimizer MCP

1# Token Optimizer MCP
2 
3> Intelligent token optimization through caching, compression, and smart tooling for Claude Code and Claude Desktop
4 
5## Overview
6 
7Token Optimizer MCP is a Model Context Protocol (MCP) server that reduces context window usage by 60-90% through intelligent caching, compression, and smart tool replacements. By storing compressed content externally in SQLite and providing optimized alternatives to standard tools, the server helps you maximize your available context window.
8 
9**Production Results**: 60-90% token reduction across 38,000+ operations in real-world usage.
10 
11## Key Features
12 
13- **Smart Tool Replacements**: Automatic optimization for Read, Grep, Glob, and more
14- **Context Window Optimization**: Store content externally to free up context space
15- **High Compression**: Brotli compression (2-4x typical, up to 82x for repetitive content)
16- **Persistent Caching**: SQLite-based cache that persists across sessions
17- **Accurate Token Counting**: Uses tiktoken for precise token measurements
18- **61 Specialized Tools**: File operations, API caching, database optimization, monitoring, and more
19- **Zero External Dependencies**: Completely offline operation
20- **Production Ready**: Built with TypeScript for reliability
21 
22## Installation
23 
24### Quick Install (Recommended)
25 
26#### Windows
27 
28```powershell
29# Run PowerShell as Administrator, then:
30Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
31 
32# Install globally (hooks install automatically!)
33npm install -g @ooples/token-optimizer-mcp
34```
35 
36#### macOS / Linux
37 
38```bash
39# Install globally (hooks install automatically!)
40npm install -g @ooples/token-optimizer-mcp
41```
42 
43That's it! The postinstall script will automatically:
441. ✅ Install token-optimizer-mcp globally via npm
452. ✅ Auto-detect and configure all installed AI tools (Claude Desktop, Cursor, Cline, etc.)
463. ✅ Set up automatic token optimization on every tool call
474. ✅ Configure workspace trust and execution permissions
48 
49**Result**: 60-90% token reduction across all operations!
50 
51**Note**: If automatic installation is skipped (e.g., in CI environments), you can manually run the installer:
52- Windows: `powershell -ExecutionPolicy Bypass -File install-hooks.ps1`
53- macOS/Linux: `bash install-hooks.sh`
54 
55### Manual Configuration
56 
57For detailed platform-specific installation instructions, see [docs/HOOKS-INSTALLATION.md](./docs/HOOKS-INSTALLATION.md).
58 
59## Available Tools (65 Total)
60 
61### Core Caching & Optimization (8 tools)
62 
63<details>
64<summary>Click to expand</summary>
65 
66- **optimize_text** - Compress and cache text (primary tool for token reduction)
67- **get_cached** - Retrieve previously cached text
68- **compress_text** - Compress text using Brotli
69- **decompress_text** - Decompress Brotli-compressed text
70- **count_tokens** - Count tokens using tiktoken (GPT-4 tokenizer)
71- **analyze_optimization** - Analyze text and get optimization recommendations
72- **get_cache_stats** - View cache hit rates and compression ratios
73- **clear_cache** - Clear all cached data
74 
75**Usage Example**:
76```typescript
77// Cache large content to remove it from context window
78optimize_text({
79  text: "Large API response or file content...",
80  key: "api-response-key",
81  quality: 11
82})
83// Result: 60-90% token reduction
84```
85 
86</details>
87 
88### Smart File Operations (10 tools)
89 
90<details>
91<summary>Click to expand</summary>
92 
93Optimized replacements for standard file tools with intelligent caching and diff-based updates:
94 
95- **smart_read** - Read files with 80% token reduction through caching and diffs
96- **smart_write** - Write files with verification and change tracking
97- **smart_edit** - Line-based file editing with diff-only output (90% reduction)
98- **smart_grep** - Search file contents with match-only output (80% reduction)
99- **smart_glob** - File pattern matching with path-only results (75% reduction)
100- **smart_diff** - Git diffs with diff-only output (85% reduction)
101- **smart_branch** - Git branch listing with structured JSON (60% reduction)
102- **smart_log** - Git commit history with smart filtering (75% reduction)
103- **smart_merge** - Git merge management with conflict analysis (80% reduction)
104- **smart_status** - Git status with status-only output (70% reduction)
105 
106**Usage Example**:
107```typescript
108// Read a file with automatic caching
109smart_read({ path: "/path/to/file.ts" })
110// First read: full content
111// Subsequent reads: only diff (80% reduction)
112```
113 
114</details>
115 
116### API & Database Operations (10 tools)
117 
118<details>
119<summary>Click to expand</summary>
120 
121Intelligent caching and optimization for external data sources:
122 
123- **smart_api_fetch** - HTTP requests with caching and retry logic (83% reduction on cache hits)
124- **smart_cache_api** - API response caching with TTL/ETag/event-based strategies
125- **smart_database** - Database queries with connection pooling and caching (83% reduction)
126- **smart_sql** - SQL query analysis with optimization suggestions (83% reduction)
127- **smart_schema** - Database schema analysis with intelligent caching
128- **smart_graphql** - GraphQL query optimization with complexity analysis (83% reduction)
129- **smart_rest** - REST API analysis with endpoint discovery (83% reduction)
130- **smart_orm** - ORM query optimization with N+1 detection (83% reduction)
131- **smart_migration** - Database migration tracking (83% reduction)
132- **smart_websocket** - WebSocket connection management with message tracking
133 
134**Usage Example**:
135```typescript
136// Fetch API with automatic caching
137smart_api_fetch({
138  method: "GET",
139  url: "https://api.example.com/data",
140  ttl: 300
141})
142// Cached responses: 95% token reduction
143```
144 
145</details>
146 
147### Build & Test Operations (10 tools)
148 
149<details>
150<summary>Click to expand</summary>
151 
152Development workflow optimization with intelligent caching:
153 
154- **smart_build** - TypeScript builds with diff-based change detection
155- **smart_test** - Test execution with incremental test selection
156- **smart_lint** - ESLint with incremental analysis and auto-fix
157- **smart_typecheck** - TypeScript type checking with caching
158- **smart_install** - Package installation with dependency analysis
159- **smart_docker** - Docker operations with layer analysis
160- **smart_logs** - Log aggregation with pattern filtering
161- **smart_network** - Network diagnostics with anomaly detection
162- **smart_processes** - Process monitoring with resource tracking
163- **smart_system_metrics** - System resource monitoring with performance recommendations
164 
165**Usage Example**:
166```typescript
167// Run tests with caching
168smart_test({
169  onlyChanged: true,  // Only test changed files
170  coverage: true
171})
172```
173 
174</details>
175 
176### Advanced Caching (10 tools)
177 
178<details>
179<summary>Click to expand</summary>
180 
181Enterprise-grade caching strategies with 87-92% token reduction:
182 
183- **smart_cache** - Multi-tier cache (L1/L2/L3) with 6 eviction strategies (90% reduction)
184- **cache_warmup** - Intelligent cache pre-warming with schedule support (87% reduction)
185- **cache_analytics** - Real-time dashboards and trend analysis (88% reduction)
186- **cache_benchmark** - Performance testing and strategy comparison (89% reduction)
187- **cache_compression** - 6 compression algorithms with adaptive selection (89% reduction)
188- **cache_invalidation** - Dependency tracking and pattern-based invalidation (88% reduction)
189- **cache_optimizer** - ML-based recommendations and bottleneck detection (89% reduction)
190- **cache_partition** - Sharding and consistent hashing (87% reduction)
191- **cache_replication** - Distributed replication with conflict resolution (88% reduction)
192- **predictive_cache** - ML-based predictive caching with ARIMA/LSTM (91% reduction)
193 
194**Usage Example**:
195```typescript
196// Configure multi-tier cache
197smart_cache({
198  operation: "configure",
199  evictionStrategy: "LRU",
200  l1MaxSize: 1000,
201  l2MaxSize: 10000
202})
203```
204 
205</details>
206 
207### Monitoring & Dashboards (7 tools)
208 
209<details>
210<summary>Click to expand</summary>
211 
212Comprehensive monitoring with 88-92% token reduction through intelligent caching:
213 
214- **alert_manager** - Multi-channel alerting (email, Slack, webhook) with routing (89% reduction)
215- **metric_collector** - Time-series metrics with multi-source support (88% reduction)
216- **monitoring_integration** - External platform integration (Prometheus, Grafana, Datadog) (87% reduction)
217- **custom_widget** - Dashboard widgets with template caching (88% reduction)
218- **data_visualizer** - Interactive visualizations with SVG optimization (92% reduction)
219- **health_monitor** - System health checks with state compression (91% reduction)
220- **log_dashboard** - Log analysis with pattern detection (90% reduction)
221 
222**Usage Example**:
223```typescript
224// Create an alert
225alert_manager({
226  operation: "create-alert",
227  alertName: "high-cpu-usage",
228  channels: ["slack", "email"],
229  threshold: { type: "above", value: 80 }
230})
231```
232 
233</details>
234 
235### System Operations (6 tools)
236 
237<details>
238<summary>Click to expand</summary>
239 
240System-level operations with smart caching:
241 
242- **smart_cron** - Scheduled task management (cron/Windows Task Scheduler) (85% reduction)
243- **smart_user** - User and permission management across platforms (86% reduction)
244- **smart_ast_grep** - Structural code search with AST indexing (83% reduction)
245- **get_session_stats** - Session-level token usage statistics
246- **analyze_project_tokens** - Project-wide token analysis and cost estimation
247- **optimize_session** - Compress large file operations from current session
248 
249**Usage Example**:
250```typescript
251// View session token usage
252get_session_stats({})
253// Result: Detailed breakdown of token usage by tool
254```
255 
256</details>
257 
258## How It Works
259 
260 
261 
262### Token Analytics (4 tools)
263 
264<details>
265<summary>Click to expand</summary>
266 
267Granular token usage analytics for pinpointing optimization opportunities:
268 
269- **get_hook_analytics** - Token usage breakdown by hook phase (PreToolUse, PostToolUse, etc.)
270- **get_action_analytics** - Token usage breakdown by tool/action (Read, Write, Grep, etc.)
271- **get_mcp_server_analytics** - Token usage breakdown by MCP server (token-optimizer, filesystem, etc.)
272- **export_analytics** - Export analytics data in JSON or CSV format with filtering
273 
274**Usage Example**:
275```typescript
276// Get per-hook analytics
277get_hook_analytics({
278  startDate: "2025-01-01T00:00:00Z",
279  endDate: "2025-12-31T23:59:59Z"
280})
281// Result: Shows which hooks consume the most tokens
282 
283// Get per-action analytics
284get_action_analytics({})
285// Result: Shows which tools use the most tokens
286 
287// Export analytics as CSV
288export_analytics({
289  format: "csv",
290  hookPhase: "PreToolUse"
291})
292// Result: CSV export filtered by PreToolUse hook
293```
294 
295**Key Features**:
296- Per-hook phase tracking (PreToolUse, PostToolUse, SessionStart, etc.)
297- Per-action tracking (Read, Write, count_tokens, etc.)
298- Per-MCP-server tracking (token-optimizer, filesystem, GitHub, etc.)
299- Date range filtering
300- JSON and CSV export
301- Persistent storage with SQLite
302- Zero performance impact (async batched writes)
303 
304</details>
305 
306### Global Hooks System (7-Phase Optimization)
307 
308When global hooks are installed, token-optimizer-mcp runs automatically on **every tool call**:
309 
310```
311┌─────────────────────────────────────────────────────────────┐
312│ Phase 1: PreToolUse - Tool Replacement                      │
313│ ├─ Read   → smart_read   (80% token reduction)             │
314│ ├─ Grep   → smart_grep   (80% token reduction)             │
315│ └─ Glob   → smart_glob   (75% token reduction)             │
316└─────────────────────────────────────────────────────────────┘
317                          ↓
318┌─────────────────────────────────────────────────────────────┐
319│ Phase 2: Input Validation - Cache Lookups                   │
320│ └─ get_cached checks if operation was already done          │
321└─────────────────────────────────────────────────────────────┘
322                          ↓
323┌─────────────────────────────────────────────────────────────┐
324│ Phase 3: PostToolUse - Output Optimization                  │
325│ ├─ optimize_text for large outputs                          │
326│ └─ compress_text for repeated content                       │
327└─────────────────────────────────────────────────────────────┘
328                          ↓
329┌─────────────────────────────────────────────────────────────┐
330│ Phase 4: Session Tracking                                   │
331│ └─ Log all operations to operations-{sessionId}.csv         │
332└─────────────────────────────────────────────────────────────┘
333                          ↓
334┌─────────────────────────────────────────────────────────────┐
335│ Phase 5: UserPromptSubmit - Prompt Optimization             │
336│ └─ Optimize user prompts before sending to API              │
337└─────────────────────────────────────────────────────────────┘
338                          ↓
339┌─────────────────────────────────────────────────────────────┐
340│ Phase 6: PreCompact - Pre-Compaction Optimization           │
341│ └─ Optimize before Claude Code compacts the conversation    │
342└─────────────────────────────────────────────────────────────┘
343                          ↓
344┌─────────────────────────────────────────────────────────────┐
345│ Phase 7: Metrics & Reporting                                │
346│ └─ Track token reduction metrics and generate reports       │
347└─────────────────────────────────────────────────────────────┘
348```
349 
350## Production Performance
351 
352Based on 38,000+ operations in real-world usage:
353 
354| Tool Category | Avg Token Reduction | Cache Hit Rate |
355|--------------|-------------------|----------------|
356| File Operations | 60-90% | >80% |
357| API Responses | 83-95% | >75% |
358| Database Queries | 83-90% | >70% |
359| Build/Test Output | 70-85% | >65% |
360 
361**Per-Session Savings**: 300K-700K tokens (worth $0.90-$2.10 at $3/M tokens)
362 
363## Usage Examples
364 
365### Basic Caching
366 
367```typescript
368// Cache large content to remove from context window
369const result = await optimize_text({
370  text: "Large API response or file content...",
371  key: "cache-key",
372  quality: 11
373});
374// Result: Original tokens removed, only cache key remains (~50 tokens)
375 
376// Retrieve later
377const cached = await get_cached({ key: "cache-key" });
378// Result: Full original content restored
379```
380 
381### Smart File Reading
382 
383```typescript
384// First read: full content
385await smart_read({ path: "/src/app.ts" });
386 
387// Subsequent reads: only changes (80% reduction)
388await smart_read({ path: "/src/app.ts" });
389```
390 
391### API Caching
392 
393```typescript
394// First request: fetch and cache
395await smart_api_fetch({
396  method: "GET",
397  url: "https://api.example.com/data",
398  ttl: 300
399});
400 
401// Subsequent requests: cached (95% reduction)
402await smart_api_fetch({
403  method: "GET",
404  url: "https://api.example.com/data"
405});
406```
407 
408### Session Analysis
409 
410```typescript
411// View token usage for current session
412await get_session_stats({});
413// Result: Breakdown by tool, operation, and savings
414 
415// Analyze entire project
416await analyze_project_tokens({
417  projectPath: "/path/to/project"
418});
419// Result: Cost estimation and optimization opportunities
420```
421 
422## Technology Stack
423 
424- **Runtime**: Node.js 20+
425- **Language**: TypeScript
426- **Database**: SQLite (better-sqlite3)
427- **Token Counting**: tiktoken (GPT-4 tokenizer)
428- **Compression**: Brotli (built-in Node.js)
429- **Caching**: Multi-tier LRU/LFU/FIFO caching
430- **Protocol**: MCP SDK (@modelcontextprotocol/sdk)
431 
432## Supported AI Tools
433 
434The automated installer detects and configures token-optimizer-mcp for:
435 
436- ✅ **Claude Code** - CLI with global hooks integration
437- ✅ **Claude Desktop** - Native desktop application
438- ✅ **Cursor IDE** - AI-first code editor
439- ✅ **Cline** - VS Code extension (formerly Claude Dev)
440- ✅ **GitHub Copilot** - VS Code with MCP support
441- ✅ **Windsurf IDE** - AI-powered development environment
442 
443**No manual configuration needed** - the installer automatically detects and configures all installed tools!
444 
445## Documentation
446 
447- **[Detailed Tool Reference](./docs/TOOLS.md)** - Complete documentation for all 61 tools
448- **[Installation Guide](./docs/HOOKS-INSTALLATION.md)** - Platform-specific installation instructions
449- **[Contributing Guide](./docs/CONTRIBUTING.md)** - Development setup and contribution guidelines
450 
451## Performance Characteristics
452 
453- **Compression Ratio**: 2-4x typical (up to 82x for repetitive content)
454- **Context Window Savings**: 60-90% average across all operations
455- **Cache Hit Rate**: >80% in typical usage
456- **Operation Overhead**: <10ms for cache operations (optimized from 50-70ms)
457- **Compression Speed**: ~1ms per KB of text
458- **Hook Overhead**: <10ms per operation (7x improvement from in-memory optimizations)
459 
460### Performance Optimizations
461 
462The PowerShell hooks have been optimized to reduce overhead from 50-70ms to <10ms through:
463 
464- **In-Memory Session State**: Session data kept in memory instead of disk I/O on every operation
465- **Batched Log Writes**: Operation logs buffered and flushed every 5 seconds or 100 operations
466- **Lazy Persistence**: Disk writes only occur when necessary (session end, optimization, reports)
467 
468### Environment Variables
469 
470Control hook behavior with these environment variables:
471 
472#### Performance Controls
473 
474- **`TOKEN_OPTIMIZER_USE_FILE_SESSION`** (default: `false`)
475  - Set to `true` to revert to file-based session tracking (legacy mode)
476  - Use if you encounter issues with in-memory session state
477  - Example: `$env:TOKEN_OPTIMIZER_USE_FILE_SESSION = "true"`
478 
479- **`TOKEN_OPTIMIZER_SYNC_LOG_WRITES`** (default: `false`)
480  - Set to `true` to disable batched log writes
481  - Forces immediate writes to disk (slower but more resilient)
482  - Use for debugging or if logs are being lost
483  - Example: `$env:TOKEN_OPTIMIZER_SYNC_LOG_WRITES = "true"`
484 
485- **`TOKEN_OPTIMIZER_DEBUG_LOGGING`** (default: `true`)
486  - Set to `false` to disable DEBUG-level logging
487  - Reduces log file size and improves performance
488  - INFO/WARN/ERROR logs still written
489  - Example: `$env:TOKEN_OPTIMIZER_DEBUG_LOGGING = "false"`
490 
491#### Development Path
492 
493- **`TOKEN_OPTIMIZER_DEV_PATH`**
494  - Path to local development installation
495  - Automatically set to `~/source/repos/token-optimizer-mcp` if not specified
496  - Override for custom development paths
497  - Example: `$env:TOKEN_OPTIMIZER_DEV_PATH = "C:\dev\token-optimizer-mcp"`
498 
499**Performance Impact**: Using in-memory mode (default) provides a 7x improvement in hook overhead:
500- Before: 50-70ms per hook operation
501- After: <10ms per hook operation
502- 85% reduction in hook latency
503 
504## Monitoring Token Savings
505 
506### Real-Time Session Monitoring
507 
508**To view your actual token SAVINGS**, use the `get_session_stats` tool:
509 
510```typescript
511// View current session statistics with token savings breakdown
512await get_session_stats({});
513```
514 
515**Output includes:**
516- **Total tokens saved** (this is the actual savings amount!)
517- **Token reduction percentage** (e.g., "60% reduction")
518- **Cache hit rate** and **compression ratios**
519- **Breakdown by tool** (Read, Grep, Glob, etc.)
520- **Top 10 most optimized operations** with before/after comparison
521 
522**Example Output:**
523```json
524{
525  "sessionId": "abc-123",
526  "totalTokensSaved": 125430,  // ← THIS is your savings!
527  "tokenReductionPercent": 68.2,
528  "originalTokens": 184000,
529  "optimizedTokens": 58570,
530  "cacheHitRate": 72.0,
531  "byTool": {
532    "smart_read": { "saved": 45000, "percent": 80 },
533    "smart_grep": { "saved": 32000, "percent": 75 }
534  }
535}
536```
537 
538### Session Tracking Files
539 
540All operations are automatically tracked in session data files:
541 
542**Location**: `~/.claude-global/hooks/data/current-session.txt`
543 
544**Format**:
545 
546```json
547{
548  "sessionId": "abc-123",
549  "sessionStart": "20251031-082211",
550  "totalOperations": 1250,      // ← Number of operations
551  "totalTokens": 184000,         // ← Cumulative token COUNT
552  "lastOptimized": 1698765432,
553  "savings": {                   // ← Auto-updated every 10 operations (Issue #113)
554    "totalTokensSaved": 125430,  // Tokens saved by compression
555    "tokenReductionPercent": 68.2,  // Percentage of tokens saved
556    "originalTokens": 184000,    // Original token count before optimization
557    "optimizedTokens": 58570,    // Token count after optimization
558    "cacheHitRate": 42.5,        // Cache hit rate percentage
559    "compressionRatio": 0.32,    // Compression efficiency (lower is better)
560    "lastUpdated": "20251031-092500"  // Last savings update timestamp
561  }
562}
563```
564 
565**New in v1.x**: The `savings` object is now automatically updated every 10 operations, eliminating the need to manually call `get_session_stats()` for real-time monitoring. This provides instant visibility into token optimization performance.
566 
567**How it works**:
568- Every 10 operations, the PowerShell hooks automatically call `get_cache_stats()` MCP tool
569- Savings metrics are calculated from cache performance data (compression ratio, original vs compressed sizes)
570- The session file is atomically updated with the latest savings data
571- If the MCP call fails, the update is skipped gracefully without blocking operations
572 
573**Note**: For detailed per-operation analysis, use `get_session_stats()`. The session file provides high-level aggregate metrics.
574 
575### Project-Wide Analysis
576 
577Analyze token usage across your entire project:
578 
579```typescript
580// Analyze project token costs
581await analyze_project_tokens({
582  projectPath: "/path/to/project"
583});
584```
585 
586**Provides:**
587- Total token cost estimation
588- Largest files by token count
589- Optimization opportunities
590- Cost projections at current API rates
591 
592### Cache Performance
593 
594Monitor cache hit rates and storage efficiency:
595 
596```typescript
597// View cache statistics
598await get_cache_stats({});
599```
600 
601**Metrics:**
602- Total entries
603- Cache hit rate (%)
604- Average compression ratio
605- Total storage saved
606- Most frequently accessed keys
607 
608## Troubleshooting
609 
610### Common Issues and Solutions
611 
612#### Issue: "Invalid or malformed JSON" in Claude Code Settings
613 
614**Symptom**: Claude Code shows "Invalid Settings" error after running install-hooks
615 
616**Cause**: UTF-8 BOM (Byte Order Mark) was added to settings.json files
617 
618**Solution**: Upgrade to v3.0.2+ which fixes the BOM issue:
619```bash
620npm install -g @ooples/token-optimizer-mcp@latest
621```
622 
623If you're already on v3.0.2+, manually remove the BOM:
624```powershell
625# Windows: Remove BOM from settings.json
626$content = Get-Content "~/.claude/settings.json" -Raw
627$content = $content -replace '^\xEF\xBB\xBF', ''
628$content | Set-Content "~/.claude/settings.json" -Encoding utf8NoBOM
629```
630 
631```bash
632# Linux: Remove BOM from settings.json
633sed -i '1s/^\xEF\xBB\xBF//' ~/.claude/settings.json
634 
635# macOS: Remove BOM from settings.json (BSD sed requires empty string after -i)
636sed -i '' '1s/^\xef\xbb\xbf//' ~/.claude/settings.json
637```
638 
639#### Issue: Hooks Not Working After Installation
640 
641**Symptom**: Token optimization not occurring automatically
642 
643**Diagnosis**:
6441. Check if hooks are installed:
645   ```powershell
646   # Windows
647   Get-Content ~/.claude/settings.json | ConvertFrom-Json | Select-Object -ExpandProperty hooks
648   ```
649   ```bash
650   # macOS/Linux
651   cat ~/.claude/settings.json | jq .hooks
652   ```
653 
6542. Verify dispatcher.ps1 exists:
655   ```powershell
656   # Windows
657   Test-Path ~/.claude-global/hooks/dispatcher.ps1
658   ```
659   ```bash
660   # macOS/Linux
661   [ -f ~/.claude-global/hooks/dispatcher.sh ] && echo "Exists" || echo "Missing"
662   ```
663 
664**Solution**: Re-run the installer:
665```powershell
666# Windows
667powershell -ExecutionPolicy Bypass -File install-hooks.ps1
668```
669```bash
670# macOS/Linux
671bash install-hooks.sh
672```
673 
674#### Issue: Low Cache Hit Rate (<50%)
675 
676**Symptom**: Session stats show cache hit rate below 50%
677 
678**Causes**:
6791. Working with many new files (expected)
6802. Cache was recently cleared
6813. TTL (time-to-live) is too short
682 
683**Solutions**:
6841. **Warm up the cache** before starting work:
685   ```typescript
686   await cache_warmup({
687     paths: ["/path/to/frequently/used/files"],
688     recursive: true
689   });
690   ```
691 
6922. **Increase TTL** for stable APIs:
693   ```typescript
694   await smart_api_fetch({
695     url: "https://api.example.com/data",
696     ttl: 3600  // 1 hour instead of default 5 minutes
697   });
698   ```
699 
7003. **Check cache size limits**:
701   ```typescript
702   await smart_cache({
703     operation: "configure",
704     l1MaxSize: 2000,  // Increase from default 1000
705     l2MaxSize: 20000  // Increase from default 10000
706   });
707   ```
708 
709#### Issue: High Memory Usage
710 
711**Symptom**: Node.js process using excessive memory
712 
713**Cause**: Large cache in memory (L1/L2 tiers)
714 
715**Solution**: Configure cache limits:
716```typescript
717await smart_cache({
718  operation: "configure",
719  evictionStrategy: "LRU",  // Least Recently Used
720  l1MaxSize: 500,  // Reduce L1 cache
721  l2MaxSize: 5000  // Reduce L2 cache
722});
723```
724 
725Or clear the cache:
726```typescript
727await clear_cache({});
728```
729 
730#### Issue: Slow First-Time Operations
731 
732**Symptom**: Initial Read/Grep/Glob operations are slow
733 
734**Cause**: Cache is empty, building indexes
735 
736**Solution**: This is expected behavior. Subsequent operations will be 80-90% faster.
737 
738To pre-warm the cache:
739```typescript
740await cache_warmup({
741  paths: ["/src", "/tests", "/docs"],
742  recursive: true,
743  schedule: "startup"  // Auto-warm on every session start
744});
745```
746 
747#### Issue: "Permission denied" Errors on Windows
748 
749**Symptom**: Cannot write to cache or log files
750 
751**Cause**: PowerShell execution policy or file permissions
752 
753**Solution**:
7541. **Set execution policy**:
755   ```powershell
756   Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
757   ```
758 
7592. **Check file permissions**:
760   ```powershell
761   icacls "$env:USERPROFILE\.token-optimizer"
762   ```
763 
7643. **Re-run installer as Administrator** if needed
765 
766#### Issue: Cache Files Growing Too Large
767 
768**Symptom**: `~/.token-optimizer/cache.db` is >1GB
769 
770**Cause**: Caching very large files or many API responses
771 
772**Solution**:
7731. **Clear old entries**:
774   ```typescript
775   await clear_cache({ olderThan: 7 });  // Clear entries older than 7 days
776   ```
777 
7782. **Reduce cache retention**:
779   ```typescript
780   await smart_cache({
781     operation: "configure",
782     defaultTTL: 3600  // 1 hour instead of 7 days
783   });
784   ```
785 
7863. **Manually delete cache** (nuclear option):
787   ```bash
788   rm -rf ~/.token-optimizer/cache.db
789   ```
790 
791### Getting Help
792 
793If you encounter issues not covered here:
794 
7951. **Check the hook logs**: `~/.claude-global/hooks/logs/dispatcher.log`
7962. **Check session data**: `~/.claude-global/hooks/data/current-session.txt`
7973. **File an issue**: [GitHub Issues](https://github.com/ooples/token-optimizer-mcp/issues)
798   - Include debug logs
799   - Include your OS and Node.js version
800   - Include the output of `get_session_stats`
801 
802## Limitations
803 
804- **Small Text**: Best for content >500 characters (cache overhead on small snippets)
805- **One-Time Content**: No benefit for content that won't be referenced again
806- **Cache Storage**: Automatic cleanup after 7 days to prevent disk usage issues
807- **Token Counting**: Uses GPT-4 tokenizer (approximation for Claude, but close enough)
808 
809## License
810 
811MIT License - see [LICENSE](./LICENSE) for details
812 
813## Author
814 
815Built for optimal Claude Code token efficiency by the ooples team.
816