Context Rot Detection

1# Context Rot Detection
2 
3MCP service that gives AI agents self-awareness about their cognitive state.
4 
5Every long-running AI agent suffers from **context rot** — measurable performance degradation as the context window fills up. Research from [Chroma](https://research.trychroma.com/context-rot), [Stanford](https://arxiv.org/abs/2307.03172) ("lost-in-the-middle"), and [Redis](https://redis.io/blog/context-rot/) confirms this is the #1 practical failure mode in production agent systems.
6 
7An agent experiencing context rot doesn't *know* it's degrading — it just starts making worse decisions. This tool gives agents **real-time visibility into their own cognitive health**.
8 
9## Features
10 
11- **Health score (0–100)** based on token utilization, retrieval accuracy, and session fatigue
12- **Model-specific degradation curves** for 15+ curated models (Claude, GPT, Gemini, o-series)
13- **Auto-resolves any HuggingFace model** — pass a repo ID like `meta-llama/Llama-3.1-70B` and the context window is detected automatically, with results cached in SQLite
14- **Lost-in-the-middle risk scoring** based on Stanford research
15- **Tool-call burden** and **session fatigue** analysis
16- **Actionable recovery recommendations** — compact context, offload to memory, checkpoint, break into subtasks
17- **Per-agent health history** tracking (SQLite)
18- **Service-wide utilization statistics**
19 
20## Quick Start
21 
22### npx (zero install)
23 
24```bash
25npx context-rot-detection
26```
27 
28### npm (global install)
29 
30```bash
31npm install -g context-rot-detection
32context-rot-detection
33```
34 
35## MCP Client Configuration
36 
37### Claude Code
38 
39Add to `.mcp.json` in your project root:
40 
41```json
42{
43  "mcpServers": {
44    "context-rot-detection": {
45      "command": "npx",
46      "args": ["-y", "context-rot-detection"],
47      "env": {
48        "HEALTH_HISTORY_DB": "./health.db"
49      }
50    }
51  }
52}
53```
54 
55### Claude Desktop
56 
57Add to `claude_desktop_config.json`:
58 
59```json
60{
61  "mcpServers": {
62    "context-rot-detection": {
63      "command": "npx",
64      "args": ["-y", "context-rot-detection"],
65      "env": {
66        "HEALTH_HISTORY_DB": "/path/to/health.db"
67      }
68    }
69  }
70}
71```
72 
73### Docker
74 
75```json
76{
77  "mcpServers": {
78    "context-rot-detection": {
79      "command": "docker",
80      "args": [
81        "run", "-i", "--rm",
82        "-v", "context-rot-data:/data",
83        "ghcr.io/milos-product-maker/context-rot-detection:latest"
84      ]
85    }
86  }
87}
88```
89 
90## Configuration
91 
92| Environment Variable | Description | Default |
93|---|---|---|
94| `HEALTH_HISTORY_DB` | Path to SQLite database for health history. Use `:memory:` for ephemeral storage. | `:memory:` |
95| `LOG_FILE` | Path to append structured JSON log lines. Omit to disable file logging. | *(none)* |
96 
97## Tools
98 
99### `check_my_health`
100 
101Analyze the current context window health. Call this periodically during long sessions or before critical decisions.
102 
103**Parameters:**
104 
105| Parameter | Type | Required | Description |
106|---|---|---|---|
107| `token_count` | integer | Yes | Current estimated token count in context window |
108| `model` | string | No | LLM model identifier — a curated name (e.g., `claude-opus-4`, `gpt-4o`), a HuggingFace repo ID (e.g., `meta-llama/Llama-3.1-70B`), or any string (falls back to conservative defaults) |
109| `session_duration_minutes` | integer | No | How long this session has been running |
110| `tool_calls_count` | integer | No | Number of tool calls made in this session |
111| `context_summary` | string | No | Brief summary of current task and recent actions |
112| `agent_id` | string | No | Unique agent identifier for history tracking |
113 
114**Example response:**
115 
116```json
117{
118  "health_score": 62,
119  "status": "warning",
120  "token_utilization": {
121    "current": 155000,
122    "max_effective": 170000,
123    "percentage": 91.2,
124    "danger_zone_starts_at": 170000
125  },
126  "quality_estimate": {
127    "retrieval_accuracy": "degrading",
128    "middle_content_risk": "high",
129    "estimated_hallucination_risk": "moderate"
130  },
131  "session_fatigue": {
132    "tool_call_burden": "moderate",
133    "session_length_risk": "low",
134    "recommendation": "Consider breaking into sub-tasks if complexity increases."
135  },
136  "recommendations": [
137    {
138      "priority": "high",
139      "action": "compact_context",
140      "reason": "You are approaching the effective quality threshold. Summarize older context and remove completed task details.",
141      "estimated_quality_gain": 15
142    },
143    {
144      "priority": "high",
145      "action": "offload_to_memory",
146      "reason": "High risk of lost-in-the-middle effect. Store critical information to external memory before it is effectively lost.",
147      "estimated_quality_gain": 8
148    }
149  ]
150}
151```
152 
153### `get_health_history`
154 
155Retrieve health check history for a specific agent.
156 
157**Parameters:**
158 
159| Parameter | Type | Required | Description |
160|---|---|---|---|
161| `agent_id` | string | Yes | Unique agent identifier |
162| `limit` | integer | No | Max records to return (default: 20, max: 100) |
163 
164### `get_service_stats`
165 
166Get service-wide utilization statistics. No parameters required.
167 
168Returns total calls, unique agents, average health score, model distribution, status distribution, and recent activity (last hour / last 24h).
169 
170## Supported Models
171 
172| Model | Max Tokens | Danger Zone | Middle-Loss Risk |
173|---|---|---|---|
174| `claude-opus-4-5` | 200K | 175K | Low |
175| `claude-opus-4` | 200K | 170K | Low |
176| `claude-sonnet-4` | 200K | 165K | Low |
177| `claude-3.7-sonnet` | 200K | 160K | Low–Medium |
178| `claude-3.5-sonnet` | 200K | 152K | Medium |
179| `claude-haiku-3.5` | 200K | 130K | Medium |
180| `gpt-4.1` | 1M | 500K | Medium |
181| `gpt-4.1-mini` | 1M | 450K | Medium |
182| `gpt-4o` | 128K | 105K | Medium |
183| `gpt-4o-mini` | 128K | 95K | Medium–High |
184| `o3` | 200K | 160K | Low–Medium |
185| `o4-mini` | 200K | 150K | Medium |
186| `gemini-2.5-pro` | 1M | 600K | Medium |
187| `gemini-2.5-flash` | 1M | 520K | Medium–High |
188| `gemini-2.0-flash` | 1M | 500K | High |
189 
190### HuggingFace Auto-Resolution
191 
192Any model string containing `/` is treated as a HuggingFace repo ID. The server fetches `config.json` from the repo, extracts the context window size (`max_position_embeddings`, `n_positions`, or `max_seq_len`), and generates a conservative degradation profile:
193 
194- **65%** of max tokens → degradation onset
195- **80%** of max tokens → danger zone
196 
197Results are cached in SQLite — subsequent lookups are instant.
198 
199```
200model: "meta-llama/Llama-3.1-70B"       → 131K context, danger at 105K
201model: "mistralai/Mistral-7B-v0.1"      → 32K context, danger at 26K
202model: "mosaicml/mpt-7b"                → 65K context, danger at 52K
203```
204 
205If the fetch fails (network error, gated model, missing config), the server falls back silently to conservative defaults.
206 
207### Fallback
208 
209Any unrecognized model string without `/` falls back to conservative defaults (128K max, 100K danger zone).
210 
211## How It Works
212 
213The health score is a weighted composite of four signals:
214 
215| Signal | Weight | Source |
216|---|---|---|
217| **Token utilization quality** | 40% | Model-specific sigmoid degradation curve |
218| **Retrieval accuracy** | 25% | Base accuracy minus lost-in-the-middle penalty |
219| **Tool-call burden** | 20% | Compounding quality loss after 10+ tool calls |
220| **Session length** | 15% | Time-based fatigue heuristic |
221 
222The degradation curves are derived from empirical research:
223- [Chroma: Context Rot](https://research.trychroma.com/context-rot) — quality degrades around 147K–152K tokens on 200K models
224- [Stanford: Lost in the Middle](https://arxiv.org/abs/2307.03172) — retrieval accuracy drops for information in the middle of the context window
225- [Redis: Context Rot](https://redis.io/blog/context-rot/) — compounding degradation effects in long-running agents
226 
227## Development
228 
229```bash
230git clone https://github.com/milos-product-maker/context-rot-detection.git
231cd context-rot-detection
232npm install
233npm run dev        # Run with tsx (hot reload)
234npm test           # Run unit tests
235npm run build      # Compile TypeScript
236```
237 
238### Testing with MCP Inspector
239 
240```bash
241npx @modelcontextprotocol/inspector node dist/index.js
242```
243 
244## License
245 
246MIT
247