Context Rot Detection

Context Rot Detection

MCP service that gives AI agents self-awareness about their cognitive state.

Every long-running AI agent suffers from context rot — measurable performance degradation as the context window fills up. Research from Chroma, Stanford ("lost-in-the-middle"), and Redis confirms this is the #1 practical failure mode in production agent systems.

An agent experiencing context rot doesn't know it's degrading — it just starts making worse decisions. This tool gives agents real-time visibility into their own cognitive health.

Features

• Health score (0–100) based on token utilization, retrieval accuracy, and session fatigue
• Model-specific degradation curves for 15+ curated models (Claude, GPT, Gemini, o-series)
• Auto-resolves any HuggingFace model — pass a repo ID like meta-llama/Llama-3.1-70B and the context window is detected automatically, with results cached in SQLite
• Lost-in-the-middle risk scoring based on Stanford research
• Tool-call burden and session fatigue analysis
• Actionable recovery recommendations — compact context, offload to memory, checkpoint, break into subtasks
• Per-agent health history tracking (SQLite)
• Service-wide utilization statistics

Quick Start

npx (zero install)

npx context-rot-detection

npm (global install)

npm install -g context-rot-detection
context-rot-detection

MCP Client Configuration

Claude Code

Add to .mcp.json in your project root:

{
  "mcpServers": {
    "context-rot-detection": {
      "command": "npx",
      "args": ["-y", "context-rot-detection"],
      "env": {
        "HEALTH_HISTORY_DB": "./health.db"
      }
    }
  }
}

Claude Desktop

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "context-rot-detection": {
      "command": "npx",
      "args": ["-y", "context-rot-detection"],
      "env": {
        "HEALTH_HISTORY_DB": "/path/to/health.db"
      }
    }
  }
}

Docker

{
  "mcpServers": {
    "context-rot-detection": {
      "command": "docker",
      "args": [
        "run", "-i", "--rm",
        "-v", "context-rot-data:/data",
        "ghcr.io/milos-product-maker/context-rot-detection:latest"
      ]
    }
  }
}

Configuration

Tools

check_my_health

Analyze the current context window health. Call this periodically during long sessions or before critical decisions.

Parameters:

Example response:

{
  "health_score": 62,
  "status": "warning",
  "token_utilization": {
    "current": 155000,
    "max_effective": 170000,
    "percentage": 91.2,
    "danger_zone_starts_at": 170000
  },
  "quality_estimate": {
    "retrieval_accuracy": "degrading",
    "middle_content_risk": "high",
    "estimated_hallucination_risk": "moderate"
  },
  "session_fatigue": {
    "tool_call_burden": "moderate",
    "session_length_risk": "low",
    "recommendation": "Consider breaking into sub-tasks if complexity increases."
  },
  "recommendations": [
    {
      "priority": "high",
      "action": "compact_context",
      "reason": "You are approaching the effective quality threshold. Summarize older context and remove completed task details.",
      "estimated_quality_gain": 15
    },
    {
      "priority": "high",
      "action": "offload_to_memory",
      "reason": "High risk of lost-in-the-middle effect. Store critical information to external memory before it is effectively lost.",
      "estimated_quality_gain": 8
    }
  ]
}

get_health_history

Retrieve health check history for a specific agent.

Parameters:

get_service_stats

Get service-wide utilization statistics. No parameters required.

Returns total calls, unique agents, average health score, model distribution, status distribution, and recent activity (last hour / last 24h).

Supported Models

HuggingFace Auto-Resolution

Any model string containing / is treated as a HuggingFace repo ID. The server fetches config.json from the repo, extracts the context window size (max_position_embeddings, n_positions, or max_seq_len), and generates a conservative degradation profile:

• 65% of max tokens → degradation onset
• 80% of max tokens → danger zone

Results are cached in SQLite — subsequent lookups are instant.

model: "meta-llama/Llama-3.1-70B"       → 131K context, danger at 105K
model: "mistralai/Mistral-7B-v0.1"      → 32K context, danger at 26K
model: "mosaicml/mpt-7b"                → 65K context, danger at 52K

If the fetch fails (network error, gated model, missing config), the server falls back silently to conservative defaults.

Fallback

Any unrecognized model string without / falls back to conservative defaults (128K max, 100K danger zone).

How It Works

The health score is a weighted composite of four signals:

The degradation curves are derived from empirical research:

• Chroma: Context Rot — quality degrades around 147K–152K tokens on 200K models
• Stanford: Lost in the Middle — retrieval accuracy drops for information in the middle of the context window
• Redis: Context Rot — compounding degradation effects in long-running agents

Development

git clone https://github.com/milos-product-maker/context-rot-detection.git
cd context-rot-detection
npm install
npm run dev        # Run with tsx (hot reload)
npm test           # Run unit tests
npm run build      # Compile TypeScript

Testing with MCP Inspector

npx @modelcontextprotocol/inspector node dist/index.js

License

MIT