Cortex

Cortex

An MCP-native agentic platform: planner/executor/critic over hybrid RAG, with the budget, safety and observability layers a production deployment needs.

241 tests · 82% coverage, gate-enforced · runs offline against fakes · no service required to develop

Cortex combines MCP, multi-agent orchestration, hybrid RAG, three-tier memory, LLMOps, automated evaluation and safety guardrails in one codebase. What it is not is a system that has been run at scale: it has never served production traffic, and the load-testing and scale-out work is listed in LIMITATIONS.md rather than implied here.

Why Cortex

Most AI projects are tutorials. Cortex is built the way a senior AI engineer would build it for production:

• MCP-native — all capabilities exposed as Model Context Protocol tools. Connect Claude Desktop, VS Code, or any MCP client to your running Cortex server in minutes.
• Multi-LLM via LiteLLM — swap between OpenAI, Anthropic, Azure, Bedrock, or Vertex by changing one config value.
• Full observability — every LLM call, agent step, and tool invocation is an OTEL span, visible in Arize Phoenix and Grafana.
• Automated evaluation — Ragas metrics with a regression suite on a 6-hourly schedule. Faithfulness is a tracked metric with a threshold, not a hope.
• Budget enforcement — a per-run cost ceiling checked before every model call, against a Redis ledger shared across async tasks and workers.
• Rate limiting that runs before routing — so requests that 404 or 422 are metered too. Those are cheaper for an attacker to generate than valid ones.

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                         MCP Clients                                  │
│         Claude Desktop │ VS Code │ Custom Agents │ REST API          │
└─────────────────┬───────────────────────────────────────────────────┘
                  │ MCP / HTTP
┌─────────────────▼───────────────────────────────────────────────────┐
│                    Cortex API Gateway (FastAPI)                        │
│              Auth │ Rate Limiting │ SSE Streaming │ /metrics          │
└─────────────────┬───────────────────────────────────────────────────┘
                  │
┌─────────────────▼───────────────────────────────────────────────────┐
│                 LangGraph Agent Orchestration                         │
│        load_memory → planner → executor → critic → save_memory       │
└──────┬──────────┬──────────────┬──────────────┬───────────────────-─┘
       │          │              │              │
  ┌────▼───┐ ┌───▼────┐ ┌──────▼────┐ ┌──────▼──────┐
  │  MCP   │ │  RAG   │ │  Memory  │ │   Safety    │
  │ Server │ │Pipeline│ │  3-Tier  │ │ Guardrails  │
  └────┬───┘ └───┬────┘ └──────┬────┘ └─────────────┘
       │          │              │
┌──────▼──────────▼──────────────▼──────────────────────────────────-─┐
│                      LiteLLM Router                                   │
│         OpenAI │ Anthropic │ Azure │ Bedrock │ Vertex │ Ollama        │
└───────────────────────────────────────────────────────────────────-──┘
       │          │              │
  ┌────▼──┐  ┌───▼───┐  ┌──────▼──────────────────────────────────┐
  │Qdrant │  │ Redis │  │  Observability: OTEL + Phoenix + Grafana │
  └───────┘  └───────┘  └─────────────────────────────────────────┘

Quickstart (5 minutes)

Prerequisites

• Python 3.11+
• Docker + Docker Compose
• At least one LLM API key (OpenAI recommended for quickstart)

1. Clone and configure

git clone https://github.com/your-org/cortex
cd cortex
cp .env.example .env
# Edit .env — set OPENAI_API_KEY and SECRET_KEY at minimum

2. Start all services

docker compose up -d

Services will be available at:| Service | URL ||---------|-----|| Cortex API | http://localhost:8000/docs || Cortex MCP | stdio (for Claude Desktop) || Arize Phoenix | http://localhost:6006 || Grafana | http://localhost:3000 (admin / cortex) || Qdrant UI | http://localhost:6333/dashboard || Prometheus | http://localhost:9090 |

3. Ingest a document

curl -X POST http://localhost:8000/api/v1/ingest \
  -H "Authorization: Bearer $(python scripts/gen_token.py)" \
  -H "Content-Type: application/json" \
  -d '{"text": "Your document content here", "metadata": {"source": "quickstart"}}'

4. Run an agent

curl -X POST http://localhost:8000/api/v1/runs \
  -H "Authorization: Bearer $(python scripts/gen_token.py)" \
  -H "Content-Type: application/json" \
  -d '{"goal": "Summarise the key points from the ingested documents"}'

5. Connect Claude Desktop

Add to your Claude Desktop claude_desktop_config.json:

{
  "mcpServers": {
    "cortex": {
      "command": "python",
      "args": ["-m", "cortex.mcp.server"],
      "cwd": "/path/to/cortex"
    }
  }
}

Project Structure

cortex/
├── src/cortex/
│   ├── config.py              # Pydantic Settings — all configuration
│   ├── exceptions.py          # Domain exception hierarchy
│   ├── logging_config.py      # Structlog structured logging
│   ├── mcp/
│   │   ├── server.py          # FastMCP server — MCP tool definitions
│   │   └── client.py          # MCP client for agent tool calls
│   ├── llm/
│   │   ├── router.py          # LiteLLM wrapper with retry, fallback, budget
│   │   ├── cost_tracker.py    # Per-run token cost accounting (Redis)
│   │   └── cache.py           # Semantic cache (Qdrant)
│   ├── graph/
│   │   ├── state.py           # LangGraph state schema (CortexState)
│   │   └── cortex_graph.py     # Graph topology, nodes, routing functions
│   ├── agents/
│   │   ├── planner.py         # Goal → task list decomposition
│   │   ├── executor.py        # Task execution via MCP tools
│   │   ├── critic.py          # Output quality evaluation
│   │   └── memory_agent.py    # Memory retrieval and consolidation
│   ├── rag/
│   │   └── pipeline.py        # Ingest, chunk, embed, hybrid search, rerank
│   ├── memory/                # Three-tier memory (working / episodic / semantic)
│   ├── safety/
│   │   └── middleware.py      # Guardrails, PII scanner, injection detector
│   ├── obs/
│   │   └── metrics.py         # Prometheus metrics + OTEL + Phoenix setup
│   ├── eval/
│   │   └── ragas_runner.py    # Automated Ragas evaluation suite
│   └── api/
│       ├── main.py            # FastAPI app — all HTTP endpoints
│       └── auth.py            # JWT authentication
├── config/
│   └── rails/                 # NeMo Guardrails Colang policies
├── obs/
│   ├── prometheus.yml         # Prometheus scrape config
│   └── grafana/               # Grafana dashboard JSON + provisioning
├── tests/                     # Pytest test suite (80%+ coverage required)
├── docs/adr/                  # Architecture Decision Records
├── docker-compose.yml         # Full local stack
├── Dockerfile                 # Multi-stage production image
├── pyproject.toml             # Dependencies and tooling config
└── .env.example               # All required environment variables

Documentation

Running Tests

pip install -e ".[dev]"
pytest                    # Full suite with coverage
pytest tests/test_rag/    # RAG tests only
pytest -k "test_planner"  # Single test

Coverage target: 80% minimum (enforced in CI).

Contributing

See CONTRIBUTING.md. All PRs require:

• Tests for new code
• Updated ADR if architecture changes
• Eval regression suite must still pass (pytest tests/eval/)

What was wrong with this codebase, and how it was found

Cortex was generated complete — 77 files, eight layers, an 80% coveragegate — and had never been executed. Not once. Every defect below was foundby making it run.

The reason this table is here rather than quietly fixed: the gap between"generated" and "working" is where all the engineering lives, and every rowis a question I can answer in depth.