FTR Partner Self-Assessment MCP Server (@aws/ftr-eval-mcp)

FTR Partner Self-Assessment MCP Server (@aws/ftr-eval-mcp)

An MCP server and interactive CLI that automates the AWS Foundational Technical Review (FTR) partner self-assessment process. It evaluates partner-submitted compliance documents (SOC 2 Type II reports and WAFR reports) against defined controls and returns structured PASS/FAIL decisions with reasoning.

Architecture

The system connects partner-submitted PDF reports to Amazon Bedrock for LLM-powered evaluation. A Model Context Protocol (MCP) server exposes evaluation tools to AI assistants, while a standalone CLI provides a guided terminal workflow. Both paths share a common evaluation engine backed by Bedrock, SOC 2 and WAFR control registries, and calibration guides that shape scoring decisions.

Overview

Partners seeking AWS validation must submit evidence for two distinct review tracks:

This package provides three ways to evaluate submissions:

1. MCP Server — Exposes evaluation tools to AI assistants (Kiro, Claude, etc.) via the Model Context Protocol
2. Interactive CLI — A terminal-based evaluation workflow with guided prompts, progress spinners, and color-coded results
3. Kiro Power — A native Kiro IDE integration that loads calibration criteria directly into chat, no server or CLI required

Installation

Choose the option that fits your environment.

Option 1 — npm (requires Node.js >= 18)

npm install -g @aws/ftr-eval-mcp

Once installed, the ftr-eval-mcp command is available globally:

ftr-eval-mcp evaluate
ftr-eval-mcp serve

Option 2 — Standalone Binary (no Node.js required)

Download the binary for your platform from the GitHub Releases page:

macOS / Linux — make the binary executable and run it:

chmod +x ftr-eval-mcp-macos-arm64
./ftr-eval-mcp-macos-arm64 evaluate

Windows — run it directly:

ftr-eval-mcp-win-x64.exe evaluate

Prerequisites

• AWS credentials configured (for Bedrock access)
• Node.js >= 18.0.0 (npm install path only — not required for standalone binaries)

Configuration

The server uses sensible defaults out of the box. Override via environment variables or CLI flags:

Resolution order (highest priority first): CLI flags → Environment variables → Defaults

Example with a custom region:

FTR_AWS_REGION=us-west-2 ftr-eval-mcp

Or in your MCP config:

{
  "mcpServers": {
    "ftr-eval-mcp": {
      "command": "node",
      "args": ["dist/server.js", "serve"],
      "env": {
        "FTR_AWS_REGION": "eu-west-1"
      }
    }
  }
}

MCP Config Levels

You can register this MCP server at different levels depending on your needs:

Precedence: Workspace config overrides user config. If the same server is defined at both levels, the workspace-level definition wins when that project is open. Outside that workspace, the user-level config applies.

When to use each level:

• Workspace — Best when developing or testing the server locally. The config lives with the project and won't affect other workspaces.
• User — Best when the server is stable and you want it available everywhere without per-project setup.

Example workspace config (.kiro/settings/mcp.json):

{
  "mcpServers": {
    "ftr-eval-mcp": {
      "command": "node",
      "args": ["/path/to/dist/server.js", "serve"],
      "disabled": false,
      "autoApprove": ["get_prompt_template", "parse_pdf", "evaluate_submission"]
    }
  }
}

The autoApprove array lists tool names that the AI assistant can invoke without prompting for confirmation. Tools not in this list require manual approval before each execution.

Usage

MCP Server Mode (default)

Start the MCP server for use with AI assistants:

ftr-eval-mcp

With options:

ftr-eval-mcp serve --transport stdio --region us-east-1 --model <bedrock-model-id>

Interactive CLI Mode

Launch the guided evaluation workflow:

ftr-eval-mcp evaluate

This will prompt you to:

1. Select a report type (SOC 2 or WAFR)
2. Enter the path to your PDF report
3. Choose a specific control or evaluate all

Non-Interactive Mode

For scripting and CI/CD pipelines:

ftr-eval-mcp evaluate --report-type wafr --file ./path/to/report.pdf

Evaluate a single control:

ftr-eval-mcp evaluate --report-type soc2 --file ./report.pdf --control-id SOC-001

CLI Options

ftr-eval-mcp evaluate --help

Options:
  --report-type <type>   Report type: soc2 or wafr
  --file <path>          Path to the PDF report file
  --control-id <id>      Specific control ID to evaluate (optional)
  --region <region>      AWS region (default: us-east-1)
  --model <modelId>      Bedrock model ID

MCP Tools

When running as an MCP server, the following tools are exposed:

Development

Build

npm run build

Run Tests

npm test

Run Locally (without installing globally)

npm run build
node dist/server.js evaluate

Build Standalone Binaries

npm run build:binaries

This produces platform-specific executables in binaries/ for macOS (ARM/x64), Linux (x64/ARM), and Windows (x64).

Project Structure

src/
├── server.ts                  # Entry point: commander routing (serve/evaluate)
├── cli.ts                     # CLI orchestrator (evaluation workflow)
├── cli/
│   ├── input-collector.ts     # Interactive prompts and flag validation
│   ├── credential-validator.ts # AWS credential check via STS
│   ├── progress-reporter.ts   # Spinner and progress display (ora)
│   └── results-formatter.ts   # Color-coded results output (chalk)
├── config.ts                  # Configuration resolution
├── engine/
│   ├── evaluation-engine.ts   # Core evaluation orchestration
│   ├── bedrock-client.ts      # Amazon Bedrock API client
│   ├── prompt-builder.ts      # LLM prompt construction
│   └── decision-parser.ts     # Parse LLM responses into decisions
├── parsers/
│   └── pdf-parser.ts          # PDF text extraction
├── registries/
│   ├── control-registry.ts    # Control definitions
│   └── calibration-guide-registry.ts # Calibration guides
├── tools/                     # MCP tool registrations
│   ├── evaluate-submission.ts
│   ├── get-calibration-guide.ts
│   ├── get-controls.ts
│   ├── get-prompt-template.ts
│   └── parse-pdf.ts
├── types.ts                   # Shared TypeScript types
└── assets/
    ├── calibration-guides/    # SOC 2 and WAFR calibration guides
    ├── controls/              # Control definition files
    └── prompts/               # LLM prompt templates

Kiro Power Tool

This project also includes a Kiro power at .kiro/powers/ftr-self-assessment/ for direct use within the Kiro IDE. The steering files load automatically and give Kiro full calibration criteria to evaluate FTR submissions in chat.

Controls Reference

SOC 2 Controls

WAFR Controls

Key Rules

• Expired reports (SOC 2 or WAFR older than 12 months) always FAIL
• SOC 2 Type I does not qualify; must be Type II
• Submitting a WAFR report for a SOC 2 control (or vice versa) FAILS immediately
• Only active (open/unresolved) HRIs cause failure — resolved HRIs are ignored
• Medium-Risk Issues (MRIs) never cause failure regardless of count or status

Exit Codes

Contributing

See CONTRIBUTING for guidelines on bug reports, pull requests, and the code of conduct.

Security

See CONTRIBUTING for information on reporting security issues.

License

This library is licensed under the MIT-0 License. See the LICENSE file.