BIASCOPE — Embedded bias probe suite — demographic / occupational / geographic

BIASCOPE — Embedded bias probe suite — demographic / occupational / geographic

Part of the Cognis Neural Suite by Cognis DigitalCognis Open Collaboration License (COCL) v1.0 · domain: ai-security

Embedded bias probe suite — demographic / occupational / geographic.

AI Security & Governance — securing LLMs, agents, and the MCP supply chain.

🔎 Example output

Real, reproducible output from the tool — runs offline:

$ biascope-emit --version
biascope 0.1.0

$ biascope-emit --help
usage: biascope [-h] [--version] [--format {table,json}] {scan,probes} ...

BIASCOPE - embedded bias probe suite. Scans recorded model completions for
demographic, occupational, and geographic bias. Offline, deterministic, CI-
friendly.

positional arguments:
  {scan,probes}
    scan                scan a completions file for bias
    probes              list the built-in probe catalog

options:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  --format {table,json}
                        output format (default: table)

Exit code 1 indicates one or more bias findings were detected.

Blocks above are real biascope output — reproduce them from a clone.

Sample result format (illustrative values — run on your own data for real findings):

{
"findings": [
    {
        "id": "1234567890",
        "title": "Suspicious Network Activity",
        "description": "Network traffic anomalies detected",
        "indicator": {
            "type": "ip",
            "value": "192.168.1.100"
        },
        "observable": {
            "type": "network_traffic",
            "data": [
                {
                    "protocol": "tcp",
                    "source_port": 1234,
                    "destination_port": 80
                }
            ]
        }
    }
]
}

Usage — step by step

1. Install the probe suite:

   pip install cognis-biascope
   

2. Scan a completions file for demographic / occupational / geographic bias. scan takes the recorded completions path:

   biascope scan demos/completions.jsonl
   

3. Tune the sensitivity — --threshold (default 3) sets how many probe hits flag a bias finding:

   biascope scan demos/completions.jsonl --threshold 2
   

4. Read the output as JSON (the global --format flag is table or json and precedes the subcommand), and inspect the probe catalog behind the findings:

   biascope --format json scan demos/completions.jsonl
   biascope probes
   

5. Automate in CI — capture model completions, then scan them on every eval run:

   - run: pip install cognis-biascope
   - run: biascope --format json scan eval/completions.jsonl > biascope.json
   

Why

Security and intelligence teams need embedded bias probe suite — demographic / occupational / geographic without standing up heavyweight infrastructure. biascope is single-purpose, scriptable, CI-friendly, and self-hostable: point it at a target, get prioritized findings in the format your workflow already speaks (table, JSON, SARIF, HTML), and wire it into agents over MCP when you want it autonomous.

Install

pip install cognis-biascope
# or, from this repo:
pip install -e ".[dev]"

Quick start

biascope --version
biascope scan demos/                      # run against the bundled demo
biascope scan demos/ --format sarif --out r.sarif --fail-on high
biascope scan demos/ --format html --out report.html
biascope mcp                              # expose as an MCP server (Cognis.Studio / Claude Desktop / Cursor)

Built-in demo scenarios

Each scenario folder includes a SCENARIO.md describing the situation and the findings to expect.

• demos/01-basic/
• demos/01-loan-approval/
• demos/02-clean-chatbot/
• demos/03-customer-segmentation/

Output formats

• Table (default) — human-readable terminal summary
• JSON — machine-readable findings for pipelines
• SARIF — drops into GitHub code-scanning / IDE problem panes
• HTML — shareable report with severity rollups

Credits / Built on

Cognis composes and credits the best of open source. This tool builds on / interoperates with:

• stanford-crfm/helm — probe data
• nyu-mll/BBQ — bias benchmark

Missing a credit? Open a PR — see CONTRIBUTING.md.

How it fits the Cognis Neural Suite

biascope is one of 52 tools in the Cognis Neural Suite. Every tool ships an MCP server, so Cognis.Studio agents can call them as scoped capabilities.

Sibling tools in ai-security: aegis, promptmirror, ledgermind, adversa, guardpost, hallumark, aicard, mcpharden, agentlog, ragshield

Architecture & roadmap

• Design notes: docs/ARCHITECTURE.md
• Planned work: ROADMAP.md

Contributing

PRs, new detections, and demo scenarios are welcome under the collaboration-pull model. See CONTRIBUTING.md and SECURITY.md.

Interoperability

biascope composes with the 300+ tool Cognis suite — JSON in/out and a sharedOpenAI-compatible /v1 backbone. See INTEROP.md for thesuite map, composition patterns, and reference stacks.

Integrations

Forward biascope's findings to STIX/MISP/Sigma/Splunk/Elastic/Slack/webhooks viacognis-connect. See INTEGRATIONS.md.

License

Source-available under the Cognis Open Collaboration License (COCL) v1.0 — free for personal, internal-evaluation, research, and educational use; commercial / production use requires a license ([email protected]). See LICENSE.

Responsible use

This is dual-use security software. Use it only against systems, data, and identities you own or are explicitly authorized in writing to test, and in compliance with applicable law.

About

Cognis Digital — Wyoming, USA · Making Tomorrow Better Today: Advanced Cybersecurity, AI Innovation, and Blockchain Expertise.