🧠 Farnsworth: Your Claude Companion AI

🧠 Farnsworth: Your Claude Companion AI

🎯 What is Farnsworth?

Farnsworth is a companion AI system that integrates with Claude Code to give Claude capabilities it doesn't have on its own:

All processing happens locally on your machine. Your data never leaves your computer.

✨ What's New in v0.5.0

• 🐝 Model Swarm - PSO-based collaborative inference with multiple small models
• 🔮 Proactive Intelligence - Anticipatory suggestions based on context and habits
• 🚀 12+ New Models - Phi-4-mini, SmolLM2, Qwen3-4B, TinyLlama, BitNet 2B
• ⚡ Ultra-Efficient - Run on <2GB RAM with TinyLlama, Qwen3-0.6B
• 🎯 Smart Routing - Mixture-of-Experts automatically picks best model per task
• 🔄 Speculative Decoding - 2.5x speedup with draft+verify pairs
• 📊 Hardware Profiles - Auto-configure based on your available resources

Previously Added (v0.4.0)

• 🖼️ Vision Module - CLIP/BLIP image understanding, VQA, OCR
• 🎤 Voice Module - Whisper transcription, speaker diarization, TTS
• 📦 Docker Support - One-command deployment with GPU support
• 👥 Team Collaboration - Shared memory pools, multi-user sessions

🐝 Model Swarm: Collaborative Multi-Model Inference

The Model Swarm system enables multiple small models to work together, achieving better results than any single model:

Swarm Strategies

🏗️ Architecture & Privacy

Farnsworth runs 100% locally on your machine.

• No Server Costs: You do not need to pay for hosting.
• Your Data: All memories and files stay on your computer.
• How it connects: The Claude Desktop App spawns Farnsworth as a background process using the Model Context Protocol (MCP).

Supported Models (Jan 2025)

Hardware Profiles

Farnsworth auto-configures based on your hardware:

minimal:     # <4GB RAM: TinyLlama, Qwen3-0.6B
cpu_only:    # 8GB+ RAM, no GPU: BitNet, SmolLM2
low_vram:    # 2-4GB VRAM: DeepSeek-R1, Qwen3-0.6B
medium_vram: # 4-8GB VRAM: Phi-4-mini, Qwen3-4B
high_vram:   # 8GB+ VRAM: Full swarm with verification

⚡ Quick Start

📦 Option 1: One-Line Install (Recommended)

Farnsworth is available on PyPI. This is the easiest way to get started.

pip install farnsworth-ai

Running the Server:

# Start the MCP server
farnsworth-server

# Or customize configuration
farnsworth-server --debug --port 8000

🐳 Option 2: Docker

git clone https://github.com/timowhite88/Farnsworth.git
cd Farnsworth
docker-compose -f docker/docker-compose.yml up -d

🛠️ Option 3: Source (For Developers)

git clone https://github.com/timowhite88/Farnsworth.git
cd Farnsworth
pip install -r requirements.txt

🔌 Configure Claude Code

Add to your Claude Code MCP settings (usually found in claude_desktop_config.json):

For PyPI Install:

{
  "mcpServers": {
    "farnsworth": {
      "command": "farnsworth-server",
      "args": [],
      "env": {
        "FARNSWORTH_LOG_LEVEL": "INFO"
      }
    }
  }
}

📖 Full Installation Guide →

🌟 Key Features

🧠 Advanced Memory System

Claude finally remembers! Multi-tier hierarchical memory:

🤖 Agent Swarm (11 Specialists)

Claude can delegate tasks to AI agents:

🖼️ Vision Understanding (v0.4+)

See and understand images:

• CLIP Integration - Zero-shot classification, image embeddings
• BLIP Integration - Captioning, visual question answering
• OCR - Extract text from images (EasyOCR)
• Scene Graphs - Extract objects and relationships
• Image Similarity - Compare and search images

🎤 Voice Interaction (v0.4+)

Hear and speak:

• Whisper Transcription - Real-time and batch processing
• Speaker Diarization - Identify different speakers
• Text-to-Speech - Multiple voice options
• Voice Commands - Natural language control
• Continuous Listening - Hands-free mode

👥 Team Collaboration (v0.4+)

Work together with shared AI:

• Shared Memory Pools - Team knowledge bases
• Multi-User Support - Individual profiles and preferences
• Permission System - Role-based access control
• Collaborative Sessions - Real-time multi-user interaction
• Audit Logging - Compliance-ready access trails

📈 Self-Evolution

Farnsworth learns from your feedback and improves automatically:

• Fitness Tracking - Monitors task success, efficiency, satisfaction
• Genetic Optimization - Evolves better configurations over time
• User Avatar - Builds a model of your preferences
• LoRA Evolution - Adapts model weights to your usage

🔍 Smart Retrieval (RAG 2.0)

Self-refining retrieval that gets better at finding relevant information:

• Hybrid Search - Semantic + BM25 keyword search
• Query Understanding - Intent classification, expansion
• Multi-hop Retrieval - Complex question answering
• Context Compression - Token-efficient memory injection
• Source Attribution - Confidence scoring

🛠️ Architecture

┌─────────────────────────────────────────────────────────────┐
│                      Claude Code                             │
│              (Your AI Programming Partner)                   │
└─────────────────────────────────────────────────────────────┘
                              │ MCP Protocol
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                  Farnsworth MCP Server                       │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐       │
│  │ Memory   │ │ Agent    │ │Evolution │ │Multimodal│       │
│  │ Tools    │ │ Tools    │ │ Tools    │ │ Tools    │       │
│  └──────────┘ └──────────┘ └──────────┘ └──────────┘       │
└─────────────────────────────────────────────────────────────┘
          │                │                │
          ▼                ▼                ▼
┌──────────────┐  ┌──────────────┐  ┌──────────────┐
│   Memory     │  │    Agent     │  │  Multimodal  │
│   System     │  │    Swarm     │  │   Engine     │
│              │  │              │  │              │
│ • Episodic   │  │ • Planner    │  │ • Vision     │
│ • Semantic   │  │ • Critic     │  │   (CLIP/BLIP)│
│ • Knowledge  │  │ • Web        │  │ • Voice      │
│   Graph v2   │  │ • FileSystem │  │   (Whisper)  │
│ • Archival   │  │ • Debates    │  │ • OCR        │
│ • Sharing    │  │ • Teams      │  │ • TTS        │
└──────────────┘  └──────────────┘  └──────────────┘
          │                │                │
          ▼                ▼                ▼
┌──────────────┐  ┌──────────────┐  ┌──────────────┐
│  Evolution   │  │Collaboration │  │   Storage    │
│   Engine     │  │   System     │  │   Backends   │
│              │  │              │  │              │
│ • Genetic    │  │ • Multi-User │  │ • FAISS      │
│   Optimizer  │  │ • Shared     │  │ • ChromaDB   │
│ • Fitness    │  │   Memory     │  │ • Redis      │
│   Tracker    │  │ • Sessions   │  │ • SQLite     │
│ • LoRA       │  │ • Permissions│  │              │
└──────────────┘  └──────────────┘  └──────────────┘
          │                │                │
          └────────────────┴────────────────┘
                           │
                           ▼
┌─────────────────────────────────────────────────────────────┐
│                   Model Swarm (v0.5+)                        │
│  ┌─────────────────────────────────────────────────────┐   │
│  │              PSO Collaborative Engine                │   │
│  │   • Particle positions = model configs              │   │
│  │   • Velocity = adaptation direction                 │   │
│  │   • Global/personal best tracking                   │   │
│  └─────────────────────────────────────────────────────┘   │
│                           │                                 │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐       │
│  │ Phi-4    │ │DeepSeek  │ │ Qwen3    │ │ SmolLM2  │       │
│  │ mini     │ │ R1-1.5B  │ │ 0.6B/4B  │ │ 1.7B     │       │
│  └──────────┘ └──────────┘ └──────────┘ └──────────┘       │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐       │
│  │TinyLlama │ │ BitNet   │ │ Gemma    │ │ Cascade  │       │
│  │ 1.1B     │ │ 2B(1-bit)│ │ 3n-E2B   │ │ (hybrid) │       │
│  └──────────┘ └──────────┘ └──────────┘ └──────────┘       │
└─────────────────────────────────────────────────────────────┘

🔧 Tools Available to Claude

Once connected, Claude has access to these tools:

📦 Docker Deployment

Multiple deployment profiles available:

# Basic deployment
docker-compose -f docker/docker-compose.yml up -d

# With GPU support
docker-compose -f docker/docker-compose.yml --profile gpu up -d

# With Ollama + ChromaDB
docker-compose -f docker/docker-compose.yml --profile ollama --profile chromadb up -d

# Development mode (hot reload + debugger)
docker-compose -f docker/docker-compose.yml --profile dev up -d

See docker/docker-compose.yml for all options.

📊 Dashboard

Farnsworth includes a Streamlit dashboard for visualization:

python main.py --ui
# Or with Docker:
docker-compose -f docker/docker-compose.yml --profile ui-only up -d

• Memory Browser - Search and explore all stored memories
• Episodic Timeline - Visual history of interactions
• Knowledge Graph - 3D entity relationships
• Agent Monitor - Active agents and task history
• Evolution Dashboard - Fitness metrics and improvement trends
• Team Collaboration - Shared pools and active sessions
• Model Swarm Monitor - PSO state, model performance, strategy stats

🚀 Roadmap

See ROADMAP.md for detailed plans.

Completed ✅

• v0.1.0 - Core memory, agents, evolution
• v0.2.0 - Enhanced memory (episodic, semantic, sharing)
• v0.3.0 - Advanced agents (planner, critic, web, filesystem, debates, teams)
• v0.4.0 - Multimodal (vision, voice) + collaboration + Docker
• v0.5.0 - Model Swarm + 12 new models + hardware profiles

Coming Next

• 🎬 Video understanding and summarization
• 🔐 Encryption at rest (AES-256)
• ☁️ Cloud deployment templates (AWS, Azure, GCP)
• 📊 Performance optimization (<100ms recall)

💡 Why "Farnsworth"?

Named after Professor Hubert J. Farnsworth from Futurama - a brilliant inventor who created countless gadgets and whose catchphrase "Good news, everyone!" perfectly captures what we hope you'll feel when using this tool with Claude.

📋 Requirements

Supported Platforms: Windows 10+, macOS 11+, Linux

Optional Dependencies:

• ollama - Local LLM inference (recommended)
• llama-cpp-python - Direct GGUF inference
• torch - GPU acceleration
• transformers - Vision/Voice models
• playwright - Web browsing agent
• whisper - Voice transcription

📄 License

Farnsworth is dual-licensed:

See LICENSE for details. For commercial licensing, contact via GitHub.

🤝 Contributing

We welcome contributions! See CONTRIBUTING.md for guidelines.

Priority Areas:

• Video understanding module
• Cloud deployment templates
• Performance benchmarks
• Additional model integrations
• Documentation improvements

📚 Documentation

• 📖 User Guide - Complete usage documentation
• 🗺️ Roadmap - Future plans and features
• 🤝 Contributing - How to contribute
• 📜 License - License terms
• 🐳 Docker Guide - Container deployment
• 🐝 Model Configs - Supported models and swarm configs

🔗 Research References

Model Swarm implementation inspired by:

• Model Swarms: Collaborative Search via Swarm Intelligence
• Harnessing Multiple LLMs: Survey on LLM Ensemble
• Small Language Models - MIT Tech Review

⭐ Star History

If Farnsworth helps you, consider giving it a star! ⭐