🤖 Chatbot System

A sophisticated, context-aware chatbot system with dynamic tool-based architecture, multi-platform integration, and advanced conversation state management. Built for intelligent conversations across Matrix and Farcaster platforms with comprehensive AI-driven decision making.

📋 Table of Contents

• 🌟 Key Features
• 🏗️ Architecture Overview
• 🚀 Quick Start
• 💻 Local Development
• 🌐 World State Management
• 🔧 Configuration
• 🚀 Deployment
• 🧪 Testing
• 📖 Documentation
• 🤝 Contributing
• 📄 License

🌟 Key Features

• 🔧 Dynamic Tool Architecture: Extensible tool system with runtime registration and AI integration
• 🧠 Context-Aware Conversations: Maintains evolving world state across conversations with advanced deduplication
• 🌐 Multi-Platform Integration: Support for Matrix and Farcaster with standardized tool interfaces
• 👁️ AI Conversation Continuity: Bot tracks its own messages for improved conversation flow
• 💾 Persistent State Management: Robust storage of conversation context and world state
• 🤖 AI-Powered Decision Making: Intelligent response generation with dynamic tool awareness
• 📊 Advanced Rate Limiting: Smart rate limiting with backoff and quota management
• 🔄 Thread Management: Intelligent conversation thread tracking and context preservation
• 📧 Matrix Room Management: Auto-join functionality with invite handling
• 📱 Enhanced User Profiling: Rich user metadata tracking for social platforms

🏗️ Architecture Overview

The system has been architected with a dynamic tool-based design for maximum extensibility and maintainability. The architecture follows a layered approach with clear separation of concerns.

🔧 Core Components

ToolRegistry & Tool System

• ToolRegistry: Manages dynamic tool registration and provides AI-ready descriptions
• ToolInterface: Abstract base class for all tools with standardized execution patterns
• ActionContext: Comprehensive dependency injection for tools (observers, configurations, state managers)

World State Management

• WorldStateManager: Central state coordinator with advanced message deduplication
• Message & Channel Models: Rich data models supporting multi-platform message metadata
• Thread Management: Intelligent conversation threading for platforms like Farcaster
• Rate Limiting: Built-in rate limit tracking and enforcement

AI Integration & Orchestration

• ContextAwareOrchestrator: Main coordinator using the tool system with intelligent cycle management
• AIDecisionEngine: Updated to receive dynamic tool descriptions and optimized payloads
• Context Manager: Advanced conversation context preservation and retrieval

Platform Integrations

• Matrix Integration: Full Matrix protocol support with room management and invite handling
• Farcaster Integration: Complete Farcaster API integration with enhanced user profiling
• Standardized Interfaces: Unified message and action handling across platforms

🛠️ Tool System

All platform interactions are handled through standardized tools with consistent interfaces:

Core Tools

• WaitTool - Intelligent observation and waiting actions with configurable intervals
• ObserveTool - Advanced world state observation with filtering and summarization

Matrix Tools

• SendMatrixReplyTool - Matrix reply functionality with thread context awareness
• SendMatrixMessageTool - Matrix message sending with formatting support
• JoinMatrixRoomTool - Automated room joining with invite acceptance

Farcaster Tools

• SendFarcasterPostTool - Farcaster posting with media support and rate limiting
• SendFarcasterReplyTool - Farcaster replying with thread context preservation
• LikeFarcasterPostTool - Social engagement actions with deduplication
• QuoteFarcasterPostTool - Quote casting with content attribution
• FollowFarcasterUserTool - User following functionality
• DeleteFarcasterPostTool - Delete your own Farcaster posts/casts
• DeleteFarcasterReactionTool - Remove likes/recasts from posts
• SendFarcasterDirectMessageTool - Private messaging capabilities

🎯 Key Benefits

• 🔄 Extensibility: Add new tools by implementing ToolInterface and registering
• 🧹 Maintainability: Platform logic isolated in dedicated tool classes with clear boundaries
• 🧪 Testability: Clean dependency injection via ActionContext enables comprehensive testing
• 🤖 AI Integration: Tool descriptions automatically update AI capabilities and decision-making
• 📏 Consistency: Standardized parameter schemas and error handling across all tools
• ⚡ Performance: Optimized payload generation and intelligent message filtering
• 🔒 Reliability: Robust error handling, rate limiting, and state consistency

🚀 Quick Start

🐳 Docker Deployment (Recommended)

The fastest way to get started is using Docker:

# 1. Clone the repository
git clone <repository-url>
cd python3-poetry-pyenv

# 2. Setup environment
cp .env.example .env
nano .env  # Fill in your API keys and credentials

# 3. Deploy with Docker
./scripts/deploy.sh

# 4. Monitor logs
docker-compose logs -f chatbot

🛠️ Development Setup

For local development and testing:

# 1. Install Poetry (if not already installed)
curl -sSL https://install.python-poetry.org | python3 -

# 2. Install dependencies
poetry install

# 3. Configure environment
cp .env.example .env
nano .env  # Add your credentials

# 4. Run the system
poetry run python -m chatbot.main

🎮 Available Tasks

Use VS Code tasks or run them directly:

# Run the main chatbot
poetry run python -m chatbot.main

# Run with management UI
poetry run python -m chatbot.main_with_ui

# Run control panel
poetry run python control_panel.py

# Run tests
poetry run pytest tests/ -v

# Format code
poetry run black chatbot/ && poetry run isort chatbot/

# Lint code
poetry run flake8 chatbot/ && poetry run mypy chatbot/

💻 Local Development

🏠 Migrating from GitHub Codespaces

If you're currently using GitHub Codespaces and want to migrate to local development for better performance and no resource limits:

Quick Migration

# 1. Clone to your local machine
git clone <your-repo-url>
cd matrixbot

# 2. Run the migration script
./migrate_to_local.sh

# 3. Open in VS Code and reopen in container
code .
# When prompted: "Reopen in Container"

Manual Setup

1. Install Prerequisites:

   • Docker Desktop
   • Visual Studio Code
   • Dev Containers extension

2. Setup Environment:

   cp .env.example .env
   # Edit .env with your actual values

3. Open in Dev Container:

   • Open project in VS Code
   • Press Ctrl+Shift+P (or Cmd+Shift+P on macOS)
   • Select "Dev Containers: Reopen in Container"

🔧 Local Development Benefits

• Performance: Faster file I/O and build times
• Resources: Use your full machine resources
• Persistence: Data persists between sessions
• Offline: Work without internet connection
• Debugging: Better debugging experience

📂 Development Workflow

# Start all services
docker-compose up -d

# Run the chatbot in development
poetry run python run.py

# Run with UI
poetry run python chatbot/main_with_ui.py

# Run tests
poetry run pytest

# View logs
docker-compose logs -f chatbot_backend

For detailed local development setup, see LOCAL_DEVELOPMENT.md.

🌐 World State Management

The system implements a sophisticated world state management approach that maintains comprehensive awareness of all platform activities and conversations.

📊 State Components

Message Management

• Multi-Platform Messages: Unified message model supporting Matrix and Farcaster with platform-specific metadata
• Rich User Profiles: Enhanced user information including follower counts, bios, profile pictures, and verification badges
• Deduplication: Advanced message deduplication across channels and platforms to prevent processing duplicates
• Thread Tracking: Intelligent conversation thread management for platforms supporting threaded discussions

Channel & Room Tracking

• Dynamic Channel Creation: Automatic channel discovery and registration as the bot encounters new rooms/feeds
• Activity Summarization: Real-time activity summaries with user engagement metrics and timestamp ranges
• Matrix Room Metadata: Complete room information including topics, member counts, power levels, and encryption status
• Invite Management: Pending Matrix room invites with automated acceptance workflows

Action History

• Comprehensive Logging: Complete audit trail of all bot actions with parameters and results
• Action Deduplication: Prevents duplicate actions (likes, replies, follows) with intelligent tracking
• Scheduled Action Updates: Support for updating scheduled/pending actions with final results
• Rate Limit Integration: Action history informs rate limiting decisions and backoff strategies

🔄 State Optimization

AI Payload Optimization

• Primary Channel Focus: Detailed information for the active conversation channel
• Smart Summarization: Intelligent summarization of secondary channels to reduce token usage
• User Context Filtering: Bot's own messages are filtered out to focus on external interactions
• Configurable Truncation: Adjustable limits for messages, actions, and thread history based on AI model constraints

Performance Features

• Efficient Updates: Incremental state updates with minimal memory footprint
• Background Processing: Non-blocking state updates that don't interrupt conversation flow
• Smart Caching: Intelligent caching of frequently accessed state components
• Memory Management: Automatic cleanup of old messages and actions to prevent memory bloat

📈 State Analytics

The world state provides rich analytics for understanding conversation patterns and bot performance:

• Conversation Metrics: Message frequency, user engagement, and response patterns
• Platform Activity: Cross-platform activity correlation and user behavior analysis
• Bot Performance: Action success rates, response times, and error patterns
• Social Dynamics: User interaction patterns, thread participation, and engagement quality

🔧 Configuration

The system uses environment variables for configuration. Copy .env.example to .env and configure:

� Required Settings

# AI Configuration
AI_MODEL=openai/gpt-4o-mini
OPENROUTER_API_KEY=your_openrouter_key_here

# Matrix Configuration
MATRIX_HOMESERVER=https://matrix.example.org
MATRIX_USER_ID=@your-bot:example.org
MATRIX_PASSWORD=your_secure_password
MATRIX_ROOM_ID=!yourRoom:example.org

⚙️ Optional Settings

# Farcaster Integration
NEYNAR_API_KEY=your_neynar_key
FARCASTER_BOT_FID=your_bot_fid
FARCASTER_BOT_USERNAME=your_bot_username

# Performance Tuning
OBSERVATION_INTERVAL=2.0
MAX_CYCLES_PER_HOUR=300
AI_CONVERSATION_HISTORY_LENGTH=10

# Alternative LLM Provider
PRIMARY_LLM_PROVIDER=ollama  # or "openrouter"
OLLAMA_API_URL=http://localhost:11434

See Configuration Guide for detailed options.

🚀 Deployment

🐳 Docker Production Deployment

1. Configure production environment:

cp .env.example .env.production
# Edit .env.production with production credentials

2. Deploy using Docker Compose:

docker-compose up -d

3. Monitor and manage:

# View logs
docker-compose logs -f chatbot

# Restart services
docker-compose restart

# Update deployment
git pull
docker-compose build
docker-compose up -d

🖥️ Traditional Server Deployment

# Install system dependencies
sudo apt update
sudo apt install python3.10 python3-pip

# Install Poetry
curl -sSL https://install.python-poetry.org | python3 -

# Deploy application
git clone <repository-url>
cd python3-poetry-pyenv
poetry install --only=main
cp .env.example .env
# Edit .env with production settings

# Run as service (using systemd)
sudo cp scripts/chatbot.service /etc/systemd/system/
sudo systemctl enable chatbot
sudo systemctl start chatbot

☁️ Cloud Deployment

The system supports deployment on:

• AWS ECS: Container-based deployment
• Google Cloud Run: Serverless container deployment
• Azure Container Instances: Simple container deployment
• DigitalOcean App Platform: Managed deployment

See Deployment Guide for detailed instructions.

🧪 Testing

The system includes comprehensive testing infrastructure:

# Run all tests
poetry run pytest tests/ -v

# Run with coverage
poetry run pytest tests/ --cov=chatbot --cov-report=html --cov-report=term

# Run specific test categories
poetry run pytest tests/test_core.py -v
poetry run pytest tests/test_world_state_comprehensive.py -v

# Performance testing
poetry run pytest tests/ -m "not slow"  # Skip slow tests
poetry run pytest tests/ -m "slow"      # Run only slow tests

🏗️ Test Structure

• Unit Tests: Individual component testing
• Integration Tests: Multi-component interactions
• End-to-End Tests: Complete workflow testing
• Performance Tests: Load and stress testing

See Testing Guide for detailed information. tests/ ├── test_ai_engine.py # AI decision engine testing ├── test_core.py # Core component unit tests ├── test_orchestrator_extended.py # Orchestrator integration tests ├── test_world_state_extended.py # World state management tests ├── test_tool_system.py # Tool registry and execution tests ├── test_matrix_tools_and_observer.py # Matrix platform integration ├── test_farcaster_tools_follow_dm.py # Farcaster platform features ├── test_integration.py # Full system integration tests └── test_robust_json_parsing.py # AI response parsing reliability

### 🎯 Quality Metrics

#### **Code Quality Tools**
- **Black**: Consistent code formatting across the entire codebase
- **isort**: Import statement organization and optimization
- **flake8**: Code style enforcement and basic linting
- **mypy**: Static type checking for improved reliability
- **pytest**: Comprehensive test framework with async support

#### **Coverage Reporting**
```bash
# Run tests with coverage
poetry run pytest tests/ --cov=chatbot --cov-report=html --cov-report=term

# View HTML coverage report
open htmlcov/index.html

🔧 Development Workflow

Available Tasks

# Main application
poetry run python -m chatbot.main

# Testing
poetry run pytest tests/ -v                           # Run all tests
poetry run pytest tests/ --cov=chatbot               # With coverage

# Code Quality
poetry run black chatbot/ && poetry run isort chatbot/  # Format code
poetry run flake8 chatbot/ && poetry run mypy chatbot/  # Lint and type check

# Development Tools
poetry run python control_panel.py                    # Control panel interface

VS Code Tasks

The project includes pre-configured VS Code tasks for common operations:

• Run Chatbot Main Application: Starts the main bot with background execution
• Run Control Panel: Launches the web-based control interface
• Run Tests: Executes the full test suite
• Run Tests with Coverage: Tests with HTML coverage reporting
• Format Code: Applies Black and isort formatting
• Lint Code: Runs flake8 and mypy validation

🐛 Debugging & Troubleshooting

📋 Common Issues

Connection Problems

# Check Matrix connectivity
grep "matrix_connected" chatbot.log

# Verify Farcaster API access
grep "farcaster_connected" chatbot.log

# Monitor rate limiting
grep "rate_limit" chatbot.log

State Management Issues

# Check world state consistency
grep "WorldState:" chatbot.log

# Monitor message deduplication
grep "Deduplicated message" chatbot.log

# Track action execution
grep "Action completed" chatbot.log

🔍 Debug Tools

Enhanced Logging

# Set debug level logging
export LOG_LEVEL=DEBUG

# Monitor specific components
grep "ContextAwareOrchestrator" chatbot.log
grep "ToolRegistry" chatbot.log
grep "WorldStateManager" chatbot.log

Control Panel Interface

The system includes a web-based control panel for real-time monitoring:

poetry run python control_panel.py
# Access at http://localhost:5000

Features:

• Real-time State Monitoring: Live view of world state and recent activities
• Action History: Complete audit trail of bot actions and results
• Platform Status: Connection status and health metrics for all platforms
• Configuration Viewer: Current configuration settings and environment variables

📖 Documentation

Comprehensive documentation is available:

• README.md: This file - quick start and overview
• ARCHITECTURE.md: Detailed system architecture and design
• DEVELOPMENT.md: Development setup, workflows, and guidelines
• API.md: API reference and integration details

📚 Additional Resources

• Tool System: See chatbot/tools/ for individual tool implementations
• Configuration: Review .env.example for all available settings
• Scripts: Check scripts/ directory for deployment and utility scripts

🤝 Contributing

We welcome contributions! Please see DEVELOPMENT.md for:

• Development environment setup
• Code quality standards
• Testing requirements
• Pull request process

🛠️ Development Commands

# Setup development environment
poetry install
poetry run pre-commit install

# Code quality checks
poetry run black chatbot/           # Format code
poetry run isort chatbot/           # Sort imports  
poetry run flake8 chatbot/          # Lint code
poetry run mypy chatbot/            # Type checking

# Run tests
poetry run pytest tests/ -v        # All tests
poetry run pytest tests/ --cov     # With coverage

📄 License

This project is licensed under the Creative Commons Attribution-NonCommercial 4.0 International License. See LICENSE file for details.

Key Terms:

• ✅ Share & Adapt: You can copy, redistribute, remix, and build upon the material
• 🏷️ Attribution Required: You must give appropriate credit and indicate changes
• 🚫 Non-Commercial: Commercial use requires explicit written permission
• 📧 Commercial Licensing: Contact us for commercial use permissions

For commercial use, licensing, or any revenue-generating applications, please obtain written permission from the copyright holder.

🆘 Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Documentation: Project Documentation

---

⬆ Back to Top

Built with ❤️ using Python, Poetry, and modern async technologies