Thaen/DynamicAbstractionSystem
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
DynamicAbstractionSystem/README_ARCHITECTURE.md
Sam 3a34759094
Some checks failed
Build Simulation and Test / Run All Tests (push) Failing after 8m17s
Details
Add core simulation components and configuration classes 
Major rewrite.
2025-11-08 19:17:40 -06:00

13 KiB
Raw Blame History

Dynamic Abstraction System - Complete Architecture

Overview

The Dynamic Abstraction System is a sophisticated 2D cellular simulation platform featuring autonomous entities with evolving neural networks. The system demonstrates headless-first architecture with clean separation of concerns, supporting both interactive development and production-scale batch processing.

Architecture Principles

  1. Headless-First Design: Core simulation logic runs independently of any UI dependencies
  2. Separation of Concerns: Each component has a single, well-defined responsibility
  3. Loose Coupling: Components communicate through events and interfaces rather than direct dependencies
  4. Configuration-Driven: Behavior controlled through structured configuration files
  5. Extensible Data Collection: Comprehensive metrics and evolution tracking capabilities
  6. Dual-Mode Support: Both interactive debugging and headless batch processing

Complete Directory Structure

DynamicAbstractionSystem/
├── main.py                        # Legacy entry point (interactive)
├── entry_points/                  # Application entry points
│   ├── headless_main.py          # Headless simulation entry
│   ├── interactive_main.py       # Interactive simulation entry
│   └── __init__.py
├── core/                          # Core simulation infrastructure
│   ├── simulation_core.py        # Pure simulation logic (no UI deps)
│   ├── timing.py                 # TPS/timing management
│   ├── event_bus.py              # Decoupled communication system
│   ├── simulation_engine.py      # Interactive engine (UI wrapper)
│   ├── input_handler.py          # Pure input processing
│   ├── renderer.py               # World-to-screen rendering
│   └── __init__.py
├── engines/                      # Specialized simulation engines
│   ├── headless_engine.py       # Production headless engine
│   └── __init__.py
├── world/                        # Simulation world and entities
│   ├── __init__.py
│   ├── world.py                  # Spatial partitioning system
│   ├── objects.py                # Entity implementations
│   ├── base/                     # Base entity systems
│   │   ├── brain.py             # Neural network implementations
│   │   ├── neural.py            # Flexible neural network
│   │   └── behavioral.py        # Behavioral models
│   └── simulation_interface.py   # Camera and spatial interfaces
├── ui/                          # User interface components
│   ├── hud.py                   # Heads-up display and panels
│   └── __init__.py
├── config/                      # Configuration management
│   ├── constants.py             # Global constants
│   ├── simulation_config.py     # Configuration classes
│   ├── config_loader.py         # Configuration loading utilities
│   └── __init__.py
├── output/                      # Data collection and output
│   ├── __init__.py
│   ├── collectors/              # Data collection modules
│   │   ├── base_collector.py    # Collector interface
│   │   ├── metrics_collector.py # Basic metrics collection
│   │   ├── entity_collector.py  # Entity state tracking
│   │   └── evolution_collector.py # Evolution tracking
│   ├── formatters/              # Output formatting
│   │   ├── base_formatter.py    # Formatter interface
│   │   ├── json_formatter.py    # JSON output
│   │   └── csv_formatter.py     # CSV output
│   └── writers/                 # Output destinations
│       ├── base_writer.py       # Writer interface
│       └── file_writer.py       # File system output
└── tests/                       # Test suite
    ├── __init__.py
    └── [test files]

Core System Architecture

1. Simulation Layer

SimulationCore (core/simulation_core.py)

  • Purpose: Pure simulation logic with zero UI dependencies
  • Responsibilities:
    • World management and entity lifecycle
    • Entity spawning and spatial queries
    • Event-driven state management
    • Timing coordination
  • Key Features:
    • Event-driven architecture via EventBus
    • Configurable simulation parameters
    • Complete UI isolation
    • Spatial query methods (radius, range, nearest)

TimingController (core/timing.py)

  • Purpose: Precise timing management and simulation control
  • Responsibilities:
    • TPS (ticks per second) regulation
    • Pause/sprint state management
    • Speed multipliers
    • Performance timing
  • Key Features:
    • Configurable target TPS
    • Sprint mode for accelerated execution
    • Pause/resume functionality
    • Real-time performance metrics

EventBus (core/event_bus.py)

  • Purpose: Decoupled inter-component communication
  • Responsibilities:
    • Event routing and subscription management
    • Type-safe event dispatch
    • Event history tracking
  • Key Features:
    • Type-safe event system
    • Event history for debugging
    • Publisher-subscriber pattern
    • Loose coupling between components

2. Engine Layer

SimulationEngine (core/simulation_engine.py)

  • Purpose: Interactive simulation engine with UI integration
  • Responsibilities:
    • Main game loop coordination
    • UI event processing
    • Rendering pipeline management
    • Component orchestration
  • Key Features:
    • Pygame integration
    • Real-time input handling
    • Rendering coordination
    • UI state synchronization

HeadlessSimulationEngine (engines/headless_engine.py)

  • Purpose: Production-grade headless simulation execution
  • Responsibilities:
    • Batch simulation execution
    • Data collection management
    • Output generation
    • Performance monitoring
  • Key Features:
    • No UI dependencies
    • Maximum speed execution (2000+ TPS)
    • Sprint mode integration for unlimited performance
    • Precise TPS control when specified
    • Comprehensive data collection
    • Multiple output formats
    • Configurable execution parameters
    • Graceful shutdown handling

3. World Layer

World (world/world.py)

  • Purpose: Spatial partitioning and entity management
  • Responsibilities:
    • Spatial indexing and partitioning
    • Entity lifecycle management
    • Collision detection support
    • Spatial queries
  • Key Features:
    • Grid-based spatial partitioning
    • Double buffering for performance
    • Efficient spatial queries
    • Entity add/remove operations

Entity System (world/objects.py, world/base/)

  • Purpose: Entity implementations and behavioral systems
  • Key Components:
    • BaseEntity: Abstract base class for all entities
    • DefaultCell: Autonomous cellular entity with neural networks
    • FoodObject: Energy source entities
    • CellBrain: Neural network-based decision making
    • FlexibleNeuralNetwork: Dynamic topology neural networks

4. Data Collection System

Collectors (output/collectors/)

  • BaseCollector: Abstract base with tick-based collection logic
  • MetricsCollector: Basic simulation metrics (TPS, entity counts, timing)
  • EntityCollector: Detailed entity state snapshots
  • EvolutionCollector: Neural network evolution and generational tracking
  • Tick-Based Intervals: Uses simulation ticks for accurate collection timing
  • Every-Tick Option: Configurable for maximum data resolution

Formatters (output/formatters/)

  • JSONFormatter: Human-readable JSON output
  • CSVFormatter: Spreadsheet-compatible CSV output

Writers (output/writers/)

  • FileWriter: File system output with batch processing

5. Configuration System

Configuration Classes (config/simulation_config.py)

  • HeadlessConfig: Headless simulation parameters
  • InteractiveConfig: Interactive UI configuration
  • ExperimentConfig: Automated experiment configuration
  • OutputConfig: Data collection and output settings

Configuration Loader (config/config_loader.py)

  • Purpose: Dynamic configuration loading and validation
  • Features:
    • JSON/YAML support
    • Configuration validation
    • Sample configuration generation
    • Lazy loading to avoid circular dependencies

Entry Points

Interactive Mode

# Standard interactive simulation
python entry_points/interactive_main.py

# With custom configuration
python entry_points/interactive_main.py --config configs/interactive.json

# Legacy compatibility
python main.py

Headless Mode

# Basic headless execution (maximum speed by default)
python entry_points/headless_main.py

# With custom parameters
python entry_points/headless_main.py --max-ticks 10000 --tps 60 --output-dir results

# With data collection on every tick
python entry_points/headless_main.py --max-ticks 1000 --collect-every-tick

# With configuration file
python entry_points/headless_main.py --config configs/experiment.json

Input and Control System

Keyboard Controls

  • Space: Toggle pause/play
  • Right Shift: Toggle sprint mode
  • Left Shift: Hold for temporary 2x speed boost
  • S: Step forward one tick
  • R: Reset camera position
  • G: Toggle grid display
  • I: Toggle interaction radius display
  • L: Toggle legend display
  • WASD: Move camera
  • Mouse wheel: Zoom in/out
  • Middle mouse drag: Pan camera
  • Left click/drag: Select objects

Input Handler Architecture

  • Pure Input Processing: No state management, only input mapping
  • Callback System: All simulation control actions use callbacks
  • Event-Driven: Decoupled from simulation state

Data Collection and Output

Comprehensive Metrics

  • Performance Metrics: TPS, timing statistics, execution duration
  • Entity Statistics: Population counts, energy distributions, age demographics
  • Evolution Data: Neural network architectures, generational statistics, mutation tracking
  • World State: Spatial distributions, resource availability

Output Formats

  • JSON: Structured, human-readable, web-compatible
  • CSV: Tabular format for spreadsheet analysis
  • File-based: Batch output with configurable intervals

Performance Architecture

Spatial Optimization

  • Grid-based Spatial Partitioning: Efficient entity queries
  • Double Buffering: Prevents frame interference
  • Spatial Indexing: Fast proximity and range queries

Timing Optimization

  • Configurable TPS: Adjustable simulation speed
  • Maximum Speed Mode: Unlimited execution speed for batch processing (2000+ TPS)
  • Sprint Mode Integration: Automatic sprint mode for maximum performance
  • Precise Timing: Frame-independent simulation updates
  • Accurate TPS Control: Precise timing when TPS is specified

Memory Management

  • Entity Pooling: Efficient memory allocation for entities
  • Event System: Minimal memory overhead for inter-component communication
  • Lazy Loading: Configuration and components loaded on demand

Testing Architecture

Test Coverage

  • Unit Tests: Core component isolation testing
  • Integration Tests: Component interaction testing
  • Performance Tests: Benchmarking and profiling
  • End-to-End Tests: Complete simulation workflow testing

Testing Tools

  • cProfile Integration: Built-in performance profiling
  • pytest Framework: Structured test execution
  • Mock Components: Isolated component testing

Extension Points

Custom Collectors

  • Extensible data collection system
  • Plugin architecture for custom metrics
  • Configurable collection intervals

Custom Formatters

  • Pluggable output format system
  • Custom serialization support
  • Multiple output destinations

Custom Engines

  • Specialized simulation engines
  • Custom execution patterns
  • Integration interfaces

Deployment Architecture

Production Deployment

  • Headless Execution: Server deployment without display dependencies
  • Batch Processing: Automated experiment execution
  • Configuration-Driven: Environment-specific configurations
  • Data Pipeline: Automated data collection and output

Development Environment

  • Interactive Debugging: Real-time visualization and control
  • Hot Reloading: Configuration changes without restart
  • Performance Monitoring: Real-time TPS and performance metrics
  • Extensive Logging: Detailed execution tracking

Architectural Benefits

  1. Maintainability: Clear separation of concerns and modular design
  2. Testability: Isolated components enable comprehensive unit testing
  3. Scalability: Headless mode supports large-scale batch processing
  4. Flexibility: Configuration-driven behavior adaptation
  5. Performance: Optimized spatial queries and timing management
  6. Extensibility: Plugin architecture for custom functionality
  7. Reliability: Event-driven architecture with error handling
  8. Portability: Headless operation enables deployment across platforms

This architecture provides a robust foundation for both interactive development and production-scale simulation experiments, with clean separation between simulation logic, user interface, and data collection systems.