Skip to content

Comments

Feature/websockets clean#2

Open
spicyneutrino wants to merge 5 commits intoXGraph-Team:mainfrom
spicyneutrino:feature/websockets-clean
Open

Feature/websockets clean#2
spicyneutrino wants to merge 5 commits intoXGraph-Team:mainfrom
spicyneutrino:feature/websockets-clean

Conversation

@spicyneutrino
Copy link

Pull Request

📋 Description

This PR introduces real-time WebSocket architecture, a full agentic AI chatbot system with 26 LLM tools, enhanced 3D visualization, and comprehensive frontend-backend synchronization. It migrates the app from HTTP polling to event-driven WebSocket updates, adds an OpenAI function-calling chatbot that can control every aspect of the simulation, and delivers smoother vehicle rendering with 60fps interpolation.

Commits

  1. e3fc4f4feat: websocket migration (clean history)
  2. 4b3d674feat: agentic chatbot, 3D/2D toggle, smoother visuals, and state management
  3. db52edffeat: expand agentic chatbot with 5 new tools, bug fixes, and frontend sync

🔄 Type of Change

  • 🐛 Bug fix (non-breaking change which fixes an issue)
  • ✨ New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📚 Documentation update
  • 🧹 Code refactoring (no functional changes)
  • ⚡ Performance improvement
  • 🧪 Test addition or improvement

🧪 Testing

  • Unit tests
  • Integration tests
  • Manual testing
  • End-to-end testing

Test Configuration:

  • Python version: 3.12
  • SUMO version: 1.15+
  • Operating system: Linux (Fedora/Bluefin)
  • Browser: Chrome (latest)

📸 Screenshots (if applicable)

Before:

  • Continuous HTTP polling (5-7 req/sec per client)
  • No AI chatbot with tool execution
  • Basic vehicle rendering

After:

  • Zero polling requests — all data pushed via WebSocket
  • Full agentic chatbot with 26 tools controlling the simulation
  • 60fps vehicle interpolation with battery-color-coded EVs and 3D buildings

✅ Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published

🔧 Implementation Details

Changes Made


Commit 1: WebSocket Migration (e3fc4f4)

  • main_complete_integration.py: Enhanced broadcast_state() to push V2G active vehicle data and AI map focus coordinates via system_update WebSocket event. Added v2g_websocket_callback() for real-time restoration notifications.
  • static/script.js: Removed updateLoop() and updateV2GColorsOptimized() polling loops. Added updateV2GFromWebSocket() and applyAIMapFocus() to process data from WebSocket events instead.
  • static/scenario-director.js: Removed startChatbotMonitoring() polling loop. Added WebSocket listener for v2g_restoration_complete events.
  • static/scenario-controls.js: Added Socket.IO listeners for scenario sync events (scenario_time_update, scenario_temp_update, simulation_speed_update, substation_update, v2g_update, etc.)
  • index.html: Updated cache-busting query parameters to force browser reload.
  • core/power_system.py, sumo_manager.py, scenario_controller.py: Minor fixes for WebSocket integration.

Commit 2: Agentic Chatbot & Visualization (4b3d674)

  • agentic_chatbot.py [NEW]: Full AgenticChatbot class with OpenAI function-calling loop, dynamic system prompt with live grid state, conversation history management, 3-tier fallback (agentic → ultra → world-class), and _emit_ui_updates for real-time frontend sync via Socket.IO.
  • agentic_tools.py [NEW]: 21 tool schemas and ToolExecutor class covering substation control, V2G management, simulation control, scenario management, EV configuration, system status queries, map control, and test scenarios.
  • main_complete_integration.py: Integrated agentic chatbot as primary AI handler in /api/ai/chat route with fallback chain. Added /api/config for dynamic Mapbox token delivery.
  • static/script.js: 3D terrain and buildings, triangle arrow vehicles with bearing-aligned rotation, 60fps interpolation loop with easeOutCubic, battery-color-coded EVs, traffic light zoom-fade, dynamic Mapbox token loading. Bypassed frontend LLM interception — all chat routed to backend.
  • static/scenario-controls.js: 8 Socket.IO listeners for agentic chatbot UI sync events.

Commit 3: Chatbot Expansion & Bug Fixes (db52edf)

  • agentic_tools.py: Added 5 new tools (toggle_map_layer, set_map_view, fail_ev_station, restore_ev_station, trigger_blackout). Fixed _run_ev_rush_test spawn queue format bug.
  • agentic_chatbot.py: System prompt now includes exact valid substation names, EV station IDs, and layer names. Conversation history preserves tool call summaries. Added 6 new _emit_ui_updates handlers.
  • core/power_system.py: Fixed ValueError from mismatched Series indices by aligning line_flows and line_limits before comparison (2 locations).
  • scenario_controller.py: Fixed event log time format to use _format_time().
  • static/scenario-controls.js: Added 7 new Socket.IO listeners (ev_config_update, vehicles_spawned, layer_toggle, ev_station_update, substation_update, map_view_change).
  • static/script.js: Exposed toggleLayer, updateEVPercentage, updateBatteryRange on window for socket event handlers.

Architecture Impact

  • Power Grid System: Fixed PyPSA Series index alignment to prevent crashes during line flow analysis
  • Vehicle Simulation: 60fps interpolation, vType-aware SUMO routing, broadcast frame-skipping (1 update per 5 physics steps)
  • V2G System: Real-time WebSocket push for V2G status and restoration events
  • ML/AI Components: Full agentic chatbot with 26 tools, OpenAI function calling, local LLM auto-detection, 3-tier fallback chain
  • API: New WebSocket events replace polling endpoints; /api/config added for Mapbox token; legacy endpoints remain for backward compatibility

Performance Considerations

  • Network Traffic: Eliminated ~5-7 HTTP GET req/sec per client — replaced with event-driven WebSocket push
  • CPU Usage: Removed setInterval/setTimeout polling loops on frontend; added broadcast frame-skipping on backend
  • Memory Usage: Conversation history capped at configurable max to prevent unbounded growth
  • Rendering: 60fps vehicle interpolation with adaptive timing and easeOutCubic smoothing

🔄 Migration Notes

  1. Set MAPBOX_TOKEN environment variable (Mapbox access token is no longer hardcoded)
  2. Optionally set OPENAI_API_KEY and OPENAI_BASE_URL for agentic chatbot (falls back to other AI systems if not set)
  3. Hard refresh (Ctrl+Shift+R) after deployment to clear browser cache

📋 Dependencies

No new Python dependencies required.

Utilizes existing infrastructure:

  • openai — Already in requirements
  • flask-socketio — Already in use
  • requests — Already in use

🔗 Related Issues/PRs

  • Completes WebSocket migration initiative
  • Introduces agentic AI chatbot system

📚 Documentation

  • README updated
  • Inline code comments added
  • Tutorial/guide updated
  • CHANGELOG updated

🚀 Deployment Notes

  • Requires environment variable changes — MAPBOX_TOKEN (required), OPENAI_API_KEY + OPENAI_BASE_URL (optional)
  • Requires database migration — NO
  • Requires configuration updates — NO
  • Requires external service setup — NO

Post-deployment:

  • Verify WebSocket connection stability
  • Test agentic chatbot tool execution (e.g., "switch to 2D map", "trigger a blackout")
  • Confirm browser cache invalidation across user base

🧑‍💻 Reviewer Notes

Please pay special attention to:

  • Security: No API keys or tokens are committed — all loaded from environment variables
  • Performance: Zero polling requests; WebSocket-only data flow
  • Error handling: Agentic chatbot has try/catch on all tool executions with graceful fallback
  • Test coverage: Manual testing performed; automated tests recommended
  • Documentation completeness: API docs should be updated for new WebSocket events

Testing Recommendations:

  1. Test chatbot with commands like "switch to 2D", "show me EV stations", "trigger a blackout"
  2. Verify Network tab shows 0 polling requests
  3. Test V2G restoration during active simulation
  4. Test browser refresh while data is actively updating

📝 Additional Notes

Bug Fixes Included

Bug Root Cause Fix
_run_ev_rush_test crashes Queue items had wrong format ({count: 30} instead of individual items) Push individual items with ev_percentage, battery_min_soc, battery_max_soc
ValueError in power_system.py Mismatched Series indices when comparing line flows to limits Align indices with .intersection() before comparison
Scenario event log shows raw time Used self.current_time instead of formatted string Changed to self._format_time(self.current_time_seconds)
LLM guesses wrong substation names System prompt had no list of valid names Inject exact names into prompt dynamically
LLM forgets past actions Conversation history only stored text, not tool calls Store tool call summaries with each assistant message

New Chatbot Tools (26 total)

Category Tools
Substation Control fail_substation, restore_substation, restore_all_substations
V2G Control enable_v2g, disable_v2g, get_v2g_status
Simulation start_simulation, stop_simulation, spawn_vehicles, set_simulation_speed
Scenario set_time, set_temperature, run_scenario
EV Config configure_ev
Status Queries get_system_status, get_scenario_status, get_substation_details, get_load_forecast
Map Control focus_map, toggle_map_layer, set_map_view
EV Station Control fail_ev_station, restore_ev_station
Scenarios trigger_blackout, run_ev_rush_test, run_v2g_test

spicyneutrino and others added 5 commits February 12, 2026 14:39
…gement

Agentic Chatbot:
- Add agentic_chatbot.py with OpenAI function calling, local LLM support, dynamic system prompt
- Add agentic_tools.py with 21 tool schemas and direct ToolExecutor
- 3-tier fallback chain in /api/ai/chat (agentic → ultra → worldclass)
- 8 Socket.IO listeners for real-time UI sync (scenario-controls.js)

Visualization & State Management:
- 3D/2D toggle with terrain, 3D buildings, and cinematic camera angles
- Triangle arrow vehicles (SDF icons) with bearing-aligned rotation
- 60fps vehicle interpolation loop with adaptive timing and easeOutCubic
- Battery-color-coded EVs (red/orange/green) and state-based coloring (V2G, charging, stranded)
- Traffic light zoom-fade (hidden when zoomed out)
- Broadcast frame-skipping for performance (1 update per 5 physics steps)
- vType-aware SUMO routing with edge permission error handling

Infrastructure:
- Dynamic Mapbox token loading from MAPBOX_TOKEN env var via /api/config
- Bypass frontend LLM interception — all chat routed to backend
…d sync

New Tools:
- toggle_map_layer: show/hide map layers (lights, cables, EVs, substations, vehicles)
- set_map_view: switch between 2D (flat) and 3D (tilted) map perspectives
- fail_ev_station / restore_ev_station: individual EV station control
- trigger_blackout: fail all substations except one spare in a single command

Bug Fixes:
- Fix _run_ev_rush_test spawn queue format (was pushing {count: 30} instead of individual items with ev_percentage/battery_min_soc/battery_max_soc)
- Fix PyPSA Series index alignment in power_system.py to prevent ValueError on mismatched indices
- Fix scenario event log time format (use _format_time instead of raw value)

Improvements:
- System prompt now includes exact valid substation names, EV station IDs, and layer names to prevent LLM name-guessing failures
- Conversation history preserves tool call summaries across turns so LLM remembers past actions
- 6 new _emit_ui_updates handlers for real-time frontend sync (layer_toggle, map_view_change, ev_station_update, substation_update, ev_config_update, vehicles_spawned)
- 7 new Socket.IO listeners in scenario-controls.js for all new chatbot events
- Exposed toggleLayer, updateEVPercentage, updateBatteryRange on window for socket event handlers
…controls, fix incident logging error and clean up script.js
… links

Snapshot: rewrite captureSnapshot() to produce comprehensive JSON with
grid status, traffic, V2G, KPIs, scenario info alongside the map PNG.
Add server-side /api/snapshot/state endpoint.

Report: overhaul report_generator.py with 8 PDF sections (executive
summary, per-substation table, traffic/EV, V2G ops, cable integrity,
KPIs dashboard, optional map screenshot, AI-generated analysis).
OpenAI-powered detailed analysis with rule-based fallback.

Fix chatbot report download: auto-trigger PDF download via WebSocket
event (trigger_report_download), add clickable download buttons in
chat UI, pass url/download_link through tool_calls_made response.

Also includes: clock flicker fix, initial layer visibility, scenario
controller system-time init, agentic_tools fixes (_restore_all typo,
missing _prepare_for_event, _trigger_blackout handler mapping), and
reduced auto-scenario wait from 90s to 30s.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant