Skip to content

dipan010/TouchlessUX

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

22 Commits
 
 
 
 
 
 
 
 

Repository files navigation

  1. High-Level Architecture Your backend is a microservices event-driven system using:
  • Kafka (Redpanda) for messaging between services
  • Redis for state tracking and context
  • FastAPI for HTTP APIs (adapters, app interface)
  • Python asyncio for all service logic Core Flow
  1. User Input (voice/text) → Frontend → WebSocket/API → Backend

  2. Backend (app_interface) receives transcript, pushes to Kafka (transcript or agent.cmd)

  3. Reasoning Service (GPT wrapper) decides which tool/agent to call (food, ride, mart, etc.)

  4. Router dispatches the command to the correct domain agent via HTTP (adapters) and Kafka

  5. Domain Agent (food_agent, ride_agent, mart_agent) mocks the action, writes result to Kafka

  6. State Tracker updates Redis and emits status updates

  7. Frontend can subscribe for updates or poll status

  8. Key Components a. app_interface/

  • mic_ws.py, text_ws.py: FastAPI WebSocket endpoints for ingesting voice/text from the frontend.
  • Pushes incoming transcripts to Kafka. b. reasoning/
  • main.py: The "brain"—consumes transcripts, uses GPT (LLM) to decide which tool to call (book_ride, order_food, order_mart).
  • Maintains slot-filling state in Redis.
  • If clarification is needed, triggers clarify agent. c. router/
  • main.py: Listens for commands on Kafka, routes them to the correct HTTP endpoint (adapters) and then to the correct Kafka output topic. d. adapters/
  • main.py: FastAPI app exposing REST endpoints for each domain (ride, food, mart, payment, location).
  • Each endpoint is a thin wrapper that forwards requests to the correct agent. e. Domain Agents (food_agent, ride_agent, mart_agent)
  • Each listens to agent.cmd Kafka topic.
  • If the command matches their tool, they mock a response (e.g., confirm ride, food order) and write to their output topic (agent.out.ride, agent.out.food, etc.). f. state_tracker/
  • Tracks the lifecycle of each command using Redis.
  • Listens to all relevant Kafka topics and updates state/status for each trace_id. g. error_agent/
  • Listens to a dead-letter queue (agent.error) and logs or forwards errors.
  1. Data Flow Example User says: "Order me a burger"

  2. Frontend sends transcript via WebSocket to app_interface.

  3. app_interface writes to Kafka (transcript topic).

  4. reasoning consumes transcript, determines intent (order_food), and required slots (e.g., food item, address).

  5. If info is missing, triggers clarify agent; otherwise, emits command to agent.cmd.

  6. router picks up command, POSTs to /food/order on adapters, then writes result to agent.out.food.

  7. food_agent mocks order, writes confirmation to Kafka.

  8. state_tracker updates Redis state for the trace_id.

  9. Frontend can poll or subscribe for status updates.

  10. Observations & Suggestions

  • Mock Agents: All domain agents are currently mocks—they don’t call real APIs, just return canned responses.
  • Slot Filling: The reasoning service handles slot filling and clarification using LLM prompts and Redis state.
  • Adapters Layer: Provides a clean HTTP interface for each domain, decoupling the router from agent implementation.
  • State Tracking: Centralized in Redis, keyed by trace_id, for easy status lookup.
  • Error Handling: Errors are routed to a DLQ and logged by error_agent.
  1. How to Extend or Debug
  • To add a new domain: Add a new agent, adapter endpoint, and routing entry.
  • To debug a flow: Trace the trace_id through Kafka topics and Redis state.
  • To see live state: Use Redis CLI to inspect state:{trace_id}.
  • To see logs: Check each service’s logs for info/warnings.
  1. Example Command/Topic List From commands.txt and code:
  • Kafka Topics: agent.cmd, agent.out.ride, agent.out.food, agent.out.mart, agent.error, agent.status, agent.out.clarify
  • Redis Keys: state:{trace_id}

Summary Table Service Input Topic/API Output Topic/API Purpose app_interface WebSocket Kafka (transcript) Ingest user input reasoning transcript agent.cmd LLM intent/slot reasoning router agent.cmd adapters HTTP, Kafka Route to correct agent adapters HTTP Kafka HTTP → Kafka bridge food_agent agent.cmd agent.out.food Mock food order ride_agent agent.cmd agent.out.ride Mock ride booking mart_agent agent.cmd agent.out.mart Mock mart order state_tracker all topics Redis, agent.status Track state/status error_agent agent.error log/notify DLQ/error handling In summary:Your backend is a modular, event-driven, microservice system for multi-domain command handling, using Kafka for messaging, Redis for state, and FastAPI for HTTP APIs. Each agent is currently a mock, but the architecture supports real integrations. The reasoning service is the "brain" that interprets user input and orchestrates the rest. If you want a deep-dive into a specific service, flow, or want to see how to add a new feature, let me know!

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 2

  •  
  •