Skip to content

LLM-chain: design and base setup (interfaces, chaining logic) #542

@kartpop

Description

@kartpop

Is your feature request related to a problem? Please describe.
Currently, /llm/call only supports single LLM calls. For multi-step AI workflows (e.g., STT → LLM → TTS), developers must manually chain multiple API calls, handle intermediate results, and manage errors across requests. This increases latency and complexity.

Describe the solution you'd like
Extend the existing LLM service layer to support chained execution through a new /llm/chain endpoint. This reuses the core LLM execution components (models, config resolution, provider registry, observability, job tracking) and orchestrates multiple sequential blocks at the service layer, where each block's output becomes the next block's input. This maintains architectural consistency by sharing the same infrastructure (ConfigBlob, QueryParams, provider execution, langfuse integration) while adding orchestration logic for multi-step workflows.

Describe alternatives you've considered
Check for the langchain, haystack or other agent frameworks how they handle the multi-step llm-execution

Additional context
Use a well-defined API contract for llm/chain that is flexible and closely aligned with the existing llm/call API contract. The new contract should support all functionalities currently available in llm/call, while additionally enabling prompt interpolation. This will allow users to modify, extend, or wrap additional context around previous outputs when invoking chained LLM calls.

Sub-issues

Metadata

Metadata

Assignees

Labels

sub-parentChild of parent label for roadmap view

Projects

Status

In Progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions