Skip to content

Add git indexing: orchestrator, worker, chunking, and embeddings#2

Merged
karngyan merged 3 commits intomainfrom
indexer
Mar 1, 2026
Merged

Add git indexing: orchestrator, worker, chunking, and embeddings#2
karngyan merged 3 commits intomainfrom
indexer

Conversation

@karngyan
Copy link
Member

@karngyan karngyan commented Mar 1, 2026

Summary

  • Implements phase 2 git indexing: orchestrator and worker that clone repos, chunk files, generate OpenAI embeddings, and upsert into Qdrant
  • Adds gitrepo library for cloning and traversing Git repositories with provider abstraction (GitHub/GitLab)
  • Adds chunking library for splitting Go and generic source files into semantic chunks
  • Adds openai library with embeddings client and fake for testing
  • Adds gittokens domain for managing per-workspace Git access tokens
  • Adds index_runs DB table and queries to track indexing job state
  • Expands Qdrant collection helpers and CI workflow

Test Plan

  • CI passes (unit + integration tests via make test)
  • make dev starts without errors (API + Qdrant + Postgres healthy)
  • Indexing worker processes a repo end-to-end and upserts vectors into Qdrant
  • GET /health returns healthy

@karngyan karngyan merged commit 696dd4b into main Mar 1, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant