Streaming evaluation pipeline #42
Description
Phase: 3 — Growth & Scale (Performance)
Evaluate jobs incrementally as search results arrive, instead of batch-after-search:
- Prototype streaming evaluation — evaluate each job as it is parsed from SerpAPI results
- Benchmark latency and cost vs current batch approach
- Gate LLM evaluations with cheap heuristics — skip obvious mismatches before calling Gemini
- Add A/B quality checks to ensure optimization does not reduce match relevance
This could significantly reduce end-to-end latency for the UI experience.