Performance gap: 2-stage vs. Native kNN-LM

[Rubin-Wei]

Hi authors,

Regarding the Two-Stage k-NN Search (Section 3.1), I am curious if you have compared it with the native (single-stage) kNN-LM?

I would love to know the specific differences in:

1. Effectiveness: Does the 2-stage approximation cause a noticeable drop in perplexity?

2. Efficiency: How significant is the speed/latency gap between them?

Any qualitative analysis or quantitative results would be greatly appreciated. Thanks!