feat: list tenant using RediSearch

[alexluong]

Summary

Adds a new GET /tenants endpoint for listing tenants with cursor-based pagination. This feature requires Redis with RediSearch module and auto-detects availability at startup.

Feature

• List all tenants sorted by created_at (ascending or descending)
• Cursor-based pagination with next/prev cursors
• Returns lightweight tenant objects (without computed fields like destinations_count and topics)
• Returns 501 Not Implemented when RediSearch is unavailable

Migration

Existing deployments must run these migrations before using this feature:

outpost-migrate-redis apply 002_timestamps
outpost-migrate-redis apply 003_entity

002_timestamps: Converts timestamp fields from RFC3339 strings to Unix millisecond timestamps. The RediSearch index requires consistent numeric values for correct sorting.

003_entity: Adds entity field to tenant and destination records. This field is used by the RediSearch index FILTER to distinguish tenants from destinations (both share the tenant: key prefix).

Implementation

• On startup, Outpost probes for RediSearch support via FT._LIST
• If available, creates a tenant index with created_at NUMERIC SORTABLE
• Uses keyset pagination (timestamp-based cursors) instead of offset pagination for stable results
• Soft-deleted tenants are filtered at the query level to avoid consuming pagination slots

Pagination Behavior

                        ┌─────────────────────────────────────────────────┐
                        │          Tenants (sorted by created_at DESC)    │
                        │                                                 │
  Newest ───────────────│  [T10] [T9] [T8] [T7] [T6] [T5] [T4] [T3] [T2] [T1]  │─── Oldest
                        │                                                 │
                        └─────────────────────────────────────────────────┘

  Page 1 (no cursor):    [T10, T9, T8]  → next=T8, prev=∅
                              │
                              ▼ (use next=T8)
  Page 2:                [T7, T6, T5]   → next=T5, prev=T7
                              │
                              ▼ (use next=T5)
  Page 3:                [T4, T3, T2]   → next=T2, prev=T4
                              │
                              ▼ (use next=T2)
  Page 4:                [T1]          → next=T1, prev=T1
                              │
                              ▼ (use next=T1)
  Page 5 (empty):        []            → next=∅, prev=∅  (end of data)

Forward traversal (next cursor): Returns items OLDER than cursor timestamp (exclusive)
Backward traversal (prev cursor): Returns items NEWER than cursor timestamp (exclusive)

Note: When you reach an empty page, both cursors are empty. To traverse backward, use the prev cursor from the last non-empty page. The cursors use exclusive boundaries to prevent duplicate items.

Cursor Format

Cursors are opaque, URL-safe strings that encode pagination state:

• Internal format: tntv01:<unix_timestamp_ms> (versioned for future compatibility)
• Encoded as base62 for compact, URL-safe representation
• Example: eD9YwjRTAWwJL3MZcV36caQ decodes to tntv01:1705314600000

The version prefix allows cursor format changes without breaking existing clients. Format: <entity><version> where:

• tntv01 = tenant v01
• Future: evtv01 for events, delv01 for deliveries, etc.

Tests

• Unit tests for pagination, sorting, cursor handling, and edge cases
• Tests run against both Redis Stack and Dragonfly backends
• E2E tests for API response format and error handling

Known Limitations

1. Timestamp Collision

Keyset pagination relies on created_at timestamps for ordering. If many tenants share the exact same millisecond timestamp, pagination may not traverse all records. This is extremely unlikely in normal operation due to millisecond precision—even in high-throughput systems, tenant creation is typically infrequent enough that collisions are theoretical rather than practical.

2. Shared Index with Destinations

The RediSearch tenant index includes both tenant and destination records because they share the same key prefix (tenant:{id}:*). We use an entity field to distinguish them:

• Tenants: entity: "tenant"
• Destinations: entity: "destination"

The index includes FILTER '@entity == "tenant"' but this doesn't work as expected. The actual filtering is done at query time with @entity:{tenant} in every FT.SEARCH query.

The alternative would be changing the key prefix structure (e.g., tenants:{id} vs tenant:{id}:destination:{destId}), but this would be a breaking change for existing data and require a more complex migration.

TODO

• Add migration guide with explanation and step-by-step instructions for v0.12.0

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Review	Updated (UTC)
outpost-docs	Ready	Preview, Comment	Jan 12, 2026 9:40pm
outpost-website	Ready	Preview, Comment	Jan 12, 2026 9:40pm

[alexbouchardd]

Returns lightweight tenant objects (without computed fields like destinations_count and topics)

My understanding is that those aren't computed and are instead stored on the object itself. If that's not the case, then I think we should seriously consider refactoring to do so.

Are we currently pulling all the tenant destinations in order to compute those fields of GET /:tenant_id?

[alexluong]

Sort of, but let me explain the logic to make sure we're on the same page before we decide on the next step.

Here are the relevant Redis keys structure:

{deploy}:tenant:{tenantID}:tenant [HASH]
├── id
├── created_at
├── updated_at
├── metadata

{deploy}:tenant:{tenantID}:destinations [HASH]
└── {dest_1} → JSON(DestinationSummary)
└── {dest_2} → JSON(DestinationSummary)

schema for DestinationSummary

type DestinationSummary struct {
	ID       string `json:"id"`
	Type     string `json:"type"`
	Topics   Topics `json:"topics"`
	Filter   Filter `json:"filter,omitempty"`
	Disabled bool   `json:"disabled"`
}

We use the hash that store all DestinationSummary in event ingestion to match the event with the right destination. We can merge this into the Tenant entity itself, but then Tenant will be in the hot path of event ingestion. It's not super significant from a memory standpoint given Tenant should be small, but I figured it's better to have them as 2 different hashes.

Now back to the question, on Tenant Retrieve, we do this query logic

# RetrieveTenant does this in a pipeline:
HGETALL {deploy}:tenant:{tenantID}:tenant
HGETALL {deploy}:tenant:{tenantID}:destinations

# Then in Go:
destinations_count = len(results)
topics = dedupe(flatMap(results, r => r.topics))

---

To refactor and avoid computing destinations_count and topics, this would be the updated schema.

{deploy}:tenant:{tenantID}:tenant [HASH]
├── id
├── created_at
├── updated_at
├── metadata
├── destinations_count ← stored, updated on create/delete
├── topics ← stored, recomputed on destination mutations
├── dest:dest_1 ← JSON(DestinationSummary)
├── dest:dest_2 ← JSON(DestinationSummary)
└── ...

This should be fairly doable I think, with the caveat above. Let me know what you think and we can certainly update this accordingly.

[alexbouchardd]

That's corrects and actually thought it had already been implemented without fetching the destinations. My bad I must have missed it in the review.