feat: add includeTotal option to pagination

[wschurman]

Why

A frequent operation that is needed during pagination is computation of the total number of pages. Some relay-style paginators (connections, edges) expose a totalCount field on the Connection type.

This PR adds this feature. Note that I'm not a strong proponent for this, but claude is and it wouldn't stop trying to implement it or suggest it in code review throughout this process. So we built it, but I'm open to not landing it if we think it'll be abused (query on every page, for example). Claude and my research seem to think that it makes sense and is aligned with other libraries like apollo.

How

See docblock for includeTotal for summary, but at a high level this does either a postgres window function or a count(*) to get the count. Window functions are more efficient since the postgres execution can fetch that information in the same query, but they only work when cursors are not used (otherwise the window function would apply over the rows selected by the cursor condition).

Test Plan

Full test coverage, many tests.

[wschurman]

• feat: Add ilike and trigram similarity search to pagination #431
• feat: add includeTotal option to pagination #430 👈 (View in Graphite)
• feat: Add paginated loader to entity-database-adapter-knex #422
• fix: move EntityPrivacyUtils back into core package #429
• fix: Type idField in EntityConfiguration as TIDField #425
• fix: Use TSelectedFields for knex loader order by method #424
• feat!: Add sql template and loader method #414
• feat: add codemod for upcoming v0.55.0-v0.56.0 #413
• feat: add entity-database-adapter-knex-testing-utils containing StubPostgresDatabaseAdapter #412
• chore: add CLAUDE.md and disallow test.skip #411
• feat!: move knex loaders into entity-database-adapter-knex package #410
• chore: move knex-specific loader logic out of EntityDataManager #408 : 1 other dependent PR (#409 )
• feat!: Move knex-specific loader methods to knexLoader #407
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (58deffd) to head (d3a2418).

Additional details and impacted files

@@            Coverage Diff             @@
##              main      #430    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files          108       108            
  Lines        15606     15882   +276     
  Branches      1384      1412    +28     
==========================================
+ Hits         15606     15882   +276     

Flag	Coverage Δ
integration	26.00% <85.27%> (+1.03%)	⬆️
unittest	95.91% <57.53%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[ide]

I am skeptical of this feature for a few reasons. They can be addressed but I'll write them out here:

1. We don't use this feature currently. If an LLM keeps wanting to add it, we can add a CLAUDE.md saying not to suggest it.
2. This could be problematically slow depending on the number of rows needed to be scanned. I'm thinking we'd want a way to cap the count. The UX would show the number of pages if it's under, say, 20, but otherwise just show "more".
3. Refetching the count during pagination is wasteful. Only the initial query (which isn't necessarily for page 1) needs the count, and then incremental loads should not compute it by default.

[wschurman]

I think we're in agreement. I could imagine in the future exposing a loaderMethod, countForSQLAsync or something so that it is separate from (3). But 1 and 2 are definitely still concerns. We do use count in a few places in the Expo backend (count of apps for an account), but it's definitely not the majority case for paginated interfaces.

Closing for now.