Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jan 28, 2026

Problem

When reload() is called with restat=False and no stats/counts files provided, existing stats/counts tables are not backed up to _old{n} and new empty tables are not created, causing statistics data loss.

Changes

Decouple stats table backup from stats refresh logic

  • Always include stats/counts tables in swap operation when stats.saving=True
  • Create empty _tmp versions if they don't exist
  • Only refresh stats when restat=True

Previously, stats tables were only added to the backup list when restat=True, causing them to be excluded from the swap entirely when a user explicitly disabled stat refresh.

# Before: stats tables only backed up when restat=True
if restat and self.stats.saving:
    for table in [self.stats.counts, self.stats.stats]:
        if not self._table_exists(table + suffix):
            self._clone(table, table + suffix)
    if countsfile is None or statsfile is None:
        self.stats.refresh_stats(suffix=suffix)
    for table in [self.stats.counts, self.stats.stats]:
        if table not in tables:
            tables.append(table)

# After: backup always happens, refresh only when requested
if self.stats.saving:
    for table in [self.stats.counts, self.stats.stats]:
        if table not in tables:
            if not self._table_exists(table + suffix):
                self._clone(table, table + suffix)
            tables.append(table)
    
    if restat and (countsfile is None or statsfile is None):
        self.stats.refresh_stats(suffix=suffix)

This ensures old statistics are preserved as _old{n} and new empty tables are created during reload, regardless of whether stats are being refreshed.

Original prompt

This section details on the original issue you should resolve

<issue_title>Reload doesn't seem to save the old stats, and doesn't create a new empty one.</issue_title>
<issue_description>```psql
lmfdb=# SELECT
n.nspname AS schema_name,
c.relname AS table_name,
pg_size_pretty(pg_total_relation_size(c.oid)) AS total_size,
pg_size_pretty(pg_relation_size(c.oid)) AS table_heap_size,
pg_size_pretty(pg_indexes_size(c.oid)) AS indexes_size
FROM pg_class c
JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE c.relkind IN ('r', 'p') -- ordinary tables + partitioned tables
AND c.relname ILIKE 'nf_fields_%'
AND n.nspname NOT IN ('pg_catalog', 'information_schema')
ORDER BY pg_total_relation_size(c.oid) DESC;
schema_name | table_name | total_size | table_heap_size | indexes_size
-------------+-------------------------+------------+-----------------+--------------
public | nf_fields_extras | 89 GB | 34 GB | 475 MB
public | nf_fields_extras_old1 | 89 GB | 34 GB | 475 MB
public | nf_fields_extra | 89 GB | 33 GB | 1245 MB
public | nf_fields_old1 | 34 GB | 15 GB | 20 GB
public | nf_fields_reflex | 459 MB | 397 MB | 61 MB
public | nf_fields_counts | 12 MB | 4624 kB | 7888 kB
public | nf_fields_oldstats | 104 kB | 8192 bytes | 0 bytes
public | nf_fields_stats | 56 kB | 16 kB | 0 bytes
public | nf_fields_extra_stats | 8192 bytes | 0 bytes | 0 bytes
public | nf_fields_extra_old1 | 8192 bytes | 0 bytes | 0 bytes
public | nf_fields_reflex_stats | 8192 bytes | 0 bytes | 0 bytes
public | nf_fields_reflex_counts | 8192 bytes | 0 bytes | 0 bytes
public | nf_fields_extra_counts | 8192 bytes | 0 bytes | 0 bytes
(13 rows)


## Comments on the Issue (you are @copilot in this section)

<comments>
</comments>


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 2 commits January 28, 2026 19:02
Co-authored-by: edgarcosta <322822+edgarcosta@users.noreply.github.com>
Co-authored-by: edgarcosta <322822+edgarcosta@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix reload function to save old stats and create new one Fix reload() to always backup stats tables when stats.saving enabled Jan 28, 2026
Copilot AI requested a review from edgarcosta January 28, 2026 19:08
@edgarcosta edgarcosta marked this pull request as ready for review January 28, 2026 19:13
Copy link
Collaborator

@edgarcosta edgarcosta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reload doesn't seem to save the old stats, and doesn't create a new empty one.

2 participants