feat: prune `InnerForest` by drahnr · Pull Request #1635 · 0xMiden/node

drahnr · 2026-02-03T14:44:23Z

Important

Targeting main

What

Does cleanup the InnerForest, both the lookup tables/maps and the actual SmtForest.

Required for bounding the in-memory size growth.

Context

Related #1175

How

There are a few requirements:

loading from DB is rather expensive
- limit maximum loaded storage map slot entries from DB or return limit exceeded
- use an LRU cache for the entries to avoid DB lookups for the recent history
forest cleanup

in order to cleanup effectively we need store additional meta info for each of vault and storage map slots
- per block changes of accounts -> avoid full iteration over all accounts
- forest does deduplication of leaves, but does not refcount roots -> need to do this manually, we use *_refcount tables
retain the latest version of a storage map root, just like we do with the DB -> required for partial queries to create smt proofs / witnesses

Caveats

For proofs we only need the roots, not all entries of individual storage slots
We need storage map slot map entries only if we get a request for ::AllEntries , the SmtProof contains the relevant SmtLeaf itself
- we don't have a way to access all entries from the current SmtForest for a specific key

Copilot

Pull request overview

This PR implements periodic pruning of the InnerForest in-memory data structures to bound memory growth, addressing issue #1175 about pruning account storage maps and vault tables.

Changes:

Added pruning mechanism that retains only the last 50 blocks (configurable via HISTORICAL_BLOCK_RETENTION constant) of account vault and storage map data
Introduced block-indexed tracking structures (vault_roots_by_block and storage_slots_by_block) to efficiently identify entries eligible for pruning
Refactored state/mod.rs to cache block.body() calls and improve code readability

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
crates/store/src/inner_forest/mod.rs	Added pruning logic with new tracking data structures, `HISTORICAL_BLOCK_RETENTION` constant, and helper methods to prune old vault and storage map roots
crates/store/src/inner_forest/tests.rs	Added comprehensive test coverage for pruning functionality, including edge cases like empty forest, young chains, boundary conditions, and multiple accounts/slots
crates/store/src/state/mod.rs	Refactored to cache `block.body()` result and renamed `block_data` to `block_bytes` for clarity; split forest write lock acquisition into separate line
crates/store/src/db/tests.rs	Renamed test function to follow consistent naming convention using `db_roundtrip_` prefix
CHANGELOG.md	Added enhancement entry for periodic cleanup feature

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

crates/store/src/inner_forest/mod.rs

CHANGELOG.md

crates/store/src/inner_forest/mod.rs

crates/store/src/inner_forest/tests.rs

crates/store/src/inner_forest/mod.rs

Copilot

Pull request overview

Copilot reviewed 60 out of 81 changed files in this pull request and generated 6 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Cargo.toml

crates/store/src/inner_forest/mod.rs

crates/store/src/server/block_producer.rs

crates/store/src/state/apply_block.rs

crates/store/src/inner_forest/mod.rs

crates/rpc/src/tests.rs

drahnr · 2026-02-06T16:51:44Z

Found one more bug, when using RPC with response ::AllEntries when the last updates was older than history depth kept.

The large changeset is mostly due to tests added.

igamigo

Overall LGTM, but left a couple of comments. Not sure if they are fully correct (or if they land within the scope of this PR exactly), but worth reviewing for the long term at least.

crates/store/src/inner_forest/mod.rs

crates/store/Cargo.toml

crates/store/src/state/mod.rs

crates/store/src/inner_forest/mod.rs

Mirko-von-Leipzig · 2026-02-10T09:09:07Z

@bobbinth it looks like the main branch still has the CI/test related protections on it (unless this is just some sort of github artifact)

crates/store/src/inner_forest/mod.rs

crates/store/src/db/mod.rs

Copilot

Pull request overview

Copilot reviewed 10 out of 11 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

crates/store/src/db/mod.rs

crates/store/src/state/mod.rs

…r-forest-for-main

Copilot

Pull request overview

Copilot reviewed 10 out of 11 changed files in this pull request and generated 5 comments.

Comments suppressed due to low confidence (1)

crates/store/src/state/mod.rs:1149

In the loop starting at line 1109, if a SlotData::All request is followed by a SlotData::MapKeys request and the MapKeys request fails (e.g., returns None at line 1112 or an error at line 1123), the forest_guard that was re-acquired at line 1143 will still be held when the error is returned. This keeps the write lock on the forest unnecessarily.

While Rust's RAII will drop the guard eventually, it's better to explicitly drop it before returning errors to avoid holding the lock during error handling. Consider wrapping the MapKeys logic in a scope that ensures the guard is dropped, or use an early break pattern.

        for StorageMapRequest { slot_name, slot_data } in storage_requests {
            let details = match &slot_data {
                SlotData::MapKeys(keys) => forest_guard
                    .get_storage_map_details_for_keys(
                        account_id,
                        slot_name.clone(),
                        block_num,
                        keys,
                    )
                    .ok_or_else(|| DatabaseError::StorageRootNotFound {
                        account_id,
                        slot_name: slot_name.to_string(),
                        block_num,
                    })?
                    .map_err(DatabaseError::MerkleError)?,
                SlotData::All => {
                    // we don't want to hold the forest guard for a prolonged time
                    drop(forest_guard);
                    // we collect all storage items, if the account is small enough or
                    // return `AccountStorageMapDetails::LimitExceeded`
                    let details = self
                        .db
                        .reconstruct_storage_map_from_db(
                            account_id,
                            slot_name.clone(),
                            block_num,
                            Some(
                                // TODO unify this with
                                // `AccountStorageMapDetails::MAX_RETURN_ENTRIES`
                                // and accumulated the limits
                                <QueryParamStorageMapKeyTotalLimit as QueryParamLimiter>::LIMIT,
                            ),
                        )
                        .await?;
                    forest_guard = self.forest.write().await;
                    details
                },
            };

            storage_map_details.push(details);
        }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Cargo.toml

crates/store/src/db/mod.rs

crates/store/src/inner_forest/mod.rs

crates/store/src/db/mod.rs

…r-forest-for-main

drahnr · 2026-02-25T10:50:08Z

crates/store/src/db/mod.rs

+        use miden_protocol::EMPTY_WORD;
+
+        // TODO this remains expensive with a large history until we implement pruning for DB
+        // columns


It's already done in next

drahnr · 2026-02-25T12:05:52Z

crates/store/src/inner_forest/mod.rs

+        TreeId::new(lineage, block_num.as_u64())
+    }
+
+    fn storage_lineage_id(account_id: AccountId, slot_name: &StorageSlotName) -> LineageId {


Determines the LineageId for storage slots

I don't know what a LineageId is?

Mirko-von-Leipzig · 2026-02-25T14:47:21Z

CHANGELOG.md


 ### Enhancements

+- Added cleanup of old account data from the in-memory forest ([#1175](https://github.com/0xMiden/miden-node/issues/1175)).


Suggested change

- Added cleanup of old account data from the in-memory forest ([#1175](https://github.com/0xMiden/miden-node/issues/1175)).

Mirko-von-Leipzig · 2026-02-25T14:54:47Z

crates/store/src/state/mod.rs

+                    match result {
+                        Ok(details) => details,
+                        Err(err) => {
+                            drop(forest_guard);


Why do you need to manually drop the guard?

Mirko-von-Leipzig · 2026-02-25T14:59:13Z

crates/store/src/state/mod.rs


        // Use forest for storage map queries
-        let forest_guard = self.forest.read().await;
+        let mut forest_guard = self.forest.write().await;


Why do we need write access? The only method used is get_ -- is that mutable?

Mirko-von-Leipzig · 2026-02-25T15:02:26Z

crates/store/src/state/mod.rs

+                SlotData::All => {
+                    // we don't want to hold the forest guard for a prolonged time
+                    drop(forest_guard);


Is it possible to split the request into a forest list, and a database list?

As in:

let with_keys = storage_requests.iter().filter(SlotData::MapKeys); let with_all = storage_requests.iter().filter(SlotData::All); let mut forest_guard = ..; for (slot, keys) in with_keys { forest_guard.get_storage... } drop(forest_guard); for slot in with_all { self.db.reconstruct(...) }

Yes, we can.

Mirko-von-Leipzig · 2026-02-25T15:05:24Z

crates/store/src/inner_forest/mod.rs

+            .checked_sub(HISTORICAL_BLOCK_RETENTION)
+            .unwrap_or(BlockNumber::GENESIS);


I guess we don't have BlockNumber::saturating_sub?

We don't unfortunately

Mirko-von-Leipzig · 2026-02-25T15:05:48Z

crates/store/src/inner_forest/mod.rs

+    ///
+    /// The `LargeSmtForest` itself is truncated to drop historical versions beyond the cutoff.
+    ///
+    /// Returns the number of _roots_ that got pruned.


Is this important?

It's not, but I'd rather have an indication how many we prune than flying blind.

Mirko-von-Leipzig · 2026-02-25T15:06:22Z

crates/store/src/inner_forest/mod.rs

+        TreeId::new(lineage, block_num.as_u64())
+    }
+
+    fn storage_lineage_id(account_id: AccountId, slot_name: &StorageSlotName) -> LineageId {


I don't know what a LineageId is?

Mirko-von-Leipzig · 2026-02-25T15:09:51Z

crates/store/src/db/mod.rs

 use crate::errors::{DatabaseError, DatabaseSetupError, NoteSyncError, StateSyncError};
 use crate::genesis::GenesisBlock;

+const ROW_OVERHEAD_BYTES: usize = 2 * size_of::<Word>() + size_of::<u32>() + size_of::<u8>();


Overhead of what?

drahnr requested review from Mirko-von-Leipzig, bobbinth and Copilot and removed request for Mirko-von-Leipzig February 3, 2026 14:44

Copilot started reviewing on behalf of drahnr February 3, 2026 14:46 View session

drahnr requested a review from igamigo February 3, 2026 14:51

Copilot AI reviewed Feb 3, 2026

View reviewed changes

crates/store/src/inner_forest/mod.rs Outdated Show resolved Hide resolved

CHANGELOG.md Outdated Show resolved Hide resolved

Mirko-von-Leipzig approved these changes Feb 6, 2026

View reviewed changes

crates/store/src/inner_forest/mod.rs Outdated Show resolved Hide resolved

crates/store/src/inner_forest/mod.rs Outdated Show resolved Hide resolved

crates/store/src/inner_forest/tests.rs Show resolved Hide resolved

Mirko-von-Leipzig reviewed Feb 6, 2026

View reviewed changes

crates/store/src/inner_forest/mod.rs Show resolved Hide resolved

drahnr requested a review from Copilot February 6, 2026 13:59

Copilot started reviewing on behalf of drahnr February 6, 2026 13:59 View session

Copilot AI reviewed Feb 6, 2026

View reviewed changes

drahnr force-pushed the bernhard-cleanup-inner-forest-for-main branch 2 times, most recently from 72dad48 to a3be45b Compare February 6, 2026 16:48

igamigo reviewed Feb 9, 2026

View reviewed changes

crates/store/src/inner_forest/mod.rs Outdated Show resolved Hide resolved

crates/store/src/inner_forest/mod.rs Outdated Show resolved Hide resolved

Mirko-von-Leipzig self-requested a review February 9, 2026 18:30

Mirko-von-Leipzig reviewed Feb 10, 2026

View reviewed changes

crates/store/Cargo.toml Show resolved Hide resolved

crates/store/src/state/mod.rs Show resolved Hide resolved

crates/store/src/inner_forest/mod.rs Outdated Show resolved Hide resolved

drahnr force-pushed the bernhard-cleanup-inner-forest-for-main branch from 35b80c3 to 098c805 Compare February 10, 2026 10:54

drahnr marked this pull request as draft February 10, 2026 15:36

drahnr marked this pull request as ready for review February 11, 2026 14:12

igamigo reviewed Feb 13, 2026

View reviewed changes

crates/store/src/inner_forest/mod.rs Outdated Show resolved Hide resolved

igamigo reviewed Feb 13, 2026

View reviewed changes

crates/store/src/db/mod.rs Outdated Show resolved Hide resolved

igamigo reviewed Feb 13, 2026

View reviewed changes

crates/store/src/db/mod.rs Show resolved Hide resolved

drahnr added 4 commits February 16, 2026 18:57

y

4d0c39b

yay

0c96836

y

edeb505

yes

74efb5a

drahnr added 2 commits February 16, 2026 18:58

mepty root

b201602

fix test

e2c5289

drahnr force-pushed the bernhard-cleanup-inner-forest-for-main branch from 3e0f462 to e2c5289 Compare February 16, 2026 18:04

drahnr requested a review from Copilot February 16, 2026 18:11

Copilot started reviewing on behalf of drahnr February 16, 2026 18:12 View session

Copilot AI reviewed Feb 16, 2026

View reviewed changes

crates/store/src/db/mod.rs Outdated Show resolved Hide resolved

crates/store/src/db/mod.rs Outdated Show resolved Hide resolved

crates/store/src/state/mod.rs Outdated Show resolved Hide resolved

drahnr added 2 commits February 16, 2026 20:37

hashing keys fun, deliberate naming

a938411

add test to ensure raw and hashed keys are disambiguated

68b4b04

drahnr force-pushed the bernhard-cleanup-inner-forest-for-main branch from 0e269cb to 68b4b04 Compare February 17, 2026 13:38

drahnr mentioned this pull request Feb 17, 2026

chore: backport LargeSmtForest to 0.19.x branch 0xMiden/crypto#834

Merged

drahnr added 4 commits February 18, 2026 15:54

Merge remote-tracking branch 'origin/main' into bernhard-cleanup-inne…

0743c37

…r-forest-for-main

migrate to LargeSmtForest

1c9bb4f

fixes

774c2fd

remove lru dep

63d4deb

drahnr requested a review from Copilot February 24, 2026 14:51

Copilot started reviewing on behalf of drahnr February 24, 2026 14:51 View session

Copilot AI reviewed Feb 24, 2026

View reviewed changes

drahnr added 5 commits February 24, 2026 17:09

fixup

65bc39d

fix names

843dcbe

fmt

cb18729

clippy

b9f2a45

Merge remote-tracking branch 'origin/main' into bernhard-cleanup-inne…

cb6cf7e

…r-forest-for-main

drahnr commented Feb 25, 2026

View reviewed changes

docs update

899fccb

drahnr commented Feb 25, 2026

View reviewed changes

drahnr added 2 commits February 25, 2026 13:33

self review, again

3ade4d6

cleanup

afcf072

drahnr mentioned this pull request Feb 25, 2026

perf/store: optimize updating existing accounts #1567

Open

Mirko-von-Leipzig reviewed Feb 25, 2026

View reviewed changes


		### Enhancements

		- Added cleanup of old account data from the in-memory forest ([#1175](https://github.com/0xMiden/miden-node/issues/1175)).

		.checked_sub(HISTORICAL_BLOCK_RETENTION)
		.unwrap_or(BlockNumber::GENESIS);

Comments

Conversation

drahnr commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Context

How

Caveats

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

drahnr commented Feb 6, 2026

Uh oh!

igamigo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Mirko-von-Leipzig commented Feb 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

drahnr commented Feb 3, 2026 •

edited

Loading

Mirko-von-Leipzig commented Feb 10, 2026 •

edited

Loading