The forester

The forester is the service that empties the staging buffers: it takes the keys waiting in a full buffer and inserts them into the matching indexed tree, twelve at a time, each batch with a proof that the insertion was done correctly. Anyone may run one. None is deployed, and without one the pool stops accepting spends once its nullifier buffers fill.

What it doesCopy Anchor Link

Each pool index has three staging buffers, which the code calls turnstiles, and the forester drains all three. They work the same way and differ only in what they hold and where it goes.

The nullifier staging bufferCopy Anchor Link

Holds the nullifiers of spent notes, three from every value transaction and one from every escape hatch. The forester inserts them into the nullifier tree with forester_commit. When it fills, value transactions in that pool index stop, as described below.

The registration staging bufferCopy Anchor Link

Holds newly registered identity commitments, one per registration. The forester inserts them into the registration tree with registration_forester_commit, which is what lets later registration proofs show a commitment is new.

The fee staging bufferCopy Anchor Link

Holds the nullifiers of collected fee notes. The forester inserts them into the fee-nullifier tree with fee_forester_commit.

One batch, start to finishCopy Anchor Link

A batch is twelve keys, and every batch goes through the same two phases. The first chart shows the forester gathering what it needs and checking it against the chain.

sequenceDiagram
participant F as Forester
participant P as Shielded pool
participant I as Indexer
F->>P: read staging buffer account
P-->>F: next batch, its stored keys, root
F->>I: fetch staged keys and tree
I-->>F: 12 keys, tree snapshot
F->>F: check both fold to one chain

The forester reads the staging buffer's account to find the full buffer, the next batch of 12, the 12 keys the account stores for that batch, the tree's current root and its next free index. It folds the stored keys into the batch's hash chain itself, as the program will when the batch commits. It fetches the staged keys from the indexer's archive and a snapshot of the indexed tree. Before proving anything, it checks that the archive's 12 keys fold to that same chain, so it never spends a fee on a proof that cannot verify.

The second chart shows the proof and the commit.

sequenceDiagram
participant F as Forester
participant P as Shielded pool
F->>F: prove 12 insertions
F->>P: commit new root and proof
P->>P: verify against its own state
P->>P: publish root, advance index

The forester builds the witness from its own copy of the tree, proves the batch and submits it, up to three batches per Solana transaction. The program checks the proof against values it takes from its own account, publishes the new root and advances the tree's next free index by 12.

The proofCopy Anchor Link

The batch-append circuit proves twelve sequential insertions into an indexed tree of the shape all three staging buffers share, eleven levels of 8 and one of 8, for 2³⁶ leaves. For each key it shows the leaf just below the key in sorted order, updates that leaf to point at the new one, and appends the new leaf at the next index; each insertion starts from the root the previous one left. The key must fall strictly between that leaf's value and its next value, so a key already in the tree cannot be inserted again. Its one public input is

H_on(old root, new root, keys chain, start index)

and the program builds that value itself from its own state: the old root is the tree's current root, the keys chain is the hash chain the program folds from the batch's 12 stored keys, and the start index is the tree's next free index. The forester supplies only the new root and the proof. If the proof verifies, the program publishes the new root, advances the index by 12 and counts the batch. When a buffer's 2,500th batch lands, the buffer is marked inserted and later retired, as described on the double-spend page.

Permissionless, and what that allowsCopy Anchor Link

No account signs a commit. The proof is the only authority, so anyone with the proving key and the staged keys can run a forester, and several can run at once. Commits must arrive in order, because each one must start from the current root: when two foresters race for the same batch, the first commit wins and the second fails because its old root no longer matches.

A forester can only insert exactly the keys that were staged, in the order they were staged, into the tree they belong to. It cannot add a key, drop one, reorder them or insert them wrongly, because the hash chain and the proof would not match. What it can do is not show up. A forester's only power over the pool is to withhold commits.

When no forester runsCopy Anchor Link

Staging never waits for the forester until both buffers of a staging buffer are full. Each holds 30,000 keys, so each staging buffer takes 60,000 keys before it blocks. What stops then depends on which one filled:

• The nullifier staging buffer. Every value transaction in that pool index stops, deposits, private transfers (Private Send) and withdrawals alike, and so does every escape hatch, which also stages its nullifier. At three nullifiers per value transaction, that is roughly 20,000 value transactions after the last commit.
• The registration staging buffer. Every new registration in that pool index stops, after 60,000 registrations.
• The fee staging buffer. Every fee collection in that pool index stops, after 60,000 collected fee notes.

The escape hatch does not help here: it goes around the attesters, but not around the staging buffer. Nothing is lost while the pool is blocked. Staged keys stay staged, and once a forester commits a full buffer's batches, staging resumes.

A forester that runs but falls behind causes a smaller problem. The nullifier tree keeps only its last 512 roots, and each commit publishes one, so freshness proofs made against an older root are refused and must be made again. During a long drain that happens quickly.

What it needs to runCopy Anchor Link

• The batch-append proving key. The program compiles in one batch-append verifying key, shared by all three trees and all pool indexes. The forester loads its proving key and verifying key at start and refuses to run without them. It also refuses to run when that verifying key is not the batch-append key compiled into it, which is the program's key when both are built from the same source, because the program would refuse every proof it made.
• The staged keys and the tree. The forester reads keys from the indexer's archive and a tree snapshot from the indexer, and checks both against the chain before use. A forester with no indexer cannot build witnesses.
• A funded fee payer for the commit transactions.

RelatedCopy Anchor Link