SDK 1.2 by mo4islona · Pull Request #426 · subsquid/squid-sdk

mo4islona · 2026-01-15T09:04:43Z

No description provided.

Copilot

Pull request overview

This PR introduces "SDK 1.2", a major refactoring that migrates the Solana and EVM data streaming architecture from the old Gateway/Archive + RPC approach to a unified SQD Network Portal-based system. The changes include new packages for EVM streams/objects, a new shared data source abstraction, updates to the batch processor to support hot/unfinalized blocks, and comprehensive updates to example code.

Changes:

Introduces new packages: @subsquid/util-internal-data-source, @subsquid/evm-stream, and @subsquid/evm-objects
Refactors @subsquid/solana-stream to use Portal client instead of Gateway + RPC
Updates @subsquid/batch-processor to support hot blocks and fork handling
Renames slot/height fields to standardized number field across Solana packages

Reviewed changes

Copilot reviewed 63 out of 64 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
util/util-internal-data-source/*	New package defining shared DataSource abstraction with BlockRef, fork handling, and stream interfaces
solana/solana-stream/src/source.ts	Major refactor from Gateway+RPC to Portal-only architecture with simplified API
solana/solana-stream/src/portal/*	New Portal-based data source implementation replacing old archive/RPC sources
evm/evm-stream/*	New package for EVM data streaming using Portal client
evm/evm-objects/*	New package providing augmented EVM data model with relationships
processor/batch-processor/src/run.ts	Enhanced to support hot blocks, fork navigation, and finalized/unfinalized streams
processor/batch-processor/src/database.ts	Updated interface to support hot block databases with fork management
test/erc20-transfers/src/processor.ts	Updated example to use new EVM stream API
test/solana-example/src/main.ts	Updated example to use new Portal-based API
solana/solana-normalization/src/*	Renamed `slot` to `number` for consistency
solana/solana-objects/src/*	Updated to use `number` instead of `height`

Files not reviewed (1)

common/config/rush/pnpm-lock.yaml: Language not supported

Comments suppressed due to low confidence (2)

solana/solana-stream/src/portal/schema.ts:134

Unnecessary parentheses around the array() calls. These can be removed for cleaner code: transactions: array(Transaction), etc.
evm/evm-stream/src/data/model.ts:1
The height field is marked as deprecated in favor of number, but both fields have the same value (line 145: height: number,). This creates confusion as users might not understand which to use. Consider adding a JSDoc comment explaining that height is maintained for backwards compatibility and equals number.

import type {

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-15T09:07:32Z

solana/solana-stream/src/source.ts

     *
     * @example
-     * source.setGateway('https://v2.archive.subsquid.io/network/solana-mainnet')
+     * source.setGateway('https://portal.sqd.dev/datasets/solana-mainnet')


The JSDoc example still references the old method name setGateway but the method has been renamed to setPortal. Update the example to use source.setPortal('https://portal.sqd.dev/datasets/solana-mainnet').

Copilot · 2026-01-15T09:07:32Z

util/util-internal-data-source/src/index.ts

+    // FIXME: maybe it's better to pass it ias an option to `getStream`
+    getFinalizedStream(req: DataSourceStreamOptions): AsyncIterable<BlockBatch<B>>


Corrected spelling of 'ias' to 'as'.

Copilot · 2026-01-15T09:07:33Z

evm/evm-objects/src/types.ts

@@ -0,0 +1,91 @@
+import {Bytes, FieldSelection} from '@subsquid/evm-stream'


Unused import Bytes.

Suggested change

import {Bytes, FieldSelection} from '@subsquid/evm-stream'

import {FieldSelection} from '@subsquid/evm-stream'

Copilot · 2026-01-15T09:07:33Z

evm/evm-objects/src/items.ts

@@ -0,0 +1,400 @@
+import {Bytes, Bytes20, Bytes32, Bytes8} from '@subsquid/evm-stream'


Unused import Bytes8.

Suggested change

import {Bytes, Bytes20, Bytes32, Bytes8} from '@subsquid/evm-stream'

import {Bytes, Bytes20, Bytes32} from '@subsquid/evm-stream'

Copilot · 2026-01-15T09:07:34Z

evm/evm-objects/src/items.ts

+import {
+    EIP7702Authorization,
+    EvmTraceCallAction,
+    EvmTraceCallResult,
+    EvmTraceCreateAction,
+    EvmTraceCreateResult,
+    EvmTraceRewardAction,
+    EvmTraceSuicideAction
+} from '@subsquid/evm-stream/lib/data/evm'


Unused import EIP7702Authorization.

Copilot · 2026-01-15T09:07:34Z

evm/evm-stream/src/data/fields.ts

+import {DEFAULT_FIELDS, FieldSelection} from './model'
+import {Selector} from './type-util'
+
+
+function merge<Keys extends string>(def: Selector<Keys>, requested: Selector<Keys> = {}): Selector<Keys> {
+    let fields: Selector<Keys> = {}
+


Unused imports DEFAULT_FIELDS, FieldSelection.

Suggested change

import {DEFAULT_FIELDS, FieldSelection} from './model'

import {Selector} from './type-util'

function merge<Keys extends string>(def: Selector<Keys>, requested: Selector<Keys> = {}): Selector<Keys> {

let fields: Selector<Keys> = {}

import {Selector} from './type-util'

function merge<Keys extends string>(def: Selector<Keys>, requested: Selector<Keys> = {}): Selector<Keys> {

let fields: Selector<Keys> = {}

Copilot · 2026-01-15T09:07:34Z

evm/evm-stream/src/data/fields.ts

+
+function merge<Keys extends string>(def: Selector<Keys>, requested: Selector<Keys> = {}): Selector<Keys> {
+    let fields: Selector<Keys> = {}
+
+    for (let key in def) {
+        if (requested[key] !== false) {
+            fields[key] = def[key]
+        }
+    }
+
+    for (let key in requested) {
+        if (requested[key]) {
+            fields[key] = true
+        }
+    }
+
+    return fields
+}
+
+


Unused function merge.

Suggested change

function merge<Keys extends string>(def: Selector<Keys>, requested: Selector<Keys> = {}): Selector<Keys> {

let fields: Selector<Keys> = {}

for (let key in def) {

if (requested[key] !== false) {

fields[key] = def[key]

}

}

for (let key in requested) {

if (requested[key]) {

fields[key] = true

}

}

return fields

}

Copilot · 2026-01-15T09:07:35Z

evm/evm-stream/src/portal/schema.ts

+import {
+    ANY,
+    array,
+    ANY_INT,
+    BOOLEAN,
+    BYTES,
+    constant,
+    NAT,
+    object,
+    oneOf,
+    option,
+    STRING
+} from '@subsquid/util-internal-validation'


Unused imports ANY, BOOLEAN.

Copilot · 2026-01-15T09:07:35Z

processor/batch-processor/src/run.ts

 import * as prom from 'prom-client'
-import {Database, HashAndHeight} from './database'
-import {DataSource} from './datasource'
+import {HashAndHeight, Database, HotDatabaseState} from './database'


Unused import HotDatabaseState.

Copilot · 2026-01-15T09:07:35Z

util/util-internal-data-source/src/index.ts

@@ -0,0 +1,77 @@
+import assert from 'assert'
+import {Range} from '@subsquid/util-internal-range'


Unused import Range.

- Improved stream handling by breaking the loop upon successful completion. - Simplified metrics update calculations. - Enhanced data continuity validation by removing unnecessary checks. - Updated transaction handling to use state defaults when necessary.

belopash and others added 6 commits December 8, 2025 14:19

reinit branch

00cf2ec

changes

436d1b5

Update changelogs [skip ci]

285eace

Bump versions [skip ci]

7dd0d41

Merge branch 'master' into portal-api

800339b

Merge branch 'master' into portal-api

a8e10e0

Copilot AI review requested due to automatic review settings January 15, 2026 09:04

mo4islona marked this pull request as draft January 15, 2026 09:05

Copilot started reviewing on behalf of mo4islona January 15, 2026 09:05 View session

Copilot AI reviewed Jan 15, 2026

View reviewed changes

belopash added 5 commits January 29, 2026 16:20

Merge branch 'master' into portal-api

56b1f23

Merge branch 'master' into portal-api

962b0a1

refactor batch processing logic

97c8ad4

fix package version

d0738f6

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SDK 1.2#426

SDK 1.2#426
mo4islona wants to merge 11 commits intomasterfrom
portal-api

mo4islona commented Jan 15, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 15, 2026

Uh oh!

Copilot AI Jan 15, 2026

Uh oh!

Copilot AI Jan 15, 2026

Uh oh!

Copilot AI Jan 15, 2026

Uh oh!

Copilot AI Jan 15, 2026

Uh oh!

Copilot AI Jan 15, 2026

Uh oh!

Copilot AI Jan 15, 2026

Uh oh!

Copilot AI Jan 15, 2026

Uh oh!

Copilot AI Jan 15, 2026

Uh oh!

Copilot AI Jan 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		// FIXME: maybe it's better to pass it ias an option to `getStream`
		getFinalizedStream(req: DataSourceStreamOptions): AsyncIterable<BlockBatch<B>>

		@@ -0,0 +1,91 @@
		import {Bytes, FieldSelection} from '@subsquid/evm-stream'

	import {Bytes, FieldSelection} from '@subsquid/evm-stream'
	import {FieldSelection} from '@subsquid/evm-stream'

		@@ -0,0 +1,400 @@
		import {Bytes, Bytes20, Bytes32, Bytes8} from '@subsquid/evm-stream'

	import {Bytes, Bytes20, Bytes32, Bytes8} from '@subsquid/evm-stream'
	import {Bytes, Bytes20, Bytes32} from '@subsquid/evm-stream'

		@@ -0,0 +1,77 @@
		import assert from 'assert'
		import {Range} from '@subsquid/util-internal-range'

Conversation

mo4islona commented Jan 15, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants