Run a Substreams
Running a Substreams means executing a Substreams package and consuming the data emitted. The data produced by a Substreams can be consumed in a variety of ways, such as using the CLI, a SQL database or a simple files.
Running a Local Project
A typical Substreams local project contains Protobufs (1
), modules (2
) and a manifest (substreams.yaml
) file (3
):
If you are currently working on a local project, you can run your Substreams by specifying the location of the manifest:
The --start-block
flag specifies the starting block of the Substreams (i.e. at which block the Substreams will start indexing data)
Note: In Solana, --start-block
specifies the slot, not the block numnber.
Substreams run
run
First, start the substreams
CLI passing it a run
command.
Firehose URI
The server address is required by Substreams to connect to for data retrieval. The data provider for Substreams is located at the address, which is a running Firehose instance.
-e mainnet.eth.streamingfast.io:443
Substreams YAML configuration file
Inform Substreams where to find the substreams.yaml
configuration file.
Note: The substreams.yaml
configuration file argument in the command is optional if you are within the root folder of your Substreams and your manifest file is named `substreams.yaml.
Module
The map_transfers
module is defined in the manifest and it is the module run by Substreams.
Block mapping
Start mapping at the specific block 12292922
by using passing the flag and block number --start-block 12292922
.
Cease block processing by using --stop-block +1.
The +1
option requests a single block. In the example, the next block is 12292923
.
Successful Substreams results
Messages are printed to the terminal for successfully installed and configured Substreams setups.
The substreams
run
command outputs:
Running a Substreams Package (.spkg)
If you have an spkg
file, you can run it by providing a path to the file:
Development and production mode
Substreams has two mode when executing your module(s) either development mode or production mode. Development and production modes impact the execution of Substreams, important aspects of execution include:
The time required to reach the first byte.
The speed that large ranges get executed.
The module logs and outputs sent back to the client.
Differences
Differences between production and development modes include:
Forward parallel execution is enabled in production mode and disabled in development mode
The time required to reach the first byte in development mode is faster than in production mode.
Specific attributes of development mode include:
The client will receive all of the executed module's logs.
It's possible to request specific store snapshots in the execution tree.
Multiple module's output is possible.
Examples
In most cases, you will run production mode, using a Substreams sink. Development mode is enabled by default in the CLI unless the -p
flag is specified.
Examples: (given the dependencies: [block] --> [map_pools] --> [store_pools] --> [map_transfers])
Running the
substreams run substreams.yaml map_transfers
command executes in development mode and only prints themap_transfers
module's outputs and logs.Running the
substreams run substreams.yaml map_transfers --debug-modules-output=map_pools,map_transfers,store_pools
command executes in development mode and only prints the outputs of themap_pools
,map_transfers
, andstore_pools
modules.Running the
substreams run substreams.yaml map_transfers -s 1000 -t +5 --debug-modules-initial-snapshot=store_pools
command executes in development mode and prints all the entries in thestore_pools
module at block 999, then continues with outputs and logs from themap_transfers
module in blocks 1000 through 1004.
Last updated