Substreams CLI reference

StreamingFast Substreams command line interface (CLI)

substreams CLI reference overview

The substreams command line interface (CLI) is the primary user interface and the main tool for sending requests and receiving data.

The substreams CLI exposes many commands to developers enabling a range of features.

Note: When a package is specified, it is optional. If you do use it, you can use:

• Local substreams.yaml configuration files

• Local .spkg package files

• Remote .spkg package URLs

• Local directory containing a substreams.yaml file

If you choose to not use it, make sure that you are in a directory that contains a substreams.yaml file. Otherwise, you will get a usage error back.

init

The init command allows you to initialize a Substreams project for several blockchains. It is a conversational-like command: you will be asked several questions and a project with the specicied features will be created for you.

The options included in the init command will evolve over time, but every blockchain should, at least, contain one option.

substreams init

build

The build command:

• Generates the necessary Protobufs specified in the substreams.yaml file.

• Compiles the Rust code.

• Creates a Substreams package file (.spkg).

substreams build

run

The run command connects to a Substreams endpoint and begins processing data.

run command

substreams run -e mainnet.eth.streamingfast.io:443 \
   -t +1 \
   ./substreams.yaml \
   module_name

The details of the run command are:

• -e mainnet.eth.streamingfast.io:443 is the endpoint of the provider running your Substreams.

• -t +1 or --stop-block only requests a single block; the stop block is the manifest's initialBlock + 1.

• substreams.yaml is the path where you have defined your Substreams manifest. You can use a .spkg or substreams.yaml configuration file.

• module_name is the module we want to run, referring to the module name defined in the Substreams manifest.

Tip: Passing a different -s or --start-block runs prior modules at a higher speed. Output is provided at the requested start block, keeping snapshots along the way if you want to process it again.

Headers

The -H option in of the run or gui command allows you to dynamically pass headers with the gRPC request. This is useful when overriding default parameters in the Substreams execution.

X-Sf-Substreams-Parallel-Jobs Header

The X-Sf-Substreams-Parallel-Jobs header sets the number of parallel jobs to use in the Substreams execution. By default, 10 jobs are used.

substreams run -e mainnet.eth.streamingfast.io:443 \
   -t +1 \
   -H "X-Sf-Substreams-Parallel-Jobs: 20" \
   ./substreams.yaml \
   module_name

Run example with output

substreams run

$ substreams run -e mainnet.eth.streamingfast.io:443 \
    https://github.com/Jannis/gravity-substream/releases/download/v0.0.1/gravity-v0.1.0.spkg \
    gravatar_updates -o json

The output of the gravatar_updates module starting at block 6200807 will print a message resembling:

run output

{
  "updates": [
    {
      "id": "39",
      "owner": "0xaadcc13071fdf9c73cfbb8d97639ea68aa6fd1d2",
      "displayName": "alex | OpenSea",
      "imageUrl": "https://ucarecdn.com/13a67247-cb89-417a-92d2-50a7d7aa481c/-/crop/382x382/0,0/-/preview/"
    }
  ]
}
...

Note: The -o or --output flag alters the output format.

The available output display options are:

• ui, a nicely formatted, UI-driven interface, displaying progress information and execution logs.

• json, an indented stream of data, not displaying progress information or logs, only data output for blocks proceeding the start block.

• jsonl, same as json showing every individual output on a single line.

gui

The gui command pops up a terminal-based graphical user interface.

Its parameters are very similar to those of run, but provides a UI to navigate the results instead of a stream of data.

Replay mode

When you run a gui session, a file called replay.log gets written with the contents of the streamed data that persists after closing the GUI.

You can reload the data without hitting the server again using --replay. The data is immediately reloaded in the GUI, ready for more inspection.

pack (DEPRECATED)

(DEPRECATED: use build instead)

The pack command builds a shippable, importable package from a substreams.yaml manifest file.

pack command

$ substreams pack ./substreams.yaml

The output of the pack command will print a message resembling:

pack output

...
Successfully wrote "your-package-v0.1.0.spkg".

info

The info command prints out the contents of a package for inspection. It works on both local and remote yaml or spkg configuration files.

info command

$ substreams info ./substreams.yaml

The output of the info command will print a message resembling:

info output

Package name: solana_spl_transfers
Version: v0.5.2
Doc: Solana SPL Token Transfers stream

  Stream SPL token transfers to the nearest human being.

Modules:
----
Name: spl_transfers
Initial block: 130000000
Kind: map
Output Type: proto:solana.spl.v1.TokenTransfers
Hash: 2b59e4e840f814f4154a688c2935da9c3b61dc61

Name: transfer_store
Initial block: 130000000
Kind: store
Value Type: proto:solana.spl.v1.TokenTransfers
Update Policy: UPDATE_POLICY_SET
Hash: 11fd70768029bebce3741b051c15191d099d2436

graph

The graph command prints out a visual graph of the package in the mermaid-js format.

Tip: Mermaid Live Editor is the visual editor used by Substreams.

graph command

$ substreams graph ./substreams.yaml
                    [±master ●●]
Mermaid graph:

```mermaid
graph TD;
  spl_transfers[map: spl_transfers]
  sf.solana.type.v1.Block[source: sf.solana.type.v1.Block] --> spl_transfers
  transfer_store[store: transfer_store]
  spl_transfers --> transfer_store
```

The graph command will result in a graphic resembling:

Mermaid generated graph diagram

inspect

The inspect command reaches deep into the file structure of a yaml configuration file or spkg package and is used mostly for debugging, or if you're curious_._

inspect command

$ substreams inspect ./substreams.yaml | less

The output of the inspect command will print a message resembling:

inspect output

proto_files
...
modules {
  modules {
    name: "my_module_name"
...

codegen

The codegen command generates a code for a specific sink taking as a Substreams module as input. There are two code generators available: subgraph and sql.

• Subgraph

Generates a Substreams-powered Subgraph project from the Substreams package found in the current folder.

substreams codegen subgraph

• SQL

Generates a SQL-based Substreams project from the Substreams package found in the current folder.

substreams codegen sql

Help

To view a list of available commands and brief explanations in the substreams CLI, run the substreams command in a terminal passing the -h flag. You can use this help reference at any time.

help option

substreams -h