LogoLogo
Package RegistryThe Graph
  • Introduction
  • Getting Started
  • Tutorials
    • Develop Your First Substreams
      • on EVM
      • on Solana
        • Transactions & Instructions
        • Account Changes
      • on Cosmos
        • Injective
        • MANTRA
      • on Starknet
      • on Stellar
    • Publishing a Substreams Package
  • How-To Guides
    • Developing Substreams
      • on EVM
        • Exploring Ethereum
          • Mapping Blocks
          • Filter Transactions
          • Retrieve Events of a Smart Contract
      • on Solana
        • Explore Solana
          • Filter Instructions
          • Filter Transactions
        • SPL Token Tracker
        • NFT Trades
        • DEX Trades
      • on Cosmos
        • Injective
          • Simple Substreams Example
          • Foundational Modules
          • Dojo DEX USDT Volume Subgraph Example
    • Using a Substreams Sink
      • Substreams:SQL
      • Substreams:Subgraph
        • Triggers
        • Graph Out
      • Substreams:Stream
        • JavaScript
        • Go
      • Substreams:PubSub
      • Community Sinks
        • MongoDB
        • Files
        • Key-Value Store
        • Prometheus
    • EVM Extensions
      • Making eth_calls
    • Getting Started Using Rust and Protobuf
      • Rust
        • Option struct
        • Result struct
      • Protobuf Schemas
    • From Yellowstone to Substreams
  • Reference Material
    • Chains and endpoints
      • Ethereum Data Model
    • Never Miss Data
    • Development Container Reference
    • Substreams CLI
      • Install the CLI
      • Authentication
      • Substreams CLI reference
    • Substreams Components
      • Packages
      • Modules
        • Module types
        • Inputs
        • Output
        • Module handlers
        • Module handler creation
        • Indexes
        • Keys in stores
        • Dynamic data sources
        • Aggregation Windows
        • Parameterized Modules
      • Manifests Reference
    • Substreams Architecture
    • Graph-Node
      • Local Development
      • Publish to The Graph Network
    • Indexer Reference
      • Test Substreams Locally
    • Logging, Debugging & Testing
    • Change log
    • FAQ
  • Decentralized Indexing
    • What is The Graph?
Powered by GitBook
On this page
  • Protobuf overview
  • Protobuf definition for Substreams
  • Generating protobufs
  • Protobuf and Rust optional fields

Was this helpful?

Edit on GitHub
  1. How-To Guides
  2. Getting Started Using Rust and Protobuf

Protobuf Schemas

StreamingFast Substreams protobuf schemas

PreviousResult structNextFrom Yellowstone to Substreams

Last updated 1 year ago

Was this helpful?

Protobuf overview

Substreams uses Google Protocol Buffers extensively. Protocol Buffers, also referred to as protobufs, are used as the API for data models specific to the different blockchains. Manifests contain references to the protobufs for your Substreams module.

Tip: Protobufs define the input and output for modules.

Learn more about the details of Google Protocol Buffers in the official documentation provided by Google.

Google Protocol Buffer Documentation

in the official documentation provided by Google.

Google Protocol Buffer Tutorial

for Google Protocol Buffers provided by Google.

Protobuf definition for Substreams

Define a protobuf model as representing a list of ERC721 transfers.

Note: The Transfers protobuf in the Substreams Template example is located in the proto directory.

eth/erc721/v1/erc721.proto
syntax = "proto3";

package eth.erc721.v1;

message Transfers {
  repeated Transfer transfers = 1;
}

message Transfer {
  bytes from = 1;
  bytes to = 2;
  uint64 token_id = 3;
  bytes trx_hash = 4;
  uint64 ordinal = 5;
}

Identifying data types

The ERC721 smart contract used in the Substreams Template example contains a Transfer event. You can use the event data through a custom protobuf.

The protobuf file serves as the interface between the module handlers and the data being provided by Substreams.

Tip: Protobufs are platform-independent and are defined and used for various blockchains.

  • The ERC721 smart contracts used in the Substreams Template example are generic contracts used across many different Ethereum applications.

  • The size and scope of the Substreams module dictates the number of and complexity of protobufs.

Several specific data types exist in the Ethereum smart contract ecosystem, some extending the ERC20 and ERC721 base modules. Complex protobufs are created and refined based on the various data types used across the different blockchains.

Tip: The use of fully qualified protobuf file paths reduces the risk of naming conflicts when other community members build their Substreams packages.

Generating protobufs

The substreams CLI is used to generate the associated Rust code for the protobuf.

Notice the protogen command and Substreams manifest passed into the substreams CLI.

substreams protogen ./substreams.yaml --exclude-paths="sf/ethereum,sf/substreams,google"
src/pb/mod.rs
#[path = "eth.erc721.v1.rs"]
#[allow(dead_code)]
pub mod erc721;

Protobuf and Rust optional fields

match person.Location {
    Some(location) => { /* Value is present, do something */ }
    None => { /* Value is absent, do something */ }
}
if let Some(location) = person.location {
    // Value is present, do something
}

Note: You need to be absolutely sure the field is always defined, otherwise Substreams panics and never completes, getting stuck on a block indefinitely.

file in the official Substreams Template example repository.

The Substreams Template example extracts Transfer events from the which is located on the Ethereum blockchain.

The pairing code is generated and saved into the Rust file.

The file located in the src/pb directory of the Substreams Template example is responsible for exporting the freshly generated Rust code.

View the file in the repository.

Protocol buffers define fields' type by using standard primitive data types, such as integers, booleans, and floats or a complex data type such as message, enum, oneof or map. View the of types in the .

Any primitive data types in a message generate the corresponding Rust type, for string, u64 for uint64, and assign the default value of the corresponding Rust type if the field is not present in a message, an empty string for , 0 for integer types, false for bool.

Rust generates the corresponding message type wrapped by an enum type for fields referencing other complex messages. The variant is used if the field is not present in the message.

The is used to represent the presence through or absence through of a value in Rust. allows developers to distinguish between a field containing a value versus a field without an assigned a value.

Note: The standard approach to represent nullable data in Rust is to wrap optional values in .

The Rust keyword is used to compare the value of an to a or variant. Handle a type wrapped in Rust by using:

If you are only interested in finding the presence of a value, use the statement to handle the arm of the code.

If a value is present, use the call on the to obtain the wrapped data. You'll need to account for these types of scenarios if you control the creation of the messages yourself or if the field is documented as always being present.

PROST! is a tool for generating Rust code from protobuf definitions. in the project's official GitHub repository.

in the official Rust documentation.

Learn more about Google Protocol Buffers
Explore examples and additional learning material
proto:eth.erc721.v1.Transfers
View the erc721.proto
Bored Ape Yacht Club smart contract
src/pb/eth.erc721.v1.rs
mod.rs
mod.rs
full list
Google Protocol Buffers documentation
String
String
Option
None
Option
enum
Some(x)
None
Option
Option<T>
match
Option
Some
None
Option
if let
Some(x)
match
.unwrap()
Option
Learn more about prost
Learn more about Option