substrate-etl is a large scale index of Polkadot + Kusama and their parachains based on Polkaholic.io, a multichain block explorer/indexer. Using this data warehouse, anyone can query decoded blocks, extrinsics, events, calls across both ecosystems.
We onboarded Polkadot Data into Google Public Datasets, announced in September 2023:
The substrate-etl index results in two large datasets (crypto_polkadot and crypto_kusama) covering the 2 relay chains and over 70 parachains. This represents Polkadot/Kusama alongside other Web3 ecosystems and supports complex large scale analyses with SQL using BigQuery's lightning-fast query performance on extremely large datasets. Having substrate-etl enables Dune-like analytics to guide business intelligence, inform engineering decision making, and showcase ecosystem growth.
Power users include Parity Data (led by Karim Jedda and Pavla Mijic), Subwallet/DotInsights, Web3metrics,
Dolpha.com, ChainViz and many data analysts within parachains and outside the ecosystem.
For a full guide, see substrate-etl, but you can get started with the following Analytics Hub links:
In addition to continuing support for substrate-etl/Polkaholic.io for existing and future chains, our plans for 2024 are to:
Requested: 99,025 DOT
Colorful Notion is led by Sourabh Niyogi and Michael Chung, based in the SF Bay Area. We have been active in the Polkadot ecosystem over the last 2 years and look forward to building with others in CoreJam/Polkadot 2.0’s “map reduce” ubiquitous computing future.
We are grateful for everyone who has helped us and hope we can pay it forward by helping others
Substrate-etl is a data warehouse that indexes Polkadot and Kusama and their parachains, allowing anyone to query decoded blocks, extrinsics, events, and calls across both ecosystems. It is available on Google Public Datasets and supports complex large scale analyses with SQL using BigQuery's lightning-fast query performance on extremely large datasets. The index results in two large datasets (crypto_polkadot and crypto_kusama) covering the 2 relay chains and over 70 parachains. Power users include Parity Data, Subwallet/DotInsights, Web3metrics, Dolpha.com, ChainViz, and many data analysts within parachains and outside the ecosystem. Colorful Notion, led by Sourabh Niyogi and Michael Chung, is the team behind substrate-etl, and they plan to continue supporting it for existing and future chains. They also have plans to chart a course within the Decentralized Futures, develop and support a Polkadot Builder Collective, and extend substrate-etl to support raw and fully-decoded EVM datasets.