Skip to main content
The financial industry is extremely data-heavy and relies on real-time and historical data for decision-making, risk assessment, fraud detection, and market analysis. simplifies management of these large volumes of data, while also providing you with meaningful analytical insights and optimizing storage costs. In this tutorial, you use to ingest, store, and analyze transactions on the Bitcoin blockchain. Blockchains are, at their essence, a distributed database. The transactions in a blockchain are an example of time-series data. You can use to query transactions on a blockchain, in exactly the same way as you might query time-series transactions in any other database. This tutorial uses a sample Bitcoin dataset and covers:
  1. Ingest data: set up and connect to a , create tables and s, and ingest data.
  2. Query the data: obtain information about recent transactions and blocks using basic SQL queries.
  3. Analyze the data: create s and use hyperfunctions to discover insights about transaction fees, mining revenue, and market correlations.
  4. Visualize results: graph your analytical queries in Grafana dashboards.

Prerequisites

To follow the steps on this page:
  • Create a target with Real-time analytics enabled.

    You need your connection details. This procedure also works for .
This tutorial uses a dataset that contains Bitcoin blockchain data for the past five days, in a named transactions.

Optimize time-series data using hypertables

s are tables in that automatically partition your time-series data by time. Time-series data represents the way a system, process, or behavior changes over time. s enable to work efficiently with time-series data. Each is made up of child tables called chunks. Each chunk is assigned a range of time, and only contains data from that range. When you run a query, identifies the correct chunk and runs the query on it, instead of going through the entire table. is the hybrid row-columnar storage engine in used by s. Traditional databases force a trade-off between fast inserts (row-based storage) and efficient analytics (columnar storage). eliminates this trade-off, allowing real-time analytics without sacrificing transactional capabilities. dynamically stores data in the most efficient format for its lifecycle: Move from rowstore to columstore in hypercore
  • Row-based storage for recent data: the most recent chunk (and possibly more) is always stored in the , ensuring fast inserts, updates, and low-latency single record queries. Additionally, row-based storage is used as a writethrough for inserts and updates to columnar storage.
  • Columnar storage for analytical performance: chunks are automatically compressed into the , optimizing storage efficiency and accelerating analytical queries.
Unlike traditional columnar databases, allows data to be inserted or modified at any stage, making it a flexible solution for both high-ingest transactional workloads and real-time analytics—within a single database. Because is 100% , you can use all the standard tables, indexes, stored procedures, and other objects alongside your s. This makes creating and working with s similar to standard .
  1. Connect to your In open an SQL editor. The in-Console editors display the query speed. You can also connect to your service using psql.
  2. Create a for your time-series data using CREATE TABLE. For efficient queries on data in the , remember to segmentby the column you will use most often to filter your data:
    When you create a using CREATE TABLE … WITH …, the default partitioning column is automatically the first column with a timestamp data type. Also, creates a columnstore policy that automatically converts your data to the , after an interval equal to the value of the chunk_interval, defined through compress_after in the policy. This columnar format enables fast scanning and aggregation, optimizing performance for analytical workloads while also saving significant storage space. In the conversion, s are compressed by up to 98%, and organized for efficient, large-scale queries. You can customize this policy later using alter_job. However, to change after or created_before, the compression settings, or the the policy is acting on, you must remove the columnstore policy and add a new one. You can also manually convert s in a to the .
  3. Create an index on the hash column to make queries for individual transactions faster:
  4. Create an index on the block_id column to make block-level queries faster: When you create a , it is partitioned on the time column. automatically creates an index on the time column. However, you’ll often filter your time-series data on other columns as well. You use indexes to improve query performance.
  5. Create a unique index on the time and hash columns to make sure you don’t accidentally insert duplicate records:

Load financial data

The dataset contains around 1.5 million Bitcoin transactions, the trades for five days. It includes information about each transaction, along with the value in satoshi. It also states if a trade is a coinbase transaction, and the reward a coin miner receives for mining the coin. To ingest data into the tables that you created, you need to download the dataset and copy the data to your database.
  1. Download the bitcoin_sample.zip file. The file contains a .csv file that contains Bitcoin transactions for the past five days: bitcoin_sample.zip
  2. In a new terminal window, run this command to unzip the .csv files:
  3. In Terminal, navigate to the folder where you unzipped the Bitcoin transactions, then connect to your using psql.
  4. At the psql prompt, use the COPY command to transfer data into your . If the .csv files aren’t in your current directory, specify the file paths in these commands:
    Because there is over a million rows of data, the COPY process could take a few minutes depending on your internet connection and local client resources.

Connect Grafana to Tiger Cloud

To visualize the results of your queries, enable Grafana to read the data in your :
  1. Log in to Grafana In your browser, log in to either:
    • Self-hosted Grafana: at http://localhost:3000/. The default credentials are admin, admin.
    • Grafana Cloud: use the URL and credentials you set when you created your account.
  2. Add your as a data source
    1. Open Connections > Data sources, then click Add new data source.
    2. Select PostgreSQL from the list.
    3. Configure the connection:
      • Host URL, Database name, Username, and Password Configure using your connection details. Host URL is in the format <host>:<port>.
      • TLS/SSL Mode: select require.
      • PostgreSQL options: enable TimescaleDB.
      • Leave the default setting for all other fields.
    4. Click Save & test.
    Grafana checks that your details are set correctly.

Query the data

When you have your dataset loaded, you can start constructing some queries to discover what your data tells you. In this section, you learn how to write queries that answer these questions:

What are the five most recent coinbase transactions?

Coinbase transactions are the first transaction in a block, and they include the reward a coin miner receives for mining the coin. To find the most recent coinbase transactions, you can query for transactions where is_coinbase is TRUE. You’ll notice that the fee_usd is $0 for each coinbase transaction because the miner receives the block reward directly without paying a transaction fee.
  1. Connect to the that contains the Bitcoin dataset.
  2. At the psql prompt, use this query to select the five most recent coinbase transactions:
  3. The data you get back looks a bit like this:

What are the five most recent transactions?

This dataset contains Bitcoin transactions for the last five days. To find out the most recent transactions in the dataset, you can use a SELECT statement. In this case, you want to find transactions that are not coinbase transactions, sort them by time in descending order, and take the top five results. You also want to see the block ID, and the value of the transaction in US Dollars.
  1. Connect to the that contains the Bitcoin dataset.
  2. At the psql prompt, use this query to select the five most recent non-coinbase transactions:
  3. The data you get back looks a bit like this:

What are the five most recent blocks?

In this procedure, you use a more complicated query to return the five most recent blocks, and show some additional information about each, including the block weight, number of transactions in each block, and the total block value in US Dollars.
  1. Connect to the that contains the Bitcoin dataset.
  2. At the psql prompt, use this query to select the five most recent blocks:
  3. The data you get back looks a bit like this:

Analyze the data

In this section, you use hyperfunctions to construct analytical queries that are not possible in standard . You create s to simplify and speed up your queries, then use them to answer these questions:
  • Is there any connection between the number of transactions and the transaction fees?
  • Does the transaction volume affect the BTC-USD rate?
  • Do more transactions in a block mean the block is more expensive to mine?
  • What percentage of the average miner’s revenue comes from fees compared to block rewards?
  • How does block weight affect miner fees?
  • What’s the average miner revenue per block?

Create continuous aggregates

You can use s to simplify and speed up your queries. For this analysis, you need three s focusing on three aspects of the dataset: Bitcoin transactions, blocks, and coinbase transactions. In each definition, the time_bucket() function controls how large the time buckets are. The examples all use 1-hour time buckets.
  1. Connect to the that contains the Bitcoin dataset.
  2. Create a called one_hour_transactions. This view holds aggregated data about each hour of transactions:
  3. Add a refresh policy to keep the up-to-date:
  4. Create a called one_hour_blocks. This view holds aggregated data about all the blocks that were mined each hour:
  5. Add a refresh policy to keep the up to date:
  6. Create a called one_hour_coinbase. This view holds aggregated data about all the transactions that miners received as rewards each hour:
  7. Add a refresh policy to keep the up to date:

Is there any connection between the number of transactions and the transaction fees?

Transaction fees are a major concern for blockchain users. If a blockchain is too expensive, you might not want to use it. This query shows you whether there’s any correlation between the number of Bitcoin transactions and the fees. The time range for this analysis is the last 2 days. If you choose to visualize the query in Grafana, you can see the average transaction volume and the average fee per transaction, over time. These trends might help you decide whether to submit a transaction now or wait a few days for fees to decrease.
  1. Connect to the that contains the Bitcoin dataset.
  2. At the psql prompt, use this query to average transaction volume and the fees from the one_hour_transactions :
  3. The data you get back looks a bit like this:
  4. To visualize this in Grafana:
    1. From the Dashboards page, click New and select New dashboard.
    2. Click Add visualization, then select the data source that connects to your .
    3. In the Queries section, change the Format to Time series and select Code.
    4. Type the query from the previous step and click Run query.
    Visualizing number of transactions and fees

Does the transaction volume affect the BTC-USD rate?

In cryptocurrency trading, there’s a lot of speculation. You can adopt a data-based trading strategy by looking at correlations between blockchain metrics, such as transaction volume and the current exchange rate between Bitcoin and US Dollars. If you choose to visualize the query in Grafana, you can see the average transaction volume, along with the BTC to US Dollar conversion rate.
  1. Connect to the that contains the Bitcoin dataset.
  2. At the psql prompt, use this query to return the trading volume and the BTC to US Dollar exchange rate:
  3. The data you get back looks a bit like this:
  4. To visualize this in Grafana:
    1. From the Dashboards page, click New and select New dashboard.
    2. Click Add visualization, then select the data source that connects to your .
    3. In the Queries section, change the Format to Time series and select Code.
    4. Type the query from the previous step and click Run query.
    5. Under the panel options on the right, click Add field override > Fields with name, then choose btc-usd rate in the dropdown.
    6. Click Add override property, then select Axis > Placement and click Right.
    Visualizing transaction volume and BTC-USD conversion rate

Do more transactions in a block mean the block is more expensive to mine?

The number of transactions in a block can influence the overall block mining fee. For this analysis, a larger time frame is required, so increase the analyzed time range to 5 days. If you choose to visualize the query in Grafana, you can see that the more transactions in a block, the higher the mining fee becomes.
  1. Connect to the that contains the Bitcoin dataset.
  2. At the psql prompt, use this query to return the number of transactions in a block, compared to the mining fee:
  3. The data you get back looks a bit like this:
  4. To visualize this in Grafana:
    1. From the Dashboards page, click New and select New dashboard.
    2. Click Add visualization, then select the data source that connects to your .
    3. In the Queries section, change the Format to Time series and select Code.
    4. Type the query from the previous step and click Run query.
    5. Under the panel options on the right, click Add field override > Fields with name, then choose mining fee in the dropdown.
    6. Click Add override property, then select Axis > Placement and click Right.
    Visualizing transactions in a block and the mining fee
You can extend this analysis to find if there is the same correlation between block weight and mining fee. More transactions should increase the block weight, and boost the miner fee as well. If you choose to visualize the query in Grafana, you can see the same kind of high correlation between block weight and mining fee. The relationship weakens when the block weight gets close to its maximum value, which is 4 million weight units, in which case it’s impossible for a block to include more transactions.
  1. Connect to the that contains the Bitcoin dataset.
  2. At the psql prompt, use this query to return the block weight, compared to the mining fee:
  3. The data you get back looks a bit like this:
  4. To visualize this in Grafana:
    1. From the Dashboards page, click New and select New dashboard.
    2. Click Add visualization, then select the data source that connects to your .
    3. In the Queries section, change the Format to Time series and select Code.
    4. Type the query from the previous step and click Run query.
    5. Under the panel options on the right, click Add field override > Fields with name, then choose mining fee in the dropdown.
    6. Click Add override property, then select Axis > Placement and click Right.
    Visualizing blockweight and the mining fee

What percentage of the average miner’s revenue comes from fees compared to block rewards?

In the previous queries, you saw that mining fees are higher when block weights and transaction volumes are higher. This query analyzes the data from a different perspective. Miner revenue is not only made up of miner fees, it also includes block rewards for mining a new block. This reward is currently 6.25 BTC, and it gets halved every four years. This query looks at how much of a miner’s revenue comes from fees, compares to block rewards. If you choose to visualize the query in Grafana, you can see that most miner revenue actually comes from block rewards. Fees never account for more than a few percentage points of overall revenue.
  1. Connect to the that contains the Bitcoin dataset.
  2. At the psql prompt, use this query to return coinbase transactions, along with the block fees and rewards:
  3. The data you get back looks a bit like this:
  4. To visualize this in Grafana:
    1. From the Dashboards page, click New and select New dashboard.
    2. Click Add visualization, then select the data source that connects to your .
    3. In the Queries section, change the Format to Time series and select Code.
    4. Type the query from the previous step and click Run query.
    5. In the options panel, in the Graph styles section, for Stack series select 100%.
    Visualizing coinbase revenue sources

How does block weight affect miner fees?

You’ve already found that more transactions in a block mean it’s more expensive to mine. In this query, you ask if the same is true for block weights? The more transactions a block has, the larger its weight, so the block weight and mining fee should be tightly correlated. This query uses a 12-hour moving average to calculate the block weight and block mining fee over time. If you choose to visualize the query in Grafana, you can see that the block weight and block mining fee are tightly connected. In practice, you can also see the four million weight units size limit. This means that there’s still room to grow for individual blocks, and they could include even more transactions.
  1. Connect to the that contains the Bitcoin dataset.
  2. At the psql prompt, use this query to return block weight, along with the block fees and rewards:
  3. The data you get back looks a bit like this:
  4. To visualize this in Grafana:
    1. From the Dashboards page, click New and select New dashboard.
    2. Click Add visualization, then select the data source that connects to your .
    3. In the Queries section, change the Format to Time series and select Code.
    4. Type the query from the previous step and click Run query.
    5. Under the panel options on the right, click Add field override > Fields with name, then choose mining fee in the dropdown.
    6. Click Add override property, then select Axis > Placement and click Right.
    Visualizing block weight and mining fees

What’s the average miner revenue per block?

In this final query, you analyze how much revenue miners actually generate by mining a new block on the blockchain, including fees and block rewards. To make the analysis more interesting, add the Bitcoin to US Dollar exchange rate, and increase the time range.
  1. Connect to the that contains the Bitcoin dataset.
  2. At the psql prompt, use this query to return the average miner revenue per block, with a 12-hour moving average:
  3. The data you get back looks a bit like this:
  4. To visualize this in Grafana:
    1. From the Dashboards page, click New and select New dashboard.
    2. Click Add visualization, then select the data source that connects to your .
    3. In the Queries section, change the Format to Time series and select Code.
    4. Type the query from the previous step and click Run query.
    5. Under the panel options on the right, click Add field override > Fields with name, then choose revenue in USD in the dropdown.
    6. Click Add override property, then select Axis > Placement and click Right.
    Visualizing block revenue over time