Crate chain_storage

Block storage crate.

Data model

This storage is designed to be independent from a particular block strucure. At the same time, since it is designed specifically for block chains, it handles the minimum amount of data required to maintain consistency. Such data includes:

• Block ID
• ID of the parent block
• Chain length

This data is provided alongside a block in the BlockInfo structure.

Volatile + permanent storage model

Since blockchain can be branching extensively, this library provides the mechanism to have multiple blockchain branches. However, doing so for all block screates a lot of overhead for maintaining volatile storage. At the same time, branches usually die out pretty fast.

Given that, the storage is separated into two parts:

• Volatile storage should be used for the part of a blockchain where a developer may need access to different branches.
• Permanent storage for the part of the blockchain that is guaranteed not to change anymore.

Moving blocks to the permanent store.

if you have this block structure and call store.flush_to_permanent_store(block 2 id), then block 1 and block 2 will be moved to the permanent storage. if you call store.flush_to_permanent_store(block 3 id), block 3 will also be moved to the permanent store, but block 3' will still exist in the volatile store. note that if you call store.get_blocks_by_chain_length(3) only block 3 (which is in the permanent store) will be returned; and you cannot call store.flush_to_permanent_store(block 3' id) now.

fig.1 - note that root does not actually exist, this is an id referredby the first block in the chain

+---------+       +---------+
|         |       |         |
| block 4 |       | block 4'|
|         |       |         |
+---------+       +---------+
     |                 |
     |                 |
     v                 v
+---------+       +---------+
|         |       |         |
| block 3 |       | block 3'|
|         |       |         |
+---------+       +---------+
     |                 |
     |                 |
     v                 |
+---------+            |
|         |            |
| block 2 +<-----------+
|         |
+---------+
     |
     |
     v
+---------+
|         |
| block 1 |
|         |
+---------+
     |
     |
     v
+---------+
|         |
|  (root) |
|         |
+---------+

Removing stale branches

If you want to clean up branches, do the following:

• Call store.get_tips_ids(), in our example it will return [Block 4 id, Block 4' id];
• Determine which branches do you want to remove.
• Call, for example, store.prune_branch(Block 4' id).

Performance benefits of permanent storage

Since blocks in the permanent storage are stored just one after another (the structure is block_length.to_le_bytes() ++ block_bytes and a file with references to blocks in the order they were added to the storage), it allows for the following scenarios:

• Very fast block iteration
• Transferring a portion of the data file over the network without locking it.
• O(1) access by chain length.

fig. 2 - permanent storage structure

store                  block no. index

+--------------+       +-------------+
|              |       |             |
| block1       |       | block1 pos  |
|              |       |             |
+--------------+       +-------------+
|              |       |             |
| block2       |       | block2 pos  |
|              |       |             |
+--------------+       +-------------+
|              |       |             |
| ...          |       | ...         |
|              |       |             |
+--------------+       +-------------+
|              |       |             |
| blockn       |       | blockn pos  |
|              |       |             |
+--------------+       +-------------+

Storage directory layout

store
├── permanent       - permanent storage directory
│   └── flatfile    - storage file that can be transferred over the network
└── volatile        - volatile storage

Modules

• block_info 🔒
• block_store 🔒
• error 🔒
• iterator 🔒
• permanent_store 🔒
• test_utils
  Utilities for testing the storage.
• value 🔒

Structs

• BlockInfo
  A structure that holds the information about a blocks, that is needed to maintain consistency of the storage. This include the ID of the blocks, the ID of its parent and the length of the block chain for the given block.
• BlockStore
• StorageIterator
  Iterator over blocks. Starts from n-th ancestor of the given block.
• Value
  Wrapper for data held by the database. This wrapper holds structs returned by both volatile and permanent storage to ensure we don’t have needless copying on return. Data should be accessed through the AsRef trait.

Enums

• ConsistencyFailure
• Error