Jörmungandr User Guide

Welcome to the Jörmungandr User Guide.

Jörmungandr is a node implementation, written in rust, with the initial aim to support the Ouroboros type of consensus protocol.

A node is a participant of a blockchain network, continuously making, sending, receiving, and validating blocks. Each node is responsible to make sure that all the rules of the protocol are followed.

Mythology

Jörmungandr refers to the Midgard Serpent in Norse mythology. It is a hint to Ouroboros, the Ancient Egyptian serpent, who eat its own tail, as well as the IOHK paper on proof of stake.

General Concepts

This chapter covers the general concepts of the blockchain, and their application in the node, and is followed by the node organisation and the user interaction with it.

Blockchain concepts

Time

Slots represent the basic unit of time in the blockchain, and at each slot a block could be present.

Consecutive slots are grouped into epochs, which have updatable size defined by the protocol.

Fragments

Fragments are part of the blockchain data that represent all the possible events related to the blockchain health (e.g. update to the protocol), but also and mainly the general recording of information like transactions and certificates.

Blocks

Blocks represent the spine of the blockchain, safely and securely linking blocks in a chain, whilst grouping valid fragments together.

Blocks are composed of 2 parts:

  • The header
  • The content

The header link the content with the blocks securely together, while the content is effectively a sequence of fragments.

Blockchain

The blockchain is the general set of rules and the blocks that are periodically created. Some of the rules and settings, can be changed dynamically in the system by updates, while some other are hardcoded in the genesis block (first block of the blockchain).

    +-------+      +-------+
    |Genesis+<-----+Block 1+<--- ....
    |Header |      |Header |
    +---+---+      +---+---+
        |              |
    +---v---+      +---v---+
    |Genesis|      |Block 1|
    |Content|      |Content|
    +-------+      +-------+

Consensus

The node currently support the following consensus protocol:

  • Ouroboros BFT (OBFT)
  • Ouroboros Genesis-Praos

Ouroboros BFT is a simple Byzantine Fault Tolerant (BFT) protocol where the block makers is a known list of leaders that successively create a block and broadcast it on the network.

Ouroboros Genesis Praos is a proof of stake (PoS) protocol where the block maker is made of a lottery where each stake pool has a chance proportional to their stake to be elected to create a block. Each lottery draw is private to each stake pool, so that the overall network doesn't know in advance who can or cannot create blocks.

In Genesis-Praos slot time duration is constant, however the frequency of creating blocks is not stable, since the creation of blocks is a probability that is linked to the stake and consensus_genesis_praos_active_slot_coeff.

Note: In Genesis-Praos, if there is no stake in the system, no blocks will be created anymore starting with the next epoch.

Leadership

The leadership represent in abstract term, who are the overall leaders of the system and allow each individual node to check that specific blocks are lawfully created in the system.

The leadership is re-evaluated at each new epoch and is constant for the duration of an epoch.

Leader

Leader are an abstraction related to the specific actor that have the ability to create block; In OBFT mode, the leader just the owner of a cryptographic key, whereas in Genesis-Praos mode, the leader is a stake pool.

Transaction

Transaction forms the cornerstone of the blockchain, and is one type of fragment and also the most frequent one.

Transaction is composed of inputs and outputs; On one side, the inputs represent coins being spent, and on the other side the outputs represent coins being received.

    Inputs         Alice (80$)        Bob (20$)
                        \             /
                         \           /
                          -----------
                                100$
                             ---------
                            /         \
    Outputs            Charlie (50$)  Dan (50$)

Transaction have fees that are defined by the blockchain settings and the following invariant hold:

\( \sum Inputs = \sum Outputs + fees \)

Transaction need to be authorized by each of the inputs in the transaction by their respective witness. In the most basic case, a witness is a cryptographic signature, but depending on the type of input can the type of witness vary.

Accounting

The blockchain has two methods of accounting which are interoperable:

  • Unspent Transaction Output (UTXO)
  • Accounts

UTXO behaves like cash/notes, and work like fixed denomination ticket that are cumulated. This is the accounting model found in Bitcoin. A UTXO is uniquely reference by its transaction ID and its index.

Accounts behaves like a bank account, and are simpler to use since exact amount can be used. This is the accounting model found in Ethereum. An account is uniquely identified by its public key.

Each inputs could refer arbitrarily to an account or a UTXO, and similarly each outputs could refer to an account or represent a new UTXO.

Stake

In a proof of stake, participants are issued a stake equivalent to the amount of coins they own. The stake is then used to allow participation in the protocol, simply explained as:

The more stake one has, the more likely one will participate in the good health of the network.

When using the BFT consensus, the stake doesn't influence how the system runs, but stake can still be manipulated for a later transition of the chain to another consensus mode.

Stake in the Account Model

Account are represented by 1 type of address and are just composed of a public key. The account accumulate moneys and its stake power is directly represented by the amount it contains

For example:


    A - Account with 30$ => Account A has stake of 30
    B - Account with 0$ => Account B has no stake

The account might have a bigger stake than what it actually contains, since it could also have associated UTXOs, and this case is covered in the next section.

Stake in the UTXO Model

UTXO are represented by two kind of addresses:

  • single address: those type of address have no stake associated
  • group address: those types of address have an account associated which receive the stake power of the UTXOs value

For example with the following utxos:

    UTXO1 60$ (single address) => has stake of 0

    UTXO2 50$ (group address A) \
                                 ->- A - Account with 10$ => Account A has stake of 100
    UTXO3 40$ (group address A) /

    UTXO4 20$ (group address B) -->- B - Account with 5$ => Account B has stake of 25

Stake pool

Stake pool are the trusted block creators in the genesis-praos system. A pool is declared on the network explicitly by its owners and contains, metadata and cryptographic material.

Stake pool has no stake power on their own, but participants in the network delegate their stake to a pool for running the operation.

Stake Delegation

Stake can and need to be delegated to stake pool in the system. They can change over time with a publication of a new delegation certificate.

Delegation certificate are a simple declaration statement in the form of:

    Account 'A' delegate to Stake Pool 'Z'

Effectively it assigns the stake in the account and its associated UTXO stake to the pool it delegates to until another delegation certificate is made.

Node organization

Secure Enclave

The secure enclave is the component containing the secret cryptographic material, and offering safe and secret high level interfaces to the rest of the node.

Network

The node's network is 3 components:

  • Intercommunication API (GRPC)
  • Public client API (REST)
  • Control client API (REST)

More detailed information here

Intercommunication API (GRPC)

This interface is a binary, efficient interface using the protobuf format and GRPC standard. The protobuf files of types and interfaces are available in the source code.

The interface is responsible to communicate with other node in the network:

  • block sending and receiving
  • fragments (transaction, certificates) broadcast
  • peer2peer gossip

Public API REST

This interface is for simple queries for clients like:

  • Wallet Client & Middleware
  • Analytics & Debugging tools
  • Explorer

it's recommended for this interface to not be opened to the public.

TODO: Add a high level overview of what it does

Control API REST

This interface is not finished, but is a restricted interface with ACL, to be able to do maintenance tasks on the process:

  • Shutdown
  • Load/Retire cryptographic material

TODO: Detail the ACL/Security measure

Network overview

Jörmungandr network capabilities are split into:

  1. the REST API, used for informational queries or control of the node;
  2. the gRPC API for blockchain protocol exchange and participation;

Here we will only review the gRPC API as the REST API is described in another chapter: go to the REST documentation

The protocol

The protocol is based on gRPC that combines commonly used protocols like HTTP/2 and RPC. More precisely, Jörmungandr utilises.

This choice was made because gRPC is already widely supported around the world because of it's uitilization of standard protocols HTTP/2 which makes it much easier for Proxies and Firewalls to recognise the protocol and permit the traffic.

Type of queries

The protocol allows you to send multiple types of messages between nodes:

  • sync block to remote peer's Last Block (tip).
  • propose new fragments (new transactions, certificates, ...): this is for the fragment propagation.
  • propose new blocks: for block propagation.

There are other commands that optimise the communication and synchronization between nodes that will be documented here in the future.

Another type of messages is the Gossip message. These gossip messages allow Nodes to exchange information (gossips) about other nodes on the network, allowing for peer discovery.

Peer to peer

The peer 2 peer connections are established utilising multiple components:

  • A multilayered topology (e.g. Poldercast);
  • Gossiping for node discoverability;
  • Subscription mechanism for event propagation;
  • Security and countermeasures: (such as Topology Policy for scoring and/or blacklisting nodes);

Multilayered topology

As described in the Poldercast paper, our network topology is built on multiple layers that allow for granular control of it's behavior. In practice this means a node will have different groups of nodes that it connects to based on different algorithms, each of these groups are a subset of the whole known list of nodes.

In short we have:

  • The rings layer selects a predecessor(s) and a successor(s) for each topic (Fragment or Blocks);
  • The Vicinity layer will select nodes that have similar interests;
  • The Cyclon layer, will select nodes randomly.

However, we keep the option open to remove some of these layers or to add new ones, such as:

  • A layer to allow privilege connections between stake pools;
  • A layer for the user's whitelist, a list of nodes the users considered trustworthy and that we could use to check in the current state of the network and verify the user's node is not within a long running fork;

Gossiping

Gossiping is the process used for peer discovery. It allows two things:

  1. For any nodes to advertise themselves as discoverable;
  2. To discover new nodes via exchanging a list of nodes (gossips);

The gossips are selected by the different layers of the multilayered topology. For the Poldercast modules, the gossips are selected just as in the paper. Additional modules may select new nodes in the gossip list or may decide to not add any new information.

Subscription mechanism

Based on the multilayered topology, the node will open multiplexed and bi-directional connections (thanks to industry standard gRPC, this comes for free). These bi-directional connections are used to propagate events such as:

  • Gossiping events, when 2 nodes exchange gossips for peer discovery;
  • Fragment events, when a node wants to propagate a new fragment to other nodes;
  • Block events, when a node wants to propagate a new block creation event

Security and countermeasures

In order to facilitate the handling of unreachable nodes or of misbehaving ones we have built a node policy tooling. Currently, we collect connectivity statuses for each node. The policy can then be tuned over the collected data to apply some parameters when connecting to a given node, as well as banning nodes from our topology.

For each node, the following data is collected:

Connection statuses:

  • The failed connection attempts and when it happened;
  • Latency
  • Last message used per topic item (last time a fragment has been received from that node, last time a block has been received from that node…)

In the future, we may expand the polocy to include data collected at the blockchain level lile:

  • Faults (e.g. trying to send an invalid block)
  • Contributions in the network
  • Their blockchain status (e.g. tips)

Policy

The p2p policy provides some more fine control on how to handle nodes flagged as not behaving as expected (see the list of data collected).

It currently works as a 4 levels: trusted, possible contact, quarantined, forgotten. Each gossip about a new node will create a new entry in the list of possible contact. Then the policy, based on the logged data associated to this node, may decide to put this node in quarantine for a certain amount of time.

Trusted nodes are the ones to which we were able to connect successfully. A connectivity report against those nodes will make them transition to the possible contact level, while a successful connection attempt will promote them again to trusted.

The changes from one level to another is best effort only. Applying the policy may be costly so the node applies the policy only on the node it is interested about (a gossip update or when reporting an issue against a node). This guarantees that the node does not spend too much time policing its database. And it also makes sure that only the nodes of interest are up to date. However it is possible for the node to choose, at a convenient time, to policy the whole p2p database. This is not enforced by the protocol.

DispositionDescription
availableNode is available for the p2p topology for view selection and gossips.
quarantinedNode is not available for the p2p topology for view selection or gossips. After a certain amount of time, if the node is still being gossiped about, it will be moved to available.
forgottenA node forgotten is simply removed from the whole p2p database. However, if the node is still being gossiped about it will be added back as available and the process will start again.

Quickstart

The rust node comes with tools and help in order to quickly start a node and connect to the blockchain.

It is compatible with most platforms and it is pre-packaged for some of them.

Here we will see how to install jormungandr and its helper jcli and how to connect quickly to a given blockchain.

There are three posible ways you can start jormungandr.

As a passive node in an existing network

As described here.

The passive Node is the most common type of Node on the network. It can be used to download the blocks and broadcast transactions to peers, but it doesn't have cryptographic materials or any mean to create blocks. This type of nodes are mostly used for wallets, explorers or relays.

As a node generating blocks in an existing network

The network could be running either bft or genesis consensus. In the former case the node must have the private key of a registered as a slot leader, while for the latter the private keys of a registered stake pool are needed.

More information here

Creating your own network

This is similar to the previous case, but configuring a genesis file is needed. Consult the Advanced section for more information on this procedure.

Command line tools

The software is bundled with 2 different command line software:

  1. jormungandr: the node;
  2. jcli: Jörmungandr Command Line Interface, the helpers and primitives to run and interact with the node.

Installation

From a release

This is the recommended method. Releases are all available here.

From source

Jörmungandr's code source is available on github. Follow the instructions to build the software from sources.

Help and auto completion

All commands come with usage help with the option --help or -h.

For jcli, it is possible to generate the auto completion with:

jcli auto-completion bash ${HOME}/.bash_completion.d

Supported shells are:

  • bash
  • fish
  • zsh
  • powershell
  • elvish

Note: Make sure ${HOME}/.bash_completion.d directory previously exists on your HD. In order to use auto completion you still need to:

source ${HOME}/.bash_completion.d/jcli.bash

You can also put it in your ${HOME}/.bashrc.

Starting a passive node

In order to start the node, you first need to gather the blockchain information you need to connect to.

  1. the hash of the genesis block of the blockchain, this will be the source of truth of the blockchain. It is 64 hexadecimal characters.
  2. the trusted peers identifiers and access points.

These information are essentials to start your node in a secure way.

The genesis block is the first block of the blockchain. It contains the static parameters of the blockchain as well as the initial funds. Your node will utilise the Hash to retrieve it from the other peers. It will also allows the Node to verify the integrity of the downloaded genesis block.

The trusted peers are the nodes in the public network that your Node will trust in order to initialise the Peer To Peer network.

The node configuration

Your node configuration file may look like the following:

Note

This config shouldn't work as it is, the ip address and port for the trusted peer should be those of an already running node. Also, the public_address ('u.x.v.t') should be a valid address (you can use an internal one, eg: 127.0.0.1). Furthermore, you need to have permission to write in the path specified by the storage config.

storage: "/mnt/cardano/storage"

rest:
  listen: "127.0.0.1:8443"

p2p:
  trusted_peers:
    - address: "/ip4/104.24.28.11/tcp/8299"
      id: ad24537cb009bedaebae3d247fecee9e14c57fe942e9bb0d

Description of the fields:

  • storage: (optional) Path to the storage. If omitted, the blockchain is stored in memory only.
  • log: (optional) Logging configuration:
    • level: log messages minimum severity. If not configured anywhere, defaults to "info". Possible values: "off", "critical", "error", "warn", "info", "debug", "trace".
    • format: Log output format, plain or json.
    • output: Log output destination. Possible values are:
      • stdout: standard output
      • stderr: standard error
      • syslog: syslog (only available on Unix systems)
      • syslogudp: remote syslog (only available on Unix systems)
        • host: address and port of a syslog server
        • hostname: hostname to attach to syslog messages
      • journald: journald service (only available on Linux with systemd, (if jormungandr is built with the systemd feature)
      • gelf: Configuration fields for GELF (Graylog) network logging protocol (if jormungandr is built with the gelf feature):
        • backend: hostname:port of a GELF server
        • log_id: identifier of the source of the log, for the host field in the messages.
      • file: path to the log file.
  • rest: (optional) Configuration of the REST endpoint.
    • listen: address:port to listen for requests
    • tls: (optional) enables TLS and disables plain HTTP if provided
      • cert_file: path to server X.509 certificate chain file, must be PEM-encoded and contain at least 1 item
      • priv_key_file: path to server private key file, must be PKCS8 with single PEM-encoded, unencrypted key
    • cors: (optional) CORS configuration, if not provided, CORS is disabled
      • allowed_origins: (optional) allowed origins, if none provided, echos request origin
      • max_age_secs: (optional) maximum CORS caching time in seconds, if none provided, caching is disabled
  • p2p: P2P network settings
    • trusted_peers: (optional) the list of nodes's multiaddr with their associated public_id to connect to in order to bootstrap the P2P topology (and bootstrap our local blockchain);
    • public_id: (optional) the node's public ID that will be used to identify this node to the network.
    • public_address: multiaddr string specifying address of the P2P service. This is the public address that will be distributed to other peers of the network that may find interest in participating to the blockchain dissemination with the node.
    • listen: (optional) address:port to specifies the address the node will listen to to receive p2p connection. Can be left empty and the node will listen to whatever value was given to public_address.
    • topics_of_interest: The dissemination topics this node is interested to hear about:
      • messages: Transactions and other ledger entries. Typical setting for a non-mining node: low. For a stakepool: high;
      • blocks: Notifications about new blocks. Typical setting for a non-mining node: normal. For a stakepool: high.
    • max_connections: The maximum number of simultaneous P2P connections this node should maintain.
  • explorer: (optional) Explorer settings
    • enabled: True or false
  • no_blockchain_updates_warning_interval: (optional, seconds) if no new blocks were received after this period of time, the node will start sending you warnings in the logs.

Starting the node

jormungandr --config config.yaml --genesis-block-hash 'abcdef987654321....'

The 'abcdef987654321....' part refers to the hash of the genesis, that should be given to you from one of the peers in the network you are connecting to.

In case you have the genesis file (for example block-0.bin, because you are creating the network) you can get this hash with jcli.

jcli genesis hash --input block-0.bin

or, in case you only have the yaml file

jcli genesis encode --input genesis.yaml | jcli genesis hash

REST Api

It is possible to query the node via its REST Interface.

In the node configuration, you have set something like:

# ...

rest:
  listen: "127.0.0.1:8443"

#...

This is the REST endpoint to talk to the node, to query blocks or send transaction.

It is possible to query the node stats with the following end point:

curl http://127.0.0.1:8443/api/v0/node/stats

The result may be:

{"blockRecvCnt":120,"txRecvCnt":92,"uptime":245}

THE REST API IS STILL UNDER DEVELOPMENT

Please note that the end points and the results may change in the future.

To see the whole Node API documentation, click here

Explorer mode

The node can be configured to work as a explorer. This consumes more resources, but makes it possible to query data otherwise not available.

Configuration

There are two ways of enabling the explorer api. It can either be done by passing the --enable-explorer flag on the start arguments or by the config file:

explorer:
    enabled: true

CORS

For configuring CORS the explorer API, this needs to be done on the REST section of the config, as documented here.

API

A graphql interface can be used to query the explorer data, when enabled, two endpoints are available in the REST interface: /explorer/graphql and /explorer/playground .

The first is the one that queries are made against, for example:

curl \
    -X POST \
    -H "Content-Type: application/json" \
    --data '{'\
        '"query": "{'\
        '   status {'\
        '       latestBlock {'\
        '           chainLength'\
        '           id'\
        '           previousBlock {'\
        '               id'\
        '           }'\
        '       }'\
        '   }'\
        '}"'\
    '}' \
  http://127.0.0.1:8443/explorer/graphql

While the second serves an in-browser graphql IDE that can be used to try queries interactively.

How to start a node as a leader candidate

Gathering data

Like in the passive node case, two things are needed to connect to an existing network

  1. the hash of the genesis block of the blockchain, this will be the source of truth of the blockchain. It is 64 hexadecimal characters.
  2. the trusted peers identifiers and access points.

The node configuration could be the same as that for running a passive node.

There are some differences depending if you are connecting to a network running a genesis or bft consensus protocol.

Connecting to a genesis blockchain

Registering a stake pool

In order to be able to generate blocks in an existing genesis network, a registered stake pool is needed.

Creating the secrets file

Put the node id and private keys in a yaml file in the following way:

Example

filename: node_secret.yaml

genesis:
  sig_key: Content of stake_pool_kes.prv file
  vrf_key: Content of stake_pool_vrf.prv file
  node_id: Content of stake_pool.id file

Starting the node

jormungandr --genesis-block-hash asdf1234... --config config.yaml --secret node_secret.yaml

The 'asdf1234...' part should be the actual block0 hash of the network

Connecting to a BFT blockchain

In order to generate blocks, the node should be registered as a slot leader in the network and started in the following way.

The secret file

Put secret key in a yaml file, e.g. node_secret.yaml as follows:

bft:
 signing_key: ed25519_sk1kppercsk06k03yk4qgea....

where signing_key is a private key associated to the public id of a slot leader.

Starting the node

jormungandr --genesis-block asdf1234... --config node.config --secret node_secret.yaml

The 'asdf1234...' part should be the actual block0 hash of the network

Configuration

This chapter covers the node documentation, necessary to have a working system. It covers the network, logging and storage parameters.

Node Configuration

This is an common example of a Jörmungandr node configuration file typically named node-config.yaml, however your's will vary depending on your needs. Additionally, this configuration has been tested on a specific Jörmungandr version and may change with newer versions. It's important to keep in mind that the trusted_peers portion of this configuration will be different for each Cardano blockchain network. If you're trying to connect this node to a specific network, you need to know its genesis block hash, and its associated list of trusted peers.

Example Configuration - 1:

---
log:
  output: stderr
  level:  info
  format: plain

http_fetch_block0_service:
  - https://url/jormungandr-block0/raw/master/data

skip_bootstrap: false # If set to true - will skip the bootstrapping phase

bootstrap_from_trusted_peers: false

p2p:
  public_address: "/ip4/X.X.X.X/tcp/Y" # This should match your public IP address (X) and port number (Y)
  #listen: 0.0.0.0:Y
  topics_of_interest:
    blocks: normal # Default is normal - set to high for stakepool
    messages: low  # Default is low    - set to high for stakepool
  allow_private_addresses: false
  max_connections: 256
  max_client_connections: 192
  gossip_interval: 10s
  max_bootstrap_attempts: # Default is not set
  trusted_peers:
    - address: "/ip4/13.230.137.72/tcp/3000"
      id: e4fda5a674f0838b64cacf6d22bbae38594d7903aba2226f
    - address: "/ip4/13.230.48.191/tcp/3000"
      id: c32e4e7b9e6541ce124a4bd7a990753df4183ed65ac59e34
    - address: "/ip4/18.196.168.220/tcp/3000"
      id: 74a9949645cdb06d0358da127e897cbb0a7b92a1d9db8e70
    - address: "/ip4/3.124.132.123/tcp/3000"
      id: 431214988b71f3da55a342977fea1f3d8cba460d031a839c
    - address: "/ip4/18.184.181.30/tcp/3000"
      id: e9cf7b29019e30d01a658abd32403db85269fe907819949d
    - address: "/ip4/184.169.162.15/tcp/3000"
      id: acaba9c8c4d8ca68ac8bad5fe9bd3a1ae8de13816f40697c
    - address: "/ip4/13.56.87.134/tcp/3000"
      id: bcfc82c9660e28d4dcb4d1c8a390350b18d04496c2ac8474
  policy:
    quarantine_duration: 30m
    quarantine_whitelist:
      - "/ip4/13.230.137.72/tcp/3000"
      - "/ip4/13.230.48.191/tcp/3000"
      - "/ip4/18.196.168.220/tcp/3000"
  layers:
    preferred_list:
      view_max: 20
      peers:
        - address: "/ip4/13.230.137.72/tcp/3000"
          id: e4fda5a674f0838b64cacf6d22bbae38594d7903aba2226f
        - address: "/ip4/13.230.48.191/tcp/3000"
          id: c32e4e7b9e6541ce124a4bd7a990753df4183ed65ac59e34
        - address: "/ip4/18.196.168.220/tcp/3000"
          id: 74a9949645cdb06d0358da127e897cbb0a7b92a1d9db8e70

rest:
  listen: 127.0.0.1:3100

storage: "./storage"

explorer:
  enabled: false

mempool:
    pool_max_entries: 100000
    log_max_entries: 100000

leadership:
    logs_capacity: 1024

no_blockchain_updates_warning_interval: 15m

Note: The node configuration uses the YAML format.

Advanced

Rewards report

Starting the node jormungandr with the command line option --rewards-report-all will collect a thorough report of all the reward distribution. It can then be accessed via the REST endpoints /api/v0/rewards/history/1 or /api/v0/rewards/epoch/10.

this is not a recommended setting as it may take memory and may trigger some latency.

Handling of time-consuming transactions

By default we allow a single transaction to delay a block by 50 slots. This can be changed by adjusting the block_hard_deadline setting.

The following is deprecated and will be removed

If you want to record the reward distributions in a directory it is possible to set the environment variable: JORMUNGANDR_REWARD_DUMP_DIRECTORY=/PATH/TO/DIR/TO/WRITE/REWARD.

If an error occurs while dumping the reward, the node will panic with an appropriate error message.

Logging

The following options are available in the log section:

  • level: log messages minimum severity. If not configured anywhere, defaults to info. Possible values: off, critical, error, warn, info, debug, trace

  • format: Log output format, plain or json

  • output: Log output destination (multiple destinations are supported). Possible values are:

    • stdout: standard output
    • stderr: standard error
    • journald: journald service (only available on Linux with systemd, (if jormungandr is built with the systemd feature)
    • gelf: Configuration fields for GELF (Graylog) network logging protocol (if jormungandr is built with the gelf feature):
      • backend: hostname:port of a GELF server
      • log_id: identifier of the source of the log, for the host field in the messages
    • file: path to the log file

Example

A single configurable backend is supported.

Output to stdout

log:
  output: stdout
  level:  trace
  format: plain

Output to a file

log:
  output:
    file: example.log
  level: info
  format: json

Node network

There are 2 different network interfaces which are covered by their respective section:

rest:
   ...
p2p:
   ...

REST interface configuration

  • listen: listen address
  • tls: (optional) enables TLS and disables plain HTTP if provided
    • cert_file: path to server X.509 certificate chain file, must be PEM-encoded and contain at least 1 item
    • priv_key_file: path to server private key file, must be PKCS8 with single PEM-encoded, unencrypted key
  • cors: (optional) CORS configuration, if not provided, CORS is disabled
    • allowed_origins: (optional) allowed origins, if none provided, echos request origin, note that an origin should include a scheme, for example: http://127.0.0.1:8080.
    • max_age_secs: (optional) maximum CORS caching time in seconds, if none provided, caching is disabled

Configuring TLS

In order to enable TLS there must be provided certificate and private key files.

jcli TLS requirements.

Note that jormungandr itself does not have any specific requirements for TLS certificates and you may give whatever you want including self-signed certificates as long as you do not intend to use jcli.

The cryptography standards used by jcli as well as by all modern browsers and many http clients place the following requirements on certificates:

  • A certificate should adhere to X.509 v3 with appropriate key usage settings and subject alternative name.
  • A certificate must not be self-signed.

Given that, your options are to either get a certificate from a well-known CA (Let's Encrypt will do, jcli uses Mozilla's CA bundle for verification) or create your own local CA and provide the root certificate to jcli via the --tls-cert-path option.

Creating a local CA using OpenSSL and EasyRSA

EasyRSA is a set of scripts that use OpenSSL and give you an easier experience with setting up your local CA. You can download them here.

  1. Go to easy-rsa/easy-rsa3.

  2. Configure your CA. To do that, create the configuration file (cp vars.example vars); open it with the text editor of your choise (for example, vim vars); uncomment and edit fields you need to change. Each CA needs to edit these lines (find then in your vars file according to their organization structure:

    #set_var.EASYRSA_REQ_COUNTRY----"US" #set_var.EASYRSA_REQ_PROVINCE---"California" #set_var.EASYRSA_REQ_CITY---"San.Francisco" #set_var.EASYRSA_REQ_ORG----"Copyleft.Certificate.Co" #set_var.EASYRSA_REQ_EMAIL--"me@example.net" #set_var.EASYRSA_REQ_OU-----"My.Organizational.Unit"

  3. When your configuration is ready, run ./easyrsa init-pki and ./easyrsa build-ca nopass. You will be prompted to set the name of your CA.

  4. Run ./easyrsa gen-req server nopass to create a new private key and a certificate signing request. You will be prompted to enter the host name (localhost for local testing).

  5. Run ./easyrsa sign-req server server to sign the request.

To use the generated certificate, use it and the corresponding key in your jormungandr config:

rest:
  tls:
    cert_file: <path to server.crt>
    priv_key_file: <path to server.key>

Use the CA certificate with jcli.

P2P configuration

  • trusted_peers: (optional) the list of nodes' multiaddr to connect to in order to bootstrap the p2p topology (and bootstrap our local blockchain). Note that you can use a DNS name in the following format: /dns4/node.example.com/tcp/3000. Use dns6 instead of dns4 if you want the peer to connect with IPv6.
  • public_address: multiaddr the address to listen from and accept connection from. This is the public address that will be distributed to other peers of the network that may find interest into participating to the blockchain dissemination with the node. Currently only TCP is supported.
  • node_key_file: (optional) Path to a file containing a bech32-encoded ed25519 secret key. The keys are used to advertize the node in network gossip and to authenticate a connection to the node if the node is used as a trusted peer. Most of the users don't need to set this value as the key will be randomly generated if the option is not present.
  • listen: (optional) socket address (IP address and port separated by a comma), specifies the interface address and port the node will listen at to receive p2p connection. Can be left empty and the node will listen to whatever value was given to public_address.
  • topics_of_interest: (optional) the different topics we are interested to hear about:
    • messages: notify other peers this node is interested about Transactions typical setting for a non mining node: "low". For a stakepool: "high";
    • blocks: notify other peers this node is interested about new Blocks. typical settings for a non mining node: "normal". For a stakepool: "high".
  • max_connections: the maximum number of P2P connections this node should maintain. If not specified, an internal limit is used by default [default: 256]
  • max_client_connections: the maximum number of client P2P connections this node should keep open. [default: 192]
  • policy: (optional) set the setting for the policy module
    • quarantine_duration set the time to leave a node in quarantine before allowing it back (or not) into the fold. It is recommended to leave the default value [default: 30min].
    • quarantine_whitelist set a trusted list of peers that will not be quarantined in any circumstance. It should be a list of valid addresses, for example: ["/ip4/127.0.0.1/tcp/3000"]. By default this list is empty, [default: []].
  • layers: (optional) set the settings for some of the poldercast custom layers (see below)
  • gossip_interval: (optional) interval to start gossiping with new nodes, changing the value will affect the bandwidth. The more often the node will gossip the more bandwidth the node will need. The less often the node gossips the less good the resilience to node churn. [default: 10s]
  • network-stuck_check: (optional) If no gossip has been received in the last interval, try to connect to nodes that were previously known to this node. This helps to rejoin the protocol in case there is a network outage and the node cannot reach any other peer. [default: 5min]
  • max_bootstrap_attempts: (optional) number of times to retry bootstrapping from trusted peers. If not set, default behavior, the bootstrap process will keep retrying indefinitely, until completed successfully. If set to 0 (zero), the node will skip bootstrap all together -- even if trusted peers are defined. If the node fails to bootstrap from any of the trusted peers and the number of bootstrap retry attempts is exceeded, then the node will continue to run without completing the bootstrap process. This will allow the node to act as the first node in the p2p network (i.e. genesis node), or immediately begin gossip with the trusted peers if any are defined.

The trusted peers

The trusted peers is a concept that is not fully implemented yet. One of the key element for now is that this is the first node any node tries to connect in order to meet new nodes. Right now, as far as we know, only one of them is needed. IOHK provides a few others for redundancy.

Layers

Jörmungandr provides multiple additional layers to the poldercast default ones: the preferred list or the bottle in the sea.

Preferred list

This is a special list that allows to connect multiple nodes together without relying on the auto peer discovery. All entries in the preferred list are also whitelisted automatically, so they cannot be quarantined.

configuration:
  • view_max: this is the number of entries to show in the view each round the layer will randomly select up to view_max entries from the whole preferred_list.peers list of entries. [default: 20]
  • peers: the list of peers to keep in the preferred list [default: EMPTY]

Also, the preferred list will never be quarantined or blacklisted, the node will attempt to connect to (up to view_max of) these nodes every time, even if some are down, unreachable or not operated anymore.

COMPATIBILITY NOTE: in near future the peer list will be only a list of addresses and the ID part will not be necessary.

Example:
p2p:
  layers:
    preferred_list:
      view_max: 20
      peers:
        - address: '/ip4/127.0.0.1/tcp/2029'
          id: 019abc...
        - ...

Setting the public_id

This is needed to advertise your node as a trusted peer. If not set, the node will generate a random ID, which is fine for a regular user. You can generate a public id with openssl, for example: openssl rand -hex 24

topics_of_interest

This is an optional value to set. The default is:

messages: low
blocks: normal

These values make sense for most of the users that are not running stake pools or that are not even publicly reachable.

However for a publicly reachable node, the recommended settings would be:

messages: normal
blocks: normal

and for a stake pool:

messages: high
blocks: high

Mempool

When running an active node (BFT leader or stake pool) it is interesting to be able to make choices on how to manage the pending transactions: how long to keep them, how to prioritize them etc.

The mempool field in your node config file is not mandatory, by default it is set as follow:

mempool:
    pool_max_entries: 10000
    log_max_entries: 100000
  • pool_max_entries: (optional, default is 10000). Set a maximum size of the mempool
  • log_max_entries: (optional, default is 100000). Set a maximum size of fragment logs
  • persistent_log: (optional, disabled by default) log all incoming fragments to log files, rotated on a hourly basis. The value is an object, with the dir field specifying the directory name where log files are stored.

Persistent logs

A persistent log is a collection of records comprised of a UNIX timestamp of when a fragment was registereed by the mempool followed by the hex-encoded fragment body. This log is a line-delimited JSON stream.

Keep in mind that enabling persistent logs could result in impaired performance of the node if disk operations are slow. Consider using a reasonably fast ssd for best results.

Leadership

The leadership field in your node config file is not mandatory, by default it is set as follow:

leadership:
    logs_capacity: 1024
  • logs_capacity: the maximum number of logs to keep in memory. Once the capacity is reached, older logs will be removed in order to leave more space for new ones [default: 1024]

jcli

This is the node command line helper. It is mostly meant for developers and stake pool operators. It allows offline operations:

  • generating cryptographic materials for the wallets and stake pools;
  • creating addresses, transactions and certificates;
  • prepare a new blockchain

and it allows simple interactions with the node:

  • query stats;
  • send transactions and certificates;
  • get raw blocks and UTxOs.

cryptographic keys

There are multiple type of key for multiple use cases.

typeusage
Ed25519Signing algorithm for Ed25519 algorithm
Ed25519Bip32Related to the HDWallet, Ed25519 Extended with chain code for derivation
Ed25519ExtendedRelated to Ed25519Bip32 without the chain code
SumEd25519_12For stake pool, necessary for the KES
RistrettoGroup2HashDhFor stake pool, necessary for the VRF

There is a command line parameter to generate this keys:

$ jcli key generate --type=Ed25519
ed25519_sk1cvac48ddf2rpk9na94nv2zqhj74j0j8a99q33gsqdvalkrz6ar9srnhvmt

and to extract the associated public key:

$ echo ed25519_sk1cvac48ddf2rpk9na94nv2zqhj74j0j8a99q33gsqdvalkrz6ar9srnhvmt | jcli key to-public
ed25519_pk1z2ffur59cq7t806nc9y2g64wa60pg5m6e9cmrhxz9phppaxk5d4sn8nsqg

Signing data

Sign data with private key. Supported key formats are: ed25519, ed25519bip32, ed25519extended and sumed25519_12.

jcli key sign <options> <data>

The options are

  • --secret-key <secret_key> - path to file with bech32-encoded secret key
  • -o, --output <output> - path to file to write signature into, if no value is passed, standard output will be used

<data> - path to file with data to sign, if no value is passed, standard input will be used

Verifying signed data

Verify signed data with public key. Supported key formats are: ed25519, ed25519bip32 and sumed25519_12.

jcli key verify <options> <data>

The options are

  • --public-key <public_key> - path to file with bech32-encoded public key
  • --signature <signature> - path to file with signature

<data> - path to file with data to sign, if no value is passed, standard input will be used

Address

Jormungandr comes with a separate CLI to create and manipulate addresses.

This is useful for creating addresses from their components in the CLI, for debugging addresses and for testing.

Display address info

To display an address and verify it is in a valid format you can utilise:

$ jcli address info ta1svy0mwwm7mdwcuj308aapjw6ra4c3e6cygd0f333nvtjzxg8ahdvxlswdf0
discrimination: testing
public key: ed25519e_pk1pr7mnklkmtk8y5tel0gvnksldwywwkpzrt6vvvvmzus3jpldmtpsx9rnmx

or for example:

$ jcli address \
    info \
    ca1qsy0mwwm7mdwcuj308aapjw6ra4c3e6cygd0f333nvtjzxg8ahdvxz8ah8dldkhvwfghn77se8dp76uguavzyxh5cccek9epryr7mkkr8n7kgx
discrimination: production
public key: ed25519_pk1pr7mnklkmtk8y5tel0gvnksldwywwkpzrt6vvvvmzus3jpldmtpsx9rnmx
group key:  ed25519_pk1pr7mnklkmtk8y5tel0gvnksldwywwkpzrt6vvvvmzus3jpldmtpsx9rnmx

Creating an address

Each command following allows to create addresses for production and testing chains. For chains, where the discrimination is testing, you need to use the --testing flag.

There's 3 types of addresses:

  • Single address : A simple spending key. This doesn't have any stake in the system
  • Grouped address : A spending key attached to an account key. The stake is automatically
  • Account address : An account key. The account is its own stake

Address for UTxO

You can create a single address (non-staked) using the spending public key for this address utilising the following command:

$ jcli address \
    single ed25519e_pk1jnlhwdgzv3c9frknyv7twsv82su26qm30yfpdmvkzyjsdgw80mfqduaean
ca1qw207ae4qfj8q4yw6v3ned6psa2r3tgrw9u3y9hdjcgj2p4pcaldyukyka8

To add the staking information and make a group address, simply add the account public key as a second parameter of the command:

$ jcli address \
    single \
    ed25519_pk1fxvudq6j7mfxvgk986t5f3f258sdtw89v4n3kr0fm6mpe4apxl4q0vhp3k \
    ed25519_pk1as03wxmy2426ceh8nurplvjmauwpwlcz7ycwj7xtl9gmx9u5gkqscc5ylx
ca1q3yen35r2tmdye3zc5lfw3x992s7p4dcu4jkwxcda80tv8xh5ym74mqlzudkg42443nw08cxr7e9hmcuzals9ufsa9uvh723kvteg3vpvrcxcq

Address for Account

To create an account address you need the account public key and run:

$ jcli address \
    account ed25519_pk1c4yq3hflulynn8fef0hdq92579n3c49qxljasrl9dnuvcksk84gs9sqvc2
ca1qhz5szxa8lnujwva8997a5q42nckw8z55qm7tkq0u4k03nz6zc74ze780qe

changing the address prefix

You can decide to change the address prefix, allowing you to provide more enriched data to the user. However, this prefix is not forwarded to the node, it is only for UI/UX.

$ jcli address \
    account \
    --prefix=address_ \
    ed25519_pk1yx6q8rsndawfx8hjzwntfs2h2c37v5g6edv67hmcxvrmxfjdz9wqeejchg
address_1q5smgquwzdh4eyc77gf6ddxp2atz8ej3rt94nt6l0qes0vexf5g4cw68kdx

Transaction

Tooling for offline transaction creation and signing.

jcli transaction

Those familiar with cardano-cli transaction builder will see resemblance in jcli transaction.

There is a couple of commands that can be used to:

  1. prepare a transaction:
    • new create a new empty transaction;
    • add-input
    • add-account
    • add-output
  2. finalize the transaction for signing:
  3. create witnesses and add the witnesses:
    • make-witness
    • add-witness
  4. seal the transaction, ready to send to the blockchain
  5. auth the transaction, if it contains a certificate

There are also functions to help decode and display the content information of a transaction:

  • info displays summary of transaction being constructed
  • data-for-witness get the data to sign from a given transaction
  • fragment-id get the Fragment ID from a transaction in sealed state
  • to-message to get the hexadecimal encoded message, ready to send with cli rest message

DEPRECATED:

  • id get the data to sign from a given transaction (use data-for-witness instead)

Transaction info

On every stage of building a transaction user can display its summary

jcli transaction info <options>

The options are:

  • --prefix <address-prefix> - set the address prefix to use when displaying the addresses (default: ca)

  • --fee-certificate <certificate> - fee per certificate (default: 0)

  • --fee-coefficient <coefficient> - fee per every input and output (default: 0)

  • --fee-constant <constant> - fee per transaction (default: 0)

  • --fee-owner-stake-delegation <certificate-owner-stake-delegation> - fee per owner stake delegation (default: fee-certificate)

  • --fee-pool-registration <certificate-pool-registration> - fee per pool registration (default: fee-certificate)

  • --fee-stake-delegation <certificate-stake-delegation> - fee per stake delegation (default: fee-certificate)

  • --fee-vote-cast <certificate-vote-cast> - fee per vote cast

  • --fee-vote-plan <certificate-vote-plan> - fee per vote plan

  • --output-format <format> - Format of output data. Possible values: json, yaml. Any other value is treated as a custom format using values from output data structure. Syntax is Go text template: https://golang.org/pkg/text/template/. (default: yaml)

  • --output <output> - write the info in the given file or print it to the standard output

  • --staging <staging-file> - place where the transaction is going to be saved during its staging phase. If a file is given, the transaction will be read from this file and modification will be written into this same file. If no file is given, the transaction will be read from the standard input and will be rendered in the standard output.

YAML printed on success

---
balance: 40         # transaction balance or how much input is not spent
fee: 60             # total fee for transaction
input: 200          # total input of transaction
inputs:             # list of transaction inputs, each can be of either "utxo" or "account" kind
  - index: 4        # index of transaction output
    kind: utxo      # constant value, signals that UTxO is used
                    # hex-encoded ID of transaction
    txid: 543326b2739356ab6d14624a536ca696f1020498b36456b7fdfe8344c084bfcf
    value: 130      # value of transaction output
  -                 # hex-encoded account address
    account: 3fd45a64ae5a3b9c35e37114baa099b8b01285f7d74b371597af22d5ff393d9f
    kind: account   # constant value, signals that account is used
    value: 70       # value taken from account
num_inputs: 1       # total number of inputs of transaction
num_outputs: 1      # total number of outputs of transaction
num_witnesses: 1    # total number of witnesses of transaction
output: 100         # total output of transaction
outputs:            # list of transaction outputs
  -                 # bech32-encoded address
    address: ca1swedukl830v26m8hl7e5dzrjp77yctuz79a68r8jl2l79qnpu3uwz0kg8az
    value: 100      # value sent to address
                    # hex-encoded transaction hash, when transaction is complete, it's also its ID
sign_data_hash: 26be0b8bd7e34efffb769864f00d7c4aab968760f663a7e0b3ce213c4b21651b
status: sealed      # transaction status, can be "balancing", "finalizing", "sealed" or "authed"

Examples

The following example focuses on using an utxo as input, the few differences when transfering from an account will be pointed out when necessary. Also, the simplified make-transaction command in jcli covers all this process. For more information run:

jcli transaction make-transaction --help

Let's use the following utxo as input and transfer 50 lovelaces to the destination address

Input utxo

FieldValue
UTXO's transaction ID55762218e5737603e6d27d36c8aacf8fcd16406e820361a8ac65c7dc663f6d1c
UTXO's output index0
associated addressca1q09u0nxmnfg7af8ycuygx57p5xgzmnmgtaeer9xun7hly6mlgt3pjyknplu
associated value100

Destination address

address: ca1qvnr5pvt9e5p009strshxndrsx5etcentslp2rwj6csm8sfk24a2wlqtdj6

Create a staging area

jcli transaction new --staging tx

Add input

For the input, we need to reference the utxo with the UTXO's transaction ID and UTXO'S output index fields and we need to specify how many coins there are with the associated value field.

Example - UTXO address as Input

jcli transaction add-input 55762218e5737603e6d27d36c8aacf8fcd16406e820361a8ac65c7dc663f6d1c 0 100 --staging tx

Example - Account address as Input

If the input is an account, the command is slightly different

jcli transaction add-account account_address account_funds --staging tx

Add output

For the output, we need the address we want to transfer to, and the amount.

jcli transaction add-output ca1qvnr5pvt9e5p009strshxndrsx5etcentslp2rwj6csm8sfk24a2wlqtdj6 50 --staging tx

Add fee and change address

We want to get the change in the same address that we are sending from (the associated address of the utxo). We also specify how to compute the fees. You can leave out the --fee-constant 5 --fee-coefficient 2 part if those are both 0.

jcli transaction finalize ca1q09u0nxmnfg7af8ycuygx57p5xgzmnmgtaeer9xun7hly6mlgt3pjyknplu --fee-constant 5 --fee-coefficient 2 --staging tx

Now, if you run

jcli transaction info --fee-constant 5 --fee-coefficient 2 --staging tx

You should see something like this

---
balance: 0
fee: 11
input: 100
inputs:
  - index: 0
    kind: utxo
    txid: 55762218e5737603e6d27d36c8aacf8fcd16406e820361a8ac65c7dc663f6d1c
    value: 100
num_inputs: 1
num_outputs: 2
num_witnesses: 0
output: 89
outputs:
  - address: ca1qvnr5pvt9e5p009strshxndrsx5etcentslp2rwj6csm8sfk24a2wlqtdj6
    value: 50
  - address: ca1q09u0nxmnfg7af8ycuygx57p5xgzmnmgtaeer9xun7hly6mlgt3pjyknplu
    value: 39
sign_data_hash: 0df39a87d3f18a188b40ba8c203f85f37af665df229fb4821e477f6998864273
status: finalizing

Sign the transaction

Make witness

For signing the transaction, you need:

  • the hash of the genesis block of the network you are connected to.
  • the private key associated with the input address (the one that's in the utxos).
  • the hash of the transaction, that can be retrieved in two ways:
    1. sign_data_hash value from jcli transaction info --staging tx or
    2. jcli transaction data-for-witness --staging tx

The genesis' hash is needed for ensuring that the transaction cannot be re-used in another blockchain and for security concerns on offline transaction signing, as we are signing the transaction for the specific blockchain started by this block0 hash.

First we need to get the hash of the transaction we are going to sign.

jcli transaction data-for-witness --staging tx

You should see something like this (the value may be different since it depends on the input/output data)

0df39a87d3f18a188b40ba8c203f85f37af665df229fb4821e477f6998864273

The following command takes the private key in the key.prv file and creates a witness in a file named witness in the current directory.

jcli transaction make-witness --genesis-block-hash abcdef987654321... --type utxo 0df39a87d3f18a188b40ba8c203f85f37af665df229fb4821e477f6998864273 witness key.prv

Account input

When using an account as input, the command takes account as the type and an additional parameter: --account-spending-counter, that should be increased every time the account is used as input.

e.g.

jcli transaction make-witness --genesis-block-hash abcdef987654321... --type account --account-spending-counter 0 0df39a87d3f18a188b40ba8c203f85f37af665df229fb4821e477f6998864273 witness key.prv

Add witness

jcli transaction add-witness witness --staging tx

Send the transaction

jcli transaction seal --staging tx

jcli transaction to-message --staging tx > txmsg

Send it using the rest api

jcli rest v0 message post -f txmsg --host http://127.0.0.1:8443/api

You should get some data back referring to the TransactionID (also known as FragmentID)

d6ef0b2148a51ed64531efc17978a527fd2d2584da1e344a35ad12bf5460a7e2

Checking if the transaction was accepted

You can check if the transaction was accepted by checking the node logs, for example, if the transaction is accepted

jcli rest v0 message logs -h http://127.0.0.1:8443/api

---
- fragment_id: d6ef0b2148a51ed64531efc17978a527fd2d2584da1e344a35ad12bf5460a7e2
  last_updated_at: "2019-06-11T15:38:17.070162114Z"
  received_at: "2019-06-11T15:37:09.469101162Z"
  received_from: Rest
  status:
    InABlock:
      date: "4.707"
      block: "d9040ca57e513a36ecd3bb54207dfcd10682200929cad6ada46b521417964174"

Where the InABlock status means that the transaction was accepted in the block with date "4.707" and for block d9040ca57e513a36ecd3bb54207dfcd10682200929cad6ada46b521417964174.

The status here could also be:

Pending: if the transaction is received and is pending being added in the blockchain (or rejected).

or

Rejected: with an attached message of the reason the transaction was rejected.

Certificate

Tooling for offline transaction creation

Building stake pool registration certificate

Builds a stake pool registration certificate.

jcli certificate new stake-pool-registration \
    --vrf-key <vrf-public-key> \
    --kes-key <kes-public-key> \
    --start-validity <seconds-since-start> \
    --management-threshold <THRESHOLD> \
    --owner <owner-public-key> \
    [--operator <operator-public-key>] \
    [<output-file>]

Where:

  • --operator <operator-public-key> - optional, public key of the operator(s) of the pool.
  • output-file - optional, write the output to the given file or print it to the standard output if not defined

Retiring a stake pool

It is possible to retire a stake pool from the blockchain. By doing so the stake delegated to the stake pool will become dangling and will need to be re-delegated.

Remember though that the action won't be applied until the next following epoch. I.e. the certificate will take a whole epoch before being applied, this should leave time for stakers to redistribute their stake to other pools before having their stake becoming dangling.

It might be valuable for a stake pool operator to keep the stake pool running until the stake pool retirement certificate is fully applied in order to not miss any potential rewards.

example:

jcli certificate new stake-pool-retirement \
    --pool-id <STAKE_POOL_ID> \
    --retirement-time <seconds-since-start> \
    [<output-file>]

where:

  • output-file - optional, write the output to the given file or print it to the standard output if not defined.
  • --retirement-time - is the number of seconds since the start in order to make the stake pool retire. 0 means as soon as possible.
  • --pool-id - hex-encoded stake pool ID. Can be retrieved using jcli certificate get-stake-pool-id command. See here for more details.

Building stake pool delegation certificate

Builds a stake pool delegation certificate.

jcli certificate new stake-delegation <STAKE_KEY> <STAKE_POOL_IDS> [--output <output-file>]

Where:

  • -o, --output <output-file> - optional, write the output to the given file or print it to the standard output if not defined
  • <STAKE_KEY> - the public key used in the stake key registration
  • <STAKE_POOL_IDS>... - hex-encoded stake pool IDs and their numeric weights in format "pool_id:weight". If weight is not provided, it defaults to 1.

Building update proposal certificate

Builds an update proposal certificate.

jcli certificate new update-proposal \
    <PROPOSER_ID> \
    <CONFIG_FILE> \
    [<output-file>]

Where:

  • <PROPOSER_ID> - the proposer ID, public key of the one who will sign this certificate
  • <CONFIG_FILE> - optional, the file path to the config file defining the config param changes If omitted it will be read from the standard input.
  • output-file - optional, write the output to the given file or print it to the standard output if not defined

For example your config file may look like:


  # The block0-date defines the date the blockchain starts
  # expected value in seconds since UNIX_EPOCH
  #
  # By default the value will be the current date and time. Or you can
  # add a specific time by entering the number of seconds since UNIX
  # Epoch
- Block0Date: 17

  # This is the type of discrimination of the blockchain
  # if this blockchain is meant for production then
  # use 'production' otherwise use 'test'.
- Discrimination: test

  # The initial consensus version:
  #
  # * BFT consensus: bft
  # * Genesis Praos consensus: genesis
- ConsensusVersion: bft

  # Number of slots in each epoch.
- SlotsPerEpoch: 42

  # The slot duration, in seconds, is the time between the creation
  # of 2 blocks
- SlotDuration: 79

  # Epoch stability depth
- EpochStabilityDepth: 12

  # Genesis praos active slot coefficient
  # Determines minimum stake required to try becoming slot leader, must be in range (0,1]
- ConsensusGenesisPraosActiveSlotsCoeff: "0.004"

  # This is the size, in bytes, of all the contents of the block (excluding the
  # block header).
- BlockContentMaxSize: 96

  # Add a new bft leader
- AddBftLeader: ed25519_pk1g53asm6l4gcwk2pm5ylr092umaur5yes47rqv7ng5yl525x8g8mq5nk7x7

  # Remove a bft leader
- RemoveBftLeader: ed25519_pk1a3sjcg6gt4d05k5u6uqyzmsap8cjw37ul9cgztz8m697lvkz26uqg49nm3

  # The fee calculations settings
  #
  # total fees: constant + (num_inputs + num_outputs) * coefficient [+ certificate]
- LinearFee:
    # this is the minimum value to pay for every transaction
    constant: 57
    # the additional fee to pay for every inputs and outputs
    coefficient: 14
    # the additional fee to pay if the transaction embeds a certificate
    certificate: 95
    # (optional) fees for different types of certificates, to override the one
    # given in `certificate` just above
    #
    # here: all certificate fees are set to `4` except for pool registration
    # and stake delegation which are respectively `5` and `2`.
    per_certificate_fees:
      # (optional) if not specified, the pool registration certificate fee will be
      # the one set by linear_fees.certificate
      certificate_pool_registration: 5
      # (optional) if not specified, the delegation certificate fee will be
      # the one set by linear_fees.certificate
      certificate_stake_delegation: 2
      # (optional) if not specified, the owner delegation certificate fee will be
      # the one set by linear_fees.certificate. Uncomment to set the owner stake
      # delegation to `1` instead of default `4`:
      certificate_owner_stake_delegation: 1

  # Proposal expiration in epochs
- ProposalExpiration: 68

  # The speed to update the KES Key in seconds
- KesUpdateSpeed: 120

  # Increase the treasury amount
- TreasuryAdd: 10000

  # Set the total reward supply available for monetary creation
- RewardPot: 100000000000000

  # Set the treasury parameters, this is the tax type, just as in stake pool
  # registration certificate parameters.
  #
  # When distributing the rewards, the treasury will be first serve as per
  # the incentive specification document
- TreasuryParams:
    # the fix value the treasury will take from the total reward pot of the epoch
    fixed: 1000
    # the extra percentage the the treasury will take from the reward pot of the epoch
    ratio: "1/10"
    # it is possible to add a max bound to the total value the treasury takes
    # at each reward distribution. For example, one could cap the treasury tax
    # to 10000. Uncomment the following line to apply a max limit:
    max_limit: 10000

  # set the reward supply consumption. These parameters will define how the
  # total_reward_supply is consumed for the stake pool reward
  #
  # There's fundamentally many potential choices for how rewards are contributed back, and here's two potential valid examples:
  #
  # Linear formula: constant - ratio * (#epoch after epoch_start / epoch_rate)
  # Halving formula: constant * ratio ^ (#epoch after epoch_start / epoch_rate)
- RewardParams:
    halving: # or use "linear" for the linear formula
      # In the linear formula, it represents the starting point of the contribution
      # at #epoch=0, whereas in halving formula is used as starting constant for
      # the calculation.
      constant: 2

      # In the halving formula, an effective value between 0.0 to 1.0 indicates a
      # reducing contribution, whereas above 1.0 it indicate an acceleration of contribution.
      #
      # However in linear formula the meaning is just a scaling factor for the epoch zone
      # (current_epoch - start_epoch / epoch_rate). Further requirement is that this ratio
      # is expressed in fractional form (e.g. 1/2), which allow calculation in integer form.
      ratio: 3/68

      # indicates when this contribution start. note that if the epoch is not
      # the same or after the epoch_start, the overall contribution is zero.
      epoch_start: 89

      # the rate at which the contribution is tweaked related to epoch.
      epoch_rate: 20

  # Fees for different types of certificates, to override the one
  # given in `certificate` just above.
- PerCertificateFees:
    # (optional) if not specified, the pool registration certificate fee will be
    # the one set by linear_fees.certificate
    certificate_pool_registration: 5
    # (optional) if not specified, the delegation certificate fee will be
    # the one set by linear_fees.certificate
    certificate_stake_delegation: 2
    # (optional) if not specified, the owner delegation certificate fee will be
    # the one set by linear_fees.certificate. Uncomment to set the owner stake
    # delegation to `1` instead of default `4`:
    certificate_owner_stake_delegation: 1

  # Set where to send the fees generated by transactions activity.
  #
  # It is possible to send all the generated fees to the "treasury"
- FeesInTreasury: rewards

- RewardLimitNone

  # Limit the epoch total reward drawing limit to a portion of the total
  # active stake of the system.
  #
  # for example, if set to 10%, the reward drawn will be bounded by the
  # 10% of the total active stake.
- RewardLimitByAbsoluteStake: 22/72

  # Settings to incentivize the numbers of stake pool to be registered
  # on the blockchain.
  #
  # These settings does not prevent more stake pool to be added. For example
  # if there is already 1000 stake pools, someone can still register a new
  # stake pool and affect the rewards of everyone else too.
  #
  # if the threshold is reached, the pool doesn't really have incentive to
  # create more blocks than 1 / set-value-of-pools % of stake.
- PoolRewardParticipationCapping:
    min: 8
    max: 52

  # Add a new committee id
- AddCommitteeId: 8103973beaa56f4e9440004ee8e8f8359ea18499d4199c1b018c072e7f503ea0

  # Remove a committee id
- RemoveCommitteeId: 6375dcdd714e69c197e99c32486ec28f166a50da7a1e3694807cd8a76f1c8175

- PerVoteCertificateFees:
    certificate_vote_plan: 52
    certificate_vote_cast: 57

  # The transaction max expiry epochs
- TransactionMaxExpiryEpochs: 91

Building update vote certificate

Builds an update proposal certificate.

jcli certificate new update-proposal \
    <PROPOSAL_ID> \
    <VOTER_ID> \
    [<output-file>]

Where:

  • <PROPOSAL_ID> - the proposal ID of the proposal, it is a corresponding update proposal fragment id
  • <VOTER_ID> - the voter ID, public key of the one who will sign this certificate
  • output-file - optional, write the output to the given file or print it to the standard output if not defined

Genesis

Tooling for working with a genesis file

Usage

jcli genesis [subcommand]

Subcommands

  • decode: Print the YAML file corresponding to an encoded genesis block.
  • encode: Create the genesis block of the blockchain from a given yaml file.
  • hash: Print the block hash of the genesis
  • init: Create a default Genesis file with appropriate documentation to help creating the YAML file
  • help

Examples

Encode a genesis file

jcli genesis encode --input genesis.yaml --output block-0.bin

or equivantely

cat genesis.yaml | jcli genesis encode > block-0.bin

Get the hash of an encoded genesis file

jcli genesis hash --input block-0.bin

Voting

Jormungandr supports decentralized voting with privacy features.

The voting process is controlled by a committee whose private keys can be used to decrypt and certify the tally.

Creating committee keys

Private

Please refer to jcli votes committee --help for help with the committee related cli operations and specification of arguments.

In this example we will be using 3 kind of keys for the private vote and tallying.

In order:

Committee communication key

jcli votes committee communication-key generate > ./comm.sk

We can get its public representation with:

jcli votes committee communication-key to-public --input ./comm.sk > ./comm.pk

Committee member key

jcli votes committee member-key generate --threshold 3 --crs "$crs" --index 0 --keys pk1 pk2 pk3 > ./member.sk

Where pkX are each of the committee communication public keys in bech32 format. The order of the keys shall be the same for every member invoking the command, and the --index parameter provides the 0-based index of the member this key is generated for. Note that all committee members shall use the same CRS.

We can also easily get its public representation as before:

jcli votes committee member-key to-public --input ./member.sk ./member.pk

Election public key

This key (public) is the key every vote should be encrypted with.

jcli votes election-key --keys mpk1 mpk2 mpk3 > ./vote.pk

Notice that we can always rebuild this key with the committee member public keys found within the voteplan certificate.

jcli rest v0 vote active plans > voteplan.json

Creating a vote plan

We need to provide a vote plan definition file to generate a new voteplan certificate. That file should be a yaml (or json) with the following format:

{
  "payload_type": "private",
  "vote_start": {
    "epoch": 1,
    "slot_id": 0
  },
  "vote_end": {
    "epoch": 3,
    "slot_id": 0
  },
  "committee_end": {
    "epoch": 6,
    "slot_id": 0
  },
  "proposals": [
    {
      "external_id": "d7fa4e00e408751319c3bdb84e95fd0dcffb81107a2561e691c33c1ae635c2cd",
      "options": 3,
      "action": "off_chain"
    },
    ...
  ],
  "committee_member_public_keys": [
    "pk....",
  ]
}

Where:

  • payload_type is either public or private
  • commitee_public_keys is only needed for private voting, can be empty for public.

Then, we can generate the voteplan certificate with:

jcli certificate new vote-plan voteplan_def.json --output voteplan.certificate

Casting votes

TBA

Tallying

Public vote plan

To tally public votes, a single committee member is sufficient. In the example below, the file committee.sk contains the committee member's private key in bech32 format, and block0.bin contains the genesis block of the voting chain.

genesis_block_hash=$(jcli genesis hash < block0.bin)
vote_plan_id=$(jcli rest v0 vote active plans get --output-format json|jq '.[0].id')
committee_addr=$(jcli address account $(jcli key to-public < committee.sk))
committee_addr_counter=$(jcli rest v0 account get "$committee_addr" --output-format json|jq .counter)
jcli certificate new vote-tally --vote-plan-id "$vote_plan_id" --output vote-tally.certificate
jcli transaction new --staging vote-tally.staging
jcli transaction add-account "$committee_addr" 0 --staging vote-tally.staging
jcli transaction add-certificate $(< vote-tally.certificate) --staging vote-tally.staging
jcli transaction finalize --staging vote-tally.staging
jcli transaction data-for-witness --staging vote-tally.staging > vote-tally.witness-data
jcli transaction make-witness --genesis-block-hash "$genesis_block_hash" --type account --account-spending-counter "$committee_addr_counter" $(< vote-tally.witness-data) vote-tally.witness committee.sk
jcli transaction add-witness --staging vote-tally.staging vote-tally.witness
jcli transaction seal --staging vote-tally.staging
jcli transaction auth --staging vote-tally.staging --key committee.sk
jcli transaction to-message --staging vote-tally.staging > vote-tally.fragment
jcli rest v0 message post --file vote-tally.fragment

Private

To tally private votes, all committee members are needed. The process is similar to the public one, but we need to issue different certificates.

First, we need to retrieve vote plans info:

jcli rest v0 vote active plans > active_plans.json

If there is more than one vote plan in the file, we also need to provide the id of the vote plan we are interested in to following commands. We can get the id of the first vote plan with:

...
vote_plan_id=$(cat active_plans.json |jq '.[0].id')
...

Each committee member needs to generate their shares for the vote plan, which we will use later to decrypt the tally.

jcli votes tally decryption-shares --vote-plan active_plans.json --vote-plan-id $"vote_plan_id" --key member.sk --output-format json

Then, the committee members need to exchange their shares (only one full set of shares is needed). Once all shares are available, we need to merge them in a single file with the following command (needed even if there is only one set of shares):

jcli votes tally merge-shares  share_file1 share_file2 ... > merged_shares.json

With the merged shares file, we are finally able to process the final tally result as follows:

jcli votes tally decrypt-results \
--vote-plan active_plans.json \
--vote-plan-id $"vote_plan_id" \
--shares merged_shares.json \
--threshold number_of_committee_members \
--output-format json > result.json

REST

Jormungandr comes with a CLI client for manual communication with nodes over HTTP.

Conventions

Many CLI commands have common arguments:

  • -h <addr> or --host <addr> - Node API address. Must always have http:// or https:// prefix. E.g. -h http://127.0.0.1, --host https://node.com:8443/cardano/api
  • --debug - Print additional debug information to stderr. The output format is intentionally undocumented and unstable
  • --output-format <format> - Format of output data. Possible values: json, yaml, default yaml. Any other value is treated as a custom format using values from output data structure. Syntax is Go text template: https://golang.org/pkg/text/template/.

Node stats

Fetches node stats

jcli rest v0 node stats get <options>

The options are

YAML printed on success

---
# Number of blocks received by node
blockRecvCnt: 1102
# Size in bytes of all transactions in last block
lastBlockContentSize: 484
# The Epoch and slot Number of the block (optional)
lastBlockDate: "20.29"
# Sum of all fee values in all transactions in last block
lastBlockFees: 534
# The block hash, it's unique identifier in the blockchain (optional)
lastBlockHash: b9597b45a402451540e6aabb58f2ee4d65c67953b338e04c52c00aa0886bd1f0
# The block number, in order, since the block0 (optional)
lastBlockHeight: 202901
# Sum of all input values in all transactions in last block
lastBlockSum: 51604
# The time slot of the tip block
lastBlockTime: "2020-01-30T22:37:46+00:00"
# Number of transactions in last block
lastBlockTx: 2
# The time at which we received the last block, not necessarily the current tip block (optional)
lastReceivedBlockTime: "2020-01-30T22:37:59+00:00"
# 24 bytes encoded in hexadecimal Node ID
nodeId: "ad24537cb009bedaebae3d247fecee9e14c57fe942e9bb0d"
# Number of nodes that are available for p2p discovery and events propagation
peerAvailableCnt: 321
# Number of nodes that have been quarantined by our node
peerQuarantinedCnt: 123
# Total number of nodes
peerTotalCnt: 444
# Number of nodes that are connected to ours but that are not publicly reachable
peerUnreachableCnt: 0
# State of the node
state: Running
# Number of transactions received by node
txRecvCnt: 5440
# Node uptime in seconds
uptime: 20032
# Node app version
version: jormungandr 0.8.9-30d20d2e

Get UTxO

Fetches UTxO details

jcli rest v0 utxo <fragment-id> <output-index> get <options>
  • <fragment-id> - hex-encoded ID of the transaction fragment
  • <output-index> - index of the transaction output

The options are

YAML printed on success

---
# UTxO owner address
address: ca1svs0mwkfky9htpam576mc93mee5709khre8dgnqslj6y3p5f77s5gpgv02w
# UTxO value
value: 10000

Post transaction

Posts a signed, hex-encoded transaction

jcli rest v0 message post <options>

The options are

  • -h <node_addr> - see conventions
  • --debug - see conventions
  • -f --file <file_path> - File containing hex-encoded transaction. If not provided, transaction will be read from stdin.

Fragment Id is printed on success (which can help finding transaction status using get message log command)

50f21ac6bd3f57f231c4bf9c5fff7c45e2529c4dffed68f92410dbf7647541f1

Get message log

Get the node's logs on the message pool. This will provide information on pending transaction, rejected transaction and or when a transaction has been added in a block

jcli rest v0 message logs <options>

The options are

YAML printed on success

---
- fragment_id: 7db6f91f3c92c0aef7b3dd497e9ea275229d2ab4dba6a1b30ce6b32db9c9c3b2 # hex-encoded fragment ID
  last_updated_at: 2019-06-02T16:20:26.201000000Z                               # RFC3339 timestamp of last fragment status change
  received_at: 2019-06-02T16:20:26.201000000Z                                   # RFC3339 timestamp of fragment receivement
  received_from: Network,                                                       # how fragment was received
  status: Pending,                                                              # fragment status

received_from can be one of:

received_from: Rest     # fragment was received from node's REST API

received_from: Network  # fragment was received from the network

status can be one of:

status: Pending                 # fragment is pending

status:
  Rejected:                     # fragment was rejected
    reason: reason of rejection # cause

status:                         # fragment was included in a block
  InABlock:
    date: "6637.3"            # block epoch and slot ID formed as <epoch>.<slot_id>
    block: "d9040ca57e513a36ecd3bb54207dfcd10682200929cad6ada46b521417964174"

Blockchain tip

Retrieves a hex-encoded ID of the blockchain tip

jcli rest v0 tip get <options>

The options are

Get block

Retrieves a hex-encoded block with given ID

jcli rest v0 block <block_id> get <options>
  • <block_id> - hex-encoded block ID

The options are

Get next block ID

Retrieves a list of hex-encoded IDs of descendants of block with given ID. Every list element is in separate line. The IDs are sorted from closest to farthest.

jcli rest v0 block <block_id> next-id get <options>
  • <block_id> - hex-encoded block ID

The options are

  • -h <node_addr> - see conventions
  • --debug - see conventions
  • -c --count <count> - Maximum number of IDs, must be between 1 and 100, default 1

Get account state

Get account state

jcli rest v0 account get <account-id> <options>
  • <account-id> - ID of an account, bech32-encoded

The options are

YAML printed on success

---
counter: 1
delegation: c780f14f9782770014d8bcd514b1bc664653d15f73a7158254730c6e1aa9f356
value: 990
  • value is the current balance of the account;
  • counter is the number of transactions performed using this account this is useful to know when signing new transactions;
  • delegation is the Stake Pool Identifier the account is delegating to. it is possible this value is not set if there is no delegation certificate sent associated to this account.

Node settings

Fetches node settings

jcli rest v0 settings get <options>

The options are

YAML printed on success

---
block0Hash: 8d94ecfcc9a566f492e6335858db645691f628b012bed4ac2b1338b5690355a7  # block 0 hash of
block0Time: "2019-07-09T12:32:51+00:00"         # block 0 creation time of
blockContentMaxSize: 102400                     # the block content's max size in bytes
consensusVersion: bft                           # currently used consensus
currSlotStartTime: "2019-07-09T12:55:11+00:00"  # current slot start time
epochStabilityDepth: 102400                     # the depth, number of blocks, to which we consider the blockchain to be stable and prevent rollback beyond that depth
fees:                                           # transaction fee configuration
  certificate: 4                                # fee per certificate
  coefficient: 1                                # fee per every input and output
  constant: 2                                   # fee per transaction
  per_certificate_fees:                         # fee per certificate operations, all zero if this object absent (optional)
    certificate_pool_registration: 5            # fee per pool registration, zero if absent (optional)
    certificate_stake_delegation: 15            # fee per stake delegation, zero if absent (optional)
    certificate_owner_stake_delegation: 2       # fee per pool owner stake delegation, zero if absent (optional)
rewardParams:                                   # parameters for rewards calculation
  compoundingRatio:                             # speed at which reward is reduced. Expressed as numerator/denominator
    denominator: 1024
    numerator: 1
  compoundingType: Linear                       # reward reduction algorithm. Possible values: "Linear" and "Halvening"
  epochRate: 100                                # number of epochs between reward reductions
  epochStart: 0                                 # epoch when rewarding starts
  initialValue: 10000                           # initial reward
slotDuration: 5                                 # slot duration in seconds
slotsPerEpoch: 720                              # number of slots per epoch
treasuryTax:                                    # tax from reward that goes to pot
  fixed: 5                                      # what get subtracted as fixed value
  ratio:                                        # ratio of tax after fixed amount is subtracted. Expressed as numerator/denominator
    numerator: 1
    denominator: 10000
  max: 100                                      # limit of tax (optional)

Node shutdown

Node shutdown

jcli rest v0 shutdown get <options>

The options are

Get leaders

Fetches list of leader IDs

jcli rest v0 leaders get <options>

The options are

YAML printed on success

---
- 1 # list of leader IDs
- 2

Register leader

Register new leader and get its ID

jcli rest v0 leaders post <options>

The options are

  • -h <node_addr> - see conventions
  • --debug - see conventions
  • --output-format <format> - see conventions -f, --file <file> - File containing YAML with leader secret. It must have the same format as secret YAML passed to Jormungandr as --secret. If not provided, YAML will be read from stdin.

On success created leader ID is printed

3

Delete leader

Delete leader with given ID

jcli rest v0 leaders delete <id> <options>
  • <id> - ID of deleted leader

The options are

Get leadership logs

Fetches leadership logs

jcli rest v0 leaders logs get <options>

The options are

YAML printed on success

---
- created_at_time: "2019-08-19T12:25:00.417263555+00:00"
  enclave_leader_id: 1
  finished_at_time: "2019-08-19T23:19:05.010113333+00:00"
  scheduled_at_date: "0.3923"
  scheduled_at_time: "2019-08-19T23:18:35+00:00"
  wake_at_time: "2019-08-19T23:18:35.001254555+00:00"
  status:
    Block:
      chain_length: 201018
      block: d9040ca57e513a36ecd3bb54207dfcd10682200929cad6ada46b521417964174
      parent: cc72d4ca957b03d7c795596b7fd7b1ff09c649c3e2877c508c0466abc8604832

Different value for the status:

# meaning the action is still pending to happen
status: Pending

# meaning the action successfully create the given block with the given hash and parent
status:
  Block:
    chain_length: 201018
    block: d9040ca57e513a36ecd3bb54207dfcd10682200929cad6ada46b521417964174
    parent: cc72d4ca957b03d7c795596b7fd7b1ff09c649c3e2877c508c0466abc8604832

# meaning the event has failed for some reasons
status:
  Rejected:
    reason: "Missed the deadline to compute the schedule"

Get stake pools

Fetches list of stake pool IDs

jcli rest v0 stake-pools get <options>

The options are

YAML printed on success

---
- 5cf03f333f37eb7b987dbc9017b8a928287a3d77d086cd93cd9ad05bcba7e60f # list of stake pool IDs
- 3815602c096fcbb91072f419c296c3dfe1f730e0f446a9bd2553145688e75615

Get stake distribution

Fetches stake information

jcli rest v0 stake get <options> [<epoch>]
  • <epoch> - Epoch to get the stake distribution from. (optional)

The options are

YAML printed on success

  • jcli rest v0 stake get <options> - stake distribution from the current epoch
---
epoch: 228      # Epoch of last block
stake:
  dangling: 0 # Total value stored in accounts, but assigned to nonexistent pools
  pools:
    - - 5cf03f333f37eb7b987dbc9017b8a928287a3d77d086cd93cd9ad05bcba7e60f # stake pool ID
      - 1000000000000                                                    # staked value
    - - 3815602c096fcbb91072f419c296c3dfe1f730e0f446a9bd2553145688e75615 # stake pool ID
      - 1000000000000                                                    # staked value
  unassigned: 0 # Total value stored in accounts, but not assigned to any pool
  • jcli rest v0 stake get <options> 10 - stake distribution from a specific epoch (epoch 10 in this example)
---
epoch: 10      # Epoch specified in the request
stake:
  dangling: 0 # Total value stored in accounts, but assigned to nonexistent pools
  pools:
    - - 5cf03f333f37eb7b987dbc9017b8a928287a3d77d086cd93cd9ad05bcba7e60f # stake pool ID
      - 1000000000000                                                    # staked value
    - - 3815602c096fcbb91072f419c296c3dfe1f730e0f446a9bd2553145688e75615 # stake pool ID
      - 1000000000000                                                    # staked value
  unassigned: 0 # Total value stored in accounts, but not assigned to any pool

Network stats

Fetches network stats

jcli rest v0 network stats get <options>

The options are

YAML printed on success

---
- # node address (optional)
  addr: "3.124.55.91:3000"
  # hex-encoded node ID
  nodeId: 0102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f20
  # timestamp of when the connection was established
  establishedAt: "2019-10-14T06:24:12.010231281+00:00"
  # timestamp of last time block was received from node if ever (optional)
  lastBlockReceived: "2019-10-14T00:45:57.419496113+00:00"
  # timestamp of last time fragment was received from node if ever (optional)
  lastFragmentReceived: "2019-10-14T00:45:58.419496150+00:00"
  # timestamp of last time gossip was received from node if ever (optional)
  lastGossipReceived: "2019-10-14T00:45:59.419496188+00:00"

Get stake pool details

Fetches stake pool details

jcli rest v0 stake-pool get <pool-id> <options>
  • <pool-id> - hex-encoded pool ID

The options are

YAML printed on success

---
tax:                        # pool reward
  fixed: 5                  # what get subtracted as fixed value
  ratio:                    # ratio of tax after fixed amount is subtracted. Expressed as numerator/denominator
    numerator: 1
    denominator: 10000
  max: 100                  # limit of tax (optional)
total_stake: 2000000000000  # total stake pool value
# bech32-encoded stake pool KES key
kesPublicKey: kes25519-12-pk1q7susucqwje0lpetqzjgzncgcrjzx7e2guh900qszdjskkeyqpusf3p39r
# bech32-encoded stake pool VRF key
vrfPublicKey: vrf_pk1rcm4qm3q9dtwq22x9a4avnan7a3k987zvepuxwekzj3uyu6a8v0s6sdy0l

Get rewards history for a specific epoch

Get the rewards history of a given epoch.

jcli rest v0 rewards epoch get <epoch> <options>
  • <epoch> - epoch number to get the rewards history for.

The options are

jcli rest v0 rewards epoch get 82 -h <node_addr>

[
  {
    "epoch": 82,              // the epoch number to collect rewards info from (rewards are from epoch 81)
    "drawn": 3835616440000,   // Total Drawn from reward escrow pot for the epoch
    "fees": 1828810000,       // Fees contributed into the pot the epoch
    "treasury": 462179124139, // Value added to the treasury
    "stake_pools": {
      "0087011b9c626759f19d9d0315a9b42492ba497438c12efc026d664c9f324ecb": [
        1683091391, // pool's owned rewards from taxes
        32665712521 // distributed rewards to delegators
      ],
      "014bb0d84f40900f6dd85835395bc38da3ab81435d1e6ee27d419d6eeaf7d16a": [
        47706672,
        906426770
      ],
    },
    "accounts": {
      "ed25519_pk1qqq6r7r7medu2kdpvdra5kwh8uz9frvftm9lf25shm7ygx9ayvss0nqke9": 427549785, // Amount added to each account
      "ed25519_pk1qqymlwehsztpzhy2k4szkp7j0xk0ra35jyxcpgr9p9q4ngvzzc5q4sh2gm": 24399360,
      "ed25519_pk1qq9h62jv6a0mz36xgecjrz9tm8z6ay3vj4d64ashxkgxcyhjewwsvgvelj": 22449169,
      "ed25519_pk1qq9l2qrqazk5fp4kt2kvjtsjc32g0ud888um8k2pvms0cw2r0uzsute83u": 1787992,
      "ed25519_pk1qqx6h559ee7pa67dm255d0meekt6dmq6857x302wdwrhzv47z9hqucdnt2": 369024,
    }
  }
]

Get rewards history for some epochs

Get the rewards history of the length last epoch(s) from tip.

jcli rest v0 rewards history get <length> <options>
  • <length> - number of epochs, starting from the last epoch from tip, to get the reward history for.

The options are

jcli rest v0 rewards history get 2 -h <node_addr>

[
  {
    "epoch": 93,
    "drawn": 3835616440000,
    "fees": 641300000,
    "treasury": 467151470296,
    "stake_pools": {
      "0087011b9c626759f19d9d0315a9b42492ba497438c12efc026d664c9f324ecb": [
        1121750881,
        21771124247
      ],
      "014bb0d84f40900f6dd85835395bc38da3ab81435d1e6ee27d419d6eeaf7d16a": [
        429241408,
        8155586765
      ],
      "01bd272cede02d0b0c9cd47b16e5356ab3fb2330dd9d1e972ab5494365309d2a": [
        1691506850,
        32829041110
      ],
    },
    "accounts": {
      "ed25519_pk1002kje4l8j7kvsseyauusk3s7nzef4wcvvafltjmg0rkzr6qccyqg064kz": 33311805,
      "ed25519_pk100549kxqn8tnzfzr5ndu0wx7pp2y2ck28mnykq03m2z5qcwkvazqx9fp0h": 15809,
      "ed25519_pk10054y058qfn5wnazalnkax0mthg06ucq87nn9320rphtye5ca0xszjcelk": 10007789,
      "ed25519_pk10069dsunppwttl4qtsfnyhjnqwkunuwxjxlandl2fnpwpuznf5pqmg3twe": 545094806,
      "ed25519_pk1009sfpljfgx30z70l3n63gj7w9vp3epugmd3vn62fyr07ut9pfwqjp7f8h": 4208232,
    },
  },
  {
    "epoch": 92,
    "drawn": 3835616440000,
    "fees": 620400000,
    "treasury": 480849578351,
    "stake_pools": {
      "0087011b9c626759f19d9d0315a9b42492ba497438c12efc026d664c9f324ecb": [
        979164601,
        19003786459
      ],
      "0105449dd66524111349ef677d1ebc25247a5ba2d094913f52aa4db265eac03a": [
        26977274,
        972170279
      ],
      "014bb0d84f40900f6dd85835395bc38da3ab81435d1e6ee27d419d6eeaf7d16a": [
        299744265,
        5695141053
      ],
    },
    "accounts": {
      "ed25519_pk1002kje4l8j7kvsseyauusk3s7nzef4wcvvafltjmg0rkzr6qccyqg064kz": 40581616,
      "ed25519_pk100549kxqn8tnzfzr5ndu0wx7pp2y2ck28mnykq03m2z5qcwkvazqx9fp0h": 49156,
      "ed25519_pk10054y058qfn5wnazalnkax0mthg06ucq87nn9320rphtye5ca0xszjcelk": 12306084,
      "ed25519_pk10069dsunppwttl4qtsfnyhjnqwkunuwxjxlandl2fnpwpuznf5pqmg3twe": 142737175,
      "ed25519_pk1009sfpljfgx30z70l3n63gj7w9vp3epugmd3vn62fyr07ut9pfwqjp7f8h": 3932910,
    },
  }
]

Get voting committee members

Get the list of voting committee members.

jcli rest v0 vote active committees get <options>

The options are

YAML printed on success

---
- 7ef044ba437057d6d944ace679b7f811335639a689064cd969dffc8b55a7cc19 # list of members
- f5285eeead8b5885a1420800de14b0d1960db1a990a6c2f7b517125bedc000db

Get active voting plans and proposals

Get the list of active voting plans and proposals.

jcli rest v0 vote active plans get <options>

The options are

YAML printed on success

---
- committee_end:
    epoch: 10
    slot_id: 0
  proposals:
    - external_id: adb92757155d09e7f92c9f100866a92dddd35abd2a789a44ae19ab9a1dbc3280
      options:
        OneOf:
          max_value: 3
    - external_id: 6778d37161c3962fe62c9fa8a31a55bccf6ec2d1ea254a467d8cd994709fc404
      options:
        OneOf:
          max_value: 3
  vote_end:
    epoch: 5
    slot_id: 0
  vote_start:
    epoch: 1
    slot_id: 0

Staking with Jörmungandr

Here we will describe how to:

  • delegate your stake to a stake pool - so that you can participate to the consensus and maybe collect rewards for that.
  • register a stake pool
  • retire a stake pool

Delegating your stake

how to create the delegation certificate

Stake is concentrated in accounts, and you will need account public key to delegate its associated stake.

for own account

You will need:

  • the Stake Pool ID: an hexadecimal string identifying the stake pool you want to delegate your stake to.
jcli certificate new owner-stake-delegation STAKE_POOL_ID --output stake_delegation.cert

Note that the certificate is in blaco, there's no account key used for its creation. In order for delegation to work it must be submitted to a node inside a very specific transaction:

  • Transaction must have exactly 1 input
  • The input must be from account
  • The input value must be strictly equal to fee of the transaction
  • Transaction must have 0 outputs

The account used for input will have its stake delegated to the stake pool

for any account

You will need:

  • account public key: a bech32 string of a public key
  • the Stake Pool ID: an hexadecimal string identifying the stake pool you want to delegate your stake to.
jcli certificate new stake-delegation ACCOUNT_PUBLIC_KEY STAKE_POOL_ID --output stake_delegation.cert

submitting to a node

The jcli transaction add-certificate command should be used to add a certificate before finalizing the transaction.

For example:


...

jcli transaction add-certificate $(cat stake_delegation.cert) --staging tx
jcli transaction finalize CHANGE_ADDRESS --fee-constant 5 --fee-coefficient 2 --fee-certificate 2 --staging tx

...
jcli transaction seal --staging tx
jcli transaction auth --key account_key.prv --staging tx
...

The --fee-certificate flag indicates the cost of adding a certificate, used for computing the fees, it can be omitted if it is zero.

See here for more documentation on transaction creation.

how to sign your delegation certificate

This procedure is needed only for certificates that are to be included in the genesis config file.

We need to make sure that the owner of the account is authorizing this delegation to happens, and for that we need a cryptographic signature.

We will need the account secret key to create a signature

jcli certificate sign --certificate stake_delegation.cert --key account_key.prv --output stake_delegation.signedcert

The content of stake_delegation.signedcert will be something like:

signedcert1q9uxkxptz3zx7akmugkmt4ecjjd3nmzween2qfr5enhzkt37tdt4uqt0j0039z5048mu9ayv3ujep5sl28q2cpdnx9fkvpq30lmjrrgtmqqctzczvu6e3v65m40n40c3y2pnu4vhd888dygkrtnfm0ts92fe50jy0h0ugh6wlvgy4xvr3lz4uuqzg2xgu6vv8tr24jrwhg0l09klp5wvwzl5

and can now be added in the genesis config file.

Registering a stake pool

There are multiple components to be aware of when running a stake pool:

  • your NodeId: it is the identifier within the blockchain protocol (wallet will delegate to your stake pool via this NodeId);
  • your VRF key pairs: this is the cryptographic material we will use to participate to the leader election;
  • your KES key pairs: this is the cryptographic material we will use to sign the block with.
  • the stake pool Tax: the value the stake pool will take from the total reward due to the stake pool before distributing rewards (if any left) to the delegators.

So in order to start your stake pool you will need to generate these objects.

The primitives

VRF key pair

To generate your VRF Key pairs, we will utilise jcli as described here:

jcli key generate --type=RistrettoGroup2HashDhH stake_pool_vrf.prv

stake_pool_vrf.prv file now contains the VRF private key.

jcli key to-public --input stake_pool_vrf.prv stake_pool_vrf.pub

stake_pool_vrf.pub file now contains the VRF public key.

KES key pair

Similar to above:

jcli key generate --type=SumEd25519_12 stake_pool_kes.prv

stake_pool_kes.prv file now contains the KES private key

jcli key to-public --input stake_pool_kes.prv stake_pool_kes.pub

stake_pool_kes.pub file now contains the KES public key

Choosing the Tax parameters

There are 3 values you can set to configure the stake pool's Tax:

  • tax-fixed: this is the fixed cut the stake pool will take from the total reward due to the stake pool;
  • tax-ratio: this is the percentage of the remaining value that will be taken from the total due
  • tax-limit: a value that can be set to limit the pool's Tax.

All of these values are optionals, if not set, they will be set to 0. This will mean no tax for the stake pool: rewards are all distributed to the delegators.

So how does this works

Let say you control a stake pool SP, with 2 owners (O1 and O2). During epoch 1, SP has created some blocks and is entitled to receive 10_000.

Before distributing the 10_000 among the delegators, SP will take its Tax.

  1. we extract the tax-fixed. If this is greater or equal to the total (10_000) then we stop there, there is no more rewards to distribute.
  2. with what remains the SP extracts its tax-ratio and checks the tax from the ratio is not greater than tax-limit.
  3. the total SP rewards will then be distributed equally to the owners (O1 and O2). Note that if the --reward-account is set, the rewards for SP are then distributed to that account and nothing to O1 and O2.

For example:

totalfixedratiolimitSPO1O2for delegators
takes 100%1000001/1010000500050000
fixed of 10001000010000/1010005005009000
fixed + 10%200010001/1001100550550900
fixed + 20% up to 150200010001/51501150575575850

The options to set

--tax-limit <TAX_LIMIT>
    The maximum tax value the stake pool will take.

    This will set the maximum the stake pool value will reserve for themselves from the `--tax-ratio` (excluding `--tax-fixed`).
--tax-ratio <TAX_RATIO>
    The percentage take of the stake pool.

    Once the `tax-fixed` has been take, this is the percentage the stake pool will take for themselves. [default: 0/1]
--tax-fixed <TAX_VALUE>
    set the fixed value tax the stake pool will reserve from the reward

    For example, a stake pool may set this value to cover their fixed operation costs. [default: 0]

creating a stake pool certificate

The certificate is what will be sent to the blockchain in order to register yourself to the other participants of the blockchain that you are a stake pool too.

jcli certificate new stake-pool-registration \
    --kes-key $(cat stake_pool_kes.pub) \
    --vrf-key $(cat stake_pool_vrf.pub) \
    --start-validity 0 \
    --management-threshold 1 \
    --tax-fixed 1000000 \
    --tax-limit 1000000000 \
    --tax-ratio "1/10" \
    --owner $(cat owner_key.pub) > stake_pool.cert

The --operator flag is optional.

And now you can retrieve your stake pool id (NodeId):

jcli certificate get-stake-pool-id stake_pool.cert
ea830e5d9647af89a5e9a4d4089e6e855891a533316adf4a42b7bf1372389b74

submitting to a node

The jcli transaction add-certificate command should be used to add a certificate before finalizing the transaction.

For example:

...

jcli transaction add-certificate $(cat stake_pool.cert) --staging tx
jcli transaction finalize CHANGE_ADDRESS --fee-constant 5 --fee-coefficient 2 --fee-certificate 2 --staging tx

...
jcli transaction seal --staging tx
jcli transaction auth --key owner_key.prv --staging tx
...

The --fee-certificate flag indicates the cost of adding a certificate, used for computing the fees, it can be omitted if it is zero.

See here for more documentation on transaction creation.

Retiring a stake pool

Stake pool can be retired by sending transaction with retirement certificate. From technical stand point, it is very similar to register stake pool operation. Before start we need to be sure, that:

  • you have sufficient amount of ada to pay fee for transaction with retirement certificate.
  • you know your stake pool id.

Retrieve stake pool id

To retrieve your stake pool id:

jcli certificate get-stake-pool-id stake_pool.cert
ea830e5d9647af89a5e9a4d4089e6e855891a533316adf4a42b7bf1372389b74

creating a retirement certificate

The certificate is what will be sent to the blockchain in order to retire your stake pool.

jcli certificate new stake-pool-retirement \
    --pool-id ea830e5d9647af89a5e9a4d4089e6e855891a533316adf4a42b7bf1372389b74 \
    --retirement-time 0 \
    retirement.cert

where:

  • retirement.cert - write the output of to the retirement.cert
  • --retirement-time 0 - 0 means as soon as possible. Which is until the next following epoch.
  • --pool-id ea830e5d9647af89a5e9a4d4089e6e855891a533316adf4a42b7bf1372389b74 - hex-encoded stake pool ID.

submitting to a node

The jcli transaction add-certificate command should be used to add a certificate before finalizing the transaction.

For example:

...

jcli transaction add-certificate $(cat retirement.cert) --staging tx
jcli transaction finalize CHANGE_ADDRESS --fee-constant 5 --fee-coefficient 2 --fee-certificate 2 --staging tx

...
jcli transaction seal --staging tx
jcli transaction auth --key owner_key.prv --staging tx
...

The --fee-certificate flag indicates the cost of adding a certificate, used for computing the fees, it can be omitted if it is zero.

Important ! Please be sure that you have sufficient amount of owners signatures in order to retire stake pool. At least half of owners singatures (which were provided when registering stake pool) are required to sign retirement certificate.

See here for more documentation on transaction creation.

Advanced

This section is meant for advanced users and developers of the node, or if you wish to learn more about the node.

At the moment, it only covers details on how to create your own blockchain genesis configuration, but in normal case, the blockchain configuration should be available with the specific blockchain system.

genesis file

The genesis file is the file that allows you to create a new blockchain from block 0. It lays out the different parameters of your blockchain: the initial utxo, the start time, the slot duration time, etc...

Example of a BFT genesis file with an initial address UTxO and an account UTxO. More info regarding starting a BFT blockchain here and regarding addresses there. You could also find information regarding the jcli genesis tooling.

You can generate a documented pre-generated genesis file:

jcli genesis init

For example your genesis file may look like:

# The Blockchain Configuration defines the settings of the blockchain.
blockchain_configuration:

  # The block0-date defines the date the blockchain starts
  # expected value in seconds since UNIX_EPOCH
  #
  # By default the value will be the current date and time. Or you can
  # add a specific time by entering the number of seconds since UNIX
  # Epoch
  block0_date: {default_block0_date}

  # This is the type of discrimination of the blockchain
  # if this blockchain is meant for production then
  # use 'production' instead.
  #
  # otherwise leave as this
  discrimination: {discrimination}

  # The initial consensus version:
  #
  # * BFT consensus: bft
  # * Genesis Praos consensus: genesis
  block0_consensus: bft

  # Number of slots in each epoch.
  #
  # default value is {default_slots_per_epoch}
  slots_per_epoch: {default_slots_per_epoch}

  # The slot duration, in seconds, is the time between the creation
  # of 2 blocks
  #
  # default value is {default_slot_duration}
  slot_duration: {default_slot_duration}

  # set the block content max size
  #
  # This is the size, in bytes, of all the contents of the block (excluding the
  # block header).
  #
  # default value is {default_block_content_max_size}
  block_content_max_size: {default_block_content_max_size}

  # A list of Ed25519 PublicKey that represents the
  # BFT leaders encoded as bech32. The order in the list matters.
  consensus_leader_ids:
    - {leader_1}
    - {leader_2}

  # Epoch stability depth
  #
  # Optional: default value {default_epoch_stability_depth}
  epoch_stability_depth: {default_epoch_stability_depth}

  # Genesis praos active slot coefficient
  # Determines minimum stake required to try becoming slot leader, must be in range (0,1]
  #
  # default value: {default_consensus_genesis_praos_active_slot_coeff}
  consensus_genesis_praos_active_slot_coeff: {default_consensus_genesis_praos_active_slot_coeff}

  # The fee calculations settings
  #
  # total fees: constant + (num_inputs + num_outputs) * coefficient [+ certificate]
  linear_fees:
    # this is the minimum value to pay for every transaction
    constant: 2
    # the additional fee to pay for every inputs and outputs
    coefficient: 1
    # the additional fee to pay if the transaction embeds a certificate
    certificate: 4
    # (optional) fees for different types of certificates, to override the one
    # given in `certificate` just above
    #
    # here: all certificate fees are set to `4` except for pool registration
    # and stake delegation which are respectively `5` and `2`.
    per_certificate_fees:
      # (optional) if not specified, the pool registration certificate fee will be
      # the one set by linear_fees.certificate
      certificate_pool_registration: 5
      # (optional) if not specified, the delegation certificate fee will be
      # the one set by linear_fees.certificate
      certificate_stake_delegation: 2
      # (optional) if not specified, the owner delegation certificate fee will be
      # the one set by linear_fees.certificate. Uncomment to set the owner stake
      # delegation to `1` instead of default `4`:
      # certificate_owner_stake_delegation: 1

  # Proposal expiration in epochs
  #
  # default value: {default_proposal_expiration}
  proposal_expiration: {default_proposal_expiration}

  # The speed to update the KES Key in seconds
  #
  # default value: {default_kes_update_speed}
  kes_update_speed: {default_kes_update_speed}

  # Set where to send the fees generated by transactions activity.
  #
  # by default it is send to the "rewards" pot of the epoch which is then
  # distributed to the different stake pools who created blocks that given
  # epoch.
  #
  # It is possible to send all the generated fees to the "treasury".
  #
  # Optional, default is "rewards"
  # fees_go_to: "rewards"

  # initial value the treasury will start with, if not set the treasury
  # starts at 0
  treasury: 1000000000000

  # set the treasury parameters, this is the tax type, just as in stake pool
  # registration certificate parameters.
  #
  # When distributing the rewards, the treasury will be first serve as per
  # the incentive specification document
  #
  # if not set, the treasury will not grow
  treasury_parameters:
    # the fix value the treasury will take from the total reward pot of the epoch
    fixed: 1000
    # the extra percentage the the treasury will take from the reward pot of the epoch
    ratio: "1/10"
    # It is possible to add a max bound to the total value the treasury takes
    # at each reward distribution. For example, one could cap the treasury tax
    # to 10000. Uncomment the following line to apply a max limit:
    # max_limit: 10000

  # Set the total reward supply available for monetary creation
  #
  # if not set there is no monetary creation
  # once emptied, there is no more monetary creation
  total_reward_supply: 100000000000000

  # set the reward supply consumption. These parameters will define how the
  # total_reward_supply is consumed for the stake pool reward
  #
  # There's fundamentally many potential choices for how rewards are contributed back, and here's two potential valid examples:
  #
  # Linear formula: constant - ratio * (#epoch after epoch_start / epoch_rate)
  # Halving formula: constant * ratio ^ (#epoch after epoch_start / epoch_rate)
  #
  reward_parameters:
    halving: # or use "linear" for the linear formula
      # In the linear formula, it represents the starting point of the contribution
      # at #epoch=0, whereas in halving formula is used as starting constant for
      # the calculation.
      constant: 100

      # In the halving formula, an effective value between 0.0 to 1.0 indicates a
      # reducing contribution, whereas above 1.0 it indicate an acceleration of contribution.
      #
      # However in linear formula the meaning is just a scaling factor for the epoch zone
      # (current_epoch - start_epoch / epoch_rate). Further requirement is that this ratio
      # is expressed in fractional form (e.g. 1/2), which allow calculation in integer form.
      ratio: "13/19"

      # indicates when this contribution start. note that if the epoch is not
      # the same or after the epoch_start, the overall contribution is zero.
      epoch_start: 1

      # the rate at which the contribution is tweaked related to epoch.
      epoch_rate: 3

  # set some reward constraints and limits
  #
  # this value is optional, the default is no constraints at all. The settings
  # are commented below:
  #
  #reward_constraints:
  #  # limit the epoch total reward drawing limit to a portion of the total
  #  # active stake of the system.
  #  #
  #  # for example, if set to 10%, the reward drawn will be bounded by the
  #  # 10% of the total active stake.
  #  #
  #  # this value is optional, the default is no reward drawing limit
  #  reward_drawing_limit_max: "10/100"
  #
  #  # settings to incentivize the numbers of stake pool to be registered
  #  # on the blockchain.
  #  #
  #  # These settings does not prevent more stake pool to be added. For example
  #  # if there is already 1000 stake pools, someone can still register a new
  #  # stake pool and affect the rewards of everyone else too.
  #  #
  #  # if the threshold is reached, the pool doesn't really have incentive to
  #  # create more blocks than 1 / set-value-of-pools % of stake.
  #  #
  #  # this value is optional, the default is no pool participation capping
  #  pool_participation_capping:
  #    min: 300
  #    max: 1000

  # list of the committee members, they will be used to guarantee the initial
  # valid operation of the vote as well as privacy.
  committees:
    - "7ef044ba437057d6d944ace679b7f811335639a689064cd969dffc8b55a7cc19"
    - "f5285eeead8b5885a1420800de14b0d1960db1a990a6c2f7b517125bedc000db"

# Initial state of the ledger. Each item is applied in order of this list
initial:
  # Initial deposits present in the blockchain
  - fund:
      # UTxO addresses or account
      - address: {initial_funds_address_1}
        value: 10000
      - address: {initial_funds_address_2}
        value: 10000
  # Initial token distribution
  - token:
      token_id: 00000000000000000000000000000000000000000000000000000000.7e5d6abc
      to:
        - address: {initial_funds_address_1}
          value: 150
        - address: {initial_funds_address_2}
          value: 255
  - token:
      token_id: 00000000000000000000000000000000000000000000000000000000.6c1e8abc
      to:
        - address: {initial_funds_address_1}
          value: 22
        - address: {initial_funds_address_2}
          value: 66

  # Initial certificates
  #- cert: ..

  # Initial deposits present in the blockchain
  #- legacy_fund:
  #    # Legacy Cardano address
  #    - address: 48mDfYyQn21iyEPzCfkATEHTwZBcZJqXhRJezmswfvc6Ne89u1axXsiazmgd7SwT8VbafbVnCvyXhBSMhSkPiCezMkqHC4dmxRahRC86SknFu6JF6hwSg8
  #      value: 123

There are multiple parts in the genesis file:

  • blockchain_configuration: this is a list of configuration parameters of the blockchain, some of which can be changed later via the update protocol;
  • initial: list of steps to create initial state of ledger

blockchain_configuration options

optionformatdescription
block0_datenumberthe official start time of the blockchain, in seconds since UNIX EPOCH
discriminationstringproduction or test
block0_consensusstringbft
slot_durationnumberthe number of seconds between the creation of 2 blocks
epoch_stability_depthnumberallowed size of a fork (in number of block)
consensus_leader_idsarraythe list of the BFT leader at the beginning of the blockchain
block_content_max_sizenumberthe maximum size of the block content (excluding the block header), in bytes.
linear_feesobjectlinear fee settings, set the fee for transaction and certificate publishing
consensus_genesis_praos_active_slot_coeffnumbergenesis praos active slot coefficient. Determines minimum stake required to try becoming slot leader, must be in range (0,1]
kes_update_speednumberthe speed to update the KES Key in seconds
slots_per_epochnumbernumber of slots in each epoch

for more information about the BFT leaders in the genesis file, see Starting a BFT Blockchain

initial options

Each entry can be one of 3 variants:

variantformatdescription
fundsequenceinitial deposits present in the blockchain (up to 255 outputs per entry)
certstringinitial certificate
legacy_fundsequencesame as fund, but with legacy Cardano address format

Example:

initial:
  - fund:
      - address: <address>
        value: 10000
      - address: <address2>
        value: 20000
      - address: <address3>
        value: 30000
  - cert: <certificate>
  - legacy_fund:
      - address: <legacy address>
        value: 123
  - fund:
      - address: <another address>
        value: 1001

fund and legacy_fund format

variantformatdescription
addressstringcan be a single address or an account address
valuenumberassigned value

legacy_fund differs only in address format, which is legacy Cardano

starting a bft node

BFT stands for the Byzantine Fault Tolerant (read the paper).

Jormungandr allows you to start a BFT blockchain fairly easily. The main downside is that it is centralized, only a handful of nodes will ever have the right to create blocks.

How does it work

It is fairly simple. A given number of Nodes (N) will generate a key pairs of type Ed25519 (see JCLI's Keys).

They all share the public key and add them in the genesis.yaml file. It is the source of truth, the file that will generate the first block of the blockchain: the Block 0.

Then, only by one after the other, each Node will be allowed to create a block. Utilising a Round Robin algorithm.

Example of genesis file

blockchain_configuration:
  block0_date: 1550822014
  discrimination: test
  block0_consensus: bft
  slots_per_epoch: 5
  slot_duration: 15
  epoch_stability_depth: 10
  consensus_leader_ids:
    - ed25519e_pk1k3wjgdcdcn23k6dwr0cyh88ad7a4ayenyxaherfazwy363pyy8wqppn7j3
    - ed25519e_pk13talprd9grgaqzs42mkm0x2xek5wf9mdf0eefdy8a6dk5grka2gstrp3en
  consensus_genesis_praos_active_slot_coeff: 0.22
  linear_fees:
    constant: 2
    coefficient: 1
    certificate: 4
  kes_update_speed: 43200
initial:
  - fund:
      - address: ta1svy0mwwm7mdwcuj308aapjw6ra4c3e6cygd0f333nvtjzxg8ahdvxlswdf0
        value: 10000
  - cert: cert1qgqqqqqqqqqqqqqqqqqqq0p5avfqqmgurpe7s9k7933q0wj420jl5xqvx8lywcu5jcr7fwqa9qmdn93q4nm7c4fsay3mzeqgq3c0slnut9kns08yn2qn80famup7nvgtfuyszqzqrd4lxlt5ylplfu76p8f6ks0ggprzatp2c8rn6ev3hn9dgr38tzful4h0udlwa0536vyrrug7af9ujmrr869afs0yw9gj5x7z24l8sps3zzcmv
  - legacy_fund:
      - address: 48mDfYyQn21iyEPzCfkATEHTwZBcZJqXhRJezmswfvc6Ne89u1axXsiazmgd7SwT8VbafbVnCvyXhBSMhSkPiCezMkqHC4dmxRahRC86SknFu6JF6hwSg8
        value: 123

In order to start your blockchain in BFT mode you need to be sure that:

  • consensus_leader_ids is non empty;

more information regarding the genesis file here.

Creating the block 0

jcli genesis encode --input genesis.yaml --output block-0.bin

This command will create (or replace) the Block 0 of the blockchain from the given genesis configuration file (genesis.yaml).

Starting the node

Now that the blockchain is initialized, you need to start your node.

Write your private key in a file on your HD:

$ cat node_secret.yaml
bft:
  signing_key: ed25519_sk1hpvne...

Configure your Node (config.yml) and run the following command:

$ jormungandr --genesis-block block-0.bin \
    --config example.config \
    --secret node_secret.yaml

It's possible to use the flag --secret multiple times to run a node with multiple leaders.

Step by step to start the BFT node

  1. Generate initial config jcli genesis init > genesis.yaml
  2. Generate secret key, e.g. jcli key generate --type=Ed25519 > key.prv
  3. Put secret key in a file, e.g. node_secret.yaml as follows:
bft:
 signing_key: ed25519_sk1kppercsk06k03yk4qgea....
  1. Generate public key out of previously generated key cat key.prv | jcli key to-public
  2. Put generated public key as in genesis.yaml under consensus_leader_ids:
  3. Generate block = jcli genesis encode --input genesis.yaml --output block-0.bin
  4. Create config file and store it on your HD as node.config e.g. ->
---
log:
  level: trace
  format: json
rest:
  listen: "127.0.0.1:8607"
p2p:
  public_address: /ip4/127.0.0.1/tcp/8606
  topics_of_interest:
    messages: low
    blocks: normal
  1. Start Jörmungandr node :
jormungandr --genesis-block block-0.bin --config node.config --secret node_secret.yaml

Script

Additionally, there is a script here that can be used to bootstrap a test node with bft consensus protocol.

starting a genesis blockchain

When starting a genesis praos blockchain there is an element to take into consideration while constructing the block 0: the stake distribution.

In the context of Genesis/Praos the network is fully decentralized and it is necessary to think ahead about initial stake pools and to make sure there is stake delegated to these stake pools.

In your genesis yaml file, make sure to set the following values to the appropriate values/desired values:

# The Blockchain Configuration defines the settings of the blockchain.
blockchain_configuration:
  block0_consensus: genesis_praos
  consensus_genesis_praos_active_slot_coeff: 0.1
  kes_update_speed: 43200 # 12hours

block0_consensus set to genesis_praos means you want to start a blockchain with genesis praos as the consensus layer.

consensus_genesis_praos_active_slot_coeff determines minimum stake required to try becoming slot leader, must be in range 0 exclusive and 1 inclusive.

The initial certificates

In the initial_certs field you will set the initial certificate. It is important to declare the stake pool and delegate stake to them. Otherwise no block will ever be created.

Remember that in this array the order matters:

In order to delegate your stake, you need a stake pool to already exist, so the stake pool registration certificate should go first.

Stake pool registration

Now you can register a stake pool. Follow the instructions in registering stake pool guide.

The owner key (the key you sign the stake pool registration certificate) is the secret key associated to a previously registered stake key.

Delegating stake

Now that there is both your stake key and there are stake pools available in the block0 you need to delegate to one of the stake pool. Follow the instruction in delegating stake.

And in the initial funds start adding the addresses. To create an address with delegation follow the instruction in JCLI's address guide. Utilise the stake key registered previously as group address:

jcli address single $(cat wallet_key.pub) $(cat stake_key.pub)
ta1sjx4j3jwel94g0cgwzq9au7h6m8f5q3qnyh0gfnryl3xan6qnmjse3k2uv062mzj34eacjnxthxqv8fvdcn6f4xhxwa7ms729ak3gsl4qrq2mm

You will notice that addresses with delegation are longer (about twice longer) than addresses without delegation.

For example, the most minimal setting you may have is:

initial_certs:
  # register a stake pool (P), owner of the stake pool is the stake key (K)
  - cert1qsqqqqqqqqqqqqqqqqqqq0p5avfqp9tzusr26chayeddkkmdlap6tl23ceca8unsghc22tap8clhrzslkehdycufa4ywvqvs4u36zctw4ydtg7xagprfgz0vuujh3lgtxgfszqzqj4xk4sxxyg392p5nqz8s7ev5wna7eqz7ycsuas05mrupmdsfk0fqqudanew6c0nckf5tsp0lgnk8e8j0dpnxvjk2usn52vs8umr3qrccegxaz

  # delegate stake associated to stake key (K) to stake pool (P)
  - cert1q0rv4ccl54k99rtnm39xvhwvqcwjcm385n2dwvamahpu5tmdz3plt65rpewev3a03xj7nfx5pz0xap2cjxjnxvt2ma9y9dalzder3xm5qyqyq0lx05ggrws0ghuffqrg7scqzdsd665v4m7087eam5zvw4f26v2tsea3ujrxly243sgqkn42uttk5juvq78ajvfx9ttcmj05lfuwtq9qhdxzr0

initial_funds:
  # address without delegation
  - address: ta1swx4j3jwel94g0cgwzq9au7h6m8f5q3qnyh0gfnryl3xan6qnmjsczt057x
    value: 10000
  # address delegating to stake key (K)
  - address: ta1sjx4j3jwel94g0cgwzq9au7h6m8f5q3qnyh0gfnryl3xan6qnmjse3k2uv062mzj34eacjnxthxqv8fvdcn6f4xhxwa7ms729ak3gsl4qrq2mm
    value: 1000000

Starting the node

Now, to start the node and be able to generate new blocks, you have to put your pool's private keys and id in a file and start the node with the --secret filename parameter.

For example, if you follow the examples of the registering stake pool guide

You could create a file called poolsecret.yaml with the following content.

genesis:
  sig_key: Content of stake_pool_kes.prv file
  vrf_key: Content of stake_pool_vrf.prv file
  node_id: Content of stake_pool.id file

And you could start the node with this command

jormungandr --genesis-block block-0.bin --config config.yaml --secret poolsecret.yaml

Test script

There is a script here that can be used to bootstrap a test node with a pre-set faucet and stake pool and can be used as an example.

Jormungandr Specifications

This directory contains Jormungandr's specifications.

filecontent
networkthe node to node communication and the peer to peer topology

Network

Bringing Ouroboros to the people

Introduction

This document highlights the requirements we wish to apply to a decentralised network applied to cardano blockchain. Then we will discuss the possible solutions we can provide in a timely manner and the tradeoff we will need to make.

Design decisions guidelines

This is a main of general guidelines for the design decision in this document, and to judge the merit of solutions:

  • Efficiency: the communication between the nodes needs to be succinct. to the point, avoiding unnecessary redundancies. The protocol needs to stabilise quickly to a well distributed network, guaranteeing a fast propagation of the important events;
  • Security: limit the ability for other nodes to trigger behavior that would prevent a peer from working (e.g. unbounded resources usage)
  • Simplicity: we need to easily implement the protocol for any platforms or environment that will matter for our users.

Node-to-Node communication

This section describes the communication between 2 different peers on the network. It involves synchronous queries with the context of the local state and remote state.

General Functionality

This is a general high level list of what information will need to be exchanged:

  • Bootstrap local state from nothing
  • Update local state from an arbitrary point
  • Answer Synchronous queries: RPC style
  • Asynchronous messages for state propagation (transactions, blocks, ..)
  • P2P messages (See P2P Communication)

Design

User Stories

  • Alice wants to synchronise its local state from Bob from Alice's Tip:
    • Alice downloads Block Headers from Bob (starting from Alice's Tip);
      • Bob does not know this Tip:
        • Error: unknown block
        • Alice starts again with a previous Tip;
      • Bob does know this state:
        • Bob streams back the block headers
    • Alice downloads block
      • Since Alice knows the list of Block Headers and the number of blocks to download, Alice can download from multiple peers, requiring to get block stream from different Hash in this list of Block;
    • State: tip_hash, storage
    • Pseudocode (executed by Alice): 
      
#![allow(unused)]
fn main() {
sync():
  bob.get_headers(alice.tip)
}
  • Alice wants to propagate a transaction to Bob
    • Alice send the transaction hash to Bob
    • Bob replies whether it want to hear more
    • Alice send the transaction to Bob if Bob agrees
  • Alice wants to submit a Block to Bob
    • Alice sends the Header to Bob;
    • Bob replies whether it want to hear more
    • Alice sends the Block to Bob if Bob agrees
  • Alice want to exchange peers with Bob

High Level Messages

We model everything so that we don't need any network state machine. Everything is stateless for

  • Handshake: () -> (Version, Hash)
    • This should be the first request performed by the client after connecting. The server responds with the protocol version and the hash of the genesis block.
    • The handshake is used to establish that the remote node has a compatible protocol implementation and serves the right block chain.
  • Tip: () -> Header:
    • Return the header of the latest block known by the peer (also known as at the tip of the blockchain).
    • DD? : Block vs hash: block is large but contain extra useful metadata (slotid, prevhash), whereas hash is small.
  • GetHeaders: ([Hash]) -> [Header]:
    • Fetch the headers (cryptographically verifiable metadata summaries) of the blocks identified by hashes.
  • GetBlocks: ([Hash]) -> [Block]:
    • Like GetHeaders, but returns full blocks.
  • PullBlocksToTip: ([Hash]) -> Stream<Block>:
    • Retrieve a stream of blocks descending from one of the given hashes, up to the remote's current tip.
    • This is an easy way to pull blockchain state from a single peer, for clients that don't have a need to fiddle with batched GetBlocks requests and traffic distribution among multiple peers.
  • BlockSubscription: (Stream<Header>) -> Stream<BlockEvent>
    • Establish a bidirectional subscription to send and receive announcements of new blocks and (in the client role) receive solicitations to upload blocks or push the chain of headers.
    • The stream item is a tagged enumeration: BlockEvent: Announce(Header)|Solicit([Hash])|Missing([Hash], Hash)
      • Announce propagates header information of a newly minted block.
      • Solicit requests the client to upload blocks identified by the given hashes using the UploadBlocks request.
      • Missing requests the client to stream the chain of block headers using the given range parameters. The meaning of the parameters is the same as in the PullHeaders request.
    • The client does not need to stream solicitations upwards, as it can request blocks directly with GetBlocks or PullHeaders.
    • The announcements send in either direction are used for both announcing new blocks when minted by this node in the leadership role, and propagating blocks received from other nodes on the p2p network.
  • PullHeaders: ([Hash], Hash) -> Stream<Header>
    • Retrieve a stream of headers for blocks descending from one of the hashes given in the first parameter, up to the hash given in the second parameter. The starting point that is latest in the chain is selected.
    • The client sends this request after receiving an announcement of a new block via the BlockSubscription stream, when the parent of the new block is not present in its local storage. The proposed starting points are selected from locally known blocks with exponentially receding depth.
  • PushHeaders: (Stream<Header>)
    • Streams the chain of headers in response to a Missing event received via the BlockSubscription stream.
  • UploadBlocks: (Stream<Block>)
    • Uploads blocks in response to a Solicit event received via the BlockSubscription stream.
  • ContentSubscription: (Stream<Fragment>) -> Stream<Fragment>
    • Establish a bidirectional subscription to send and receive new content for the block under construction.
    • Used for submission of new fragments submitted to the node by application clients, and for relaying of fragment gossip on the network.
  • P2P Messages: see P2P messages section.

The protobuf files describing these methods are available in the proto directory of chain-network crate in the chain-libs project repository.

Pseudocode chain sync algorithm


#![allow(unused)]
fn main() {
struct State {
  ChainState chain_state,
  HashMap<Hash, Block> blocks
}

struct ChainState {
  Hash tip,
  HashSet<Hash> ancestors,
  Utxos ...,
  ...
}

impl ChainState {
  Fn is_ancestor(hash) -> bool {
    self.ancestors.exists(hash)
  }
}

// Fetch ‘dest_tip’ from `server’ and make it our tip, if it’s better.
sync(state, server, dest_tip, dest_tip_length) {
  if is_ancestor(dest_tip, state.chain_state.tip) {
    return; // nothing to do
  }

  // find a common ancestor of `dest_tip` and our tip.
  // FIXME: do binary search to find exact most recent ancestor
  n = 0;
  loop {
    hashes = server.get_chain_hashes(dest_tip, 2^n, 1);
    if hashes == [] {
      ancestor = genesis;
      break;
    }
    ancestor = hashes[0];
    if state.chain_state.has_ancestor(ancestor): { break }
    n++;
  }

  // fetch blocks from ancestor to dest_tip, in batches of 1000
  // blocks, forwards
  // FIXME: integer arithmetic is probably off a bit here, but you get the idea.
  nr_blocks_to_fetch = 2^n;
  batch_size = 1000;
  batches = nr_blocks_to_fetch / batch_size;
  new_chain_state = reconstruct_chain_state_at(ancestor);
  for (i = batches; i > 0; i--) {
    // validate the headers ahead of downloading blocks to validate
    // cryptographically invalid blocks. It is interesting to do that
    // ahead of time because of the small size of a BlockHeader
    new_hashes = server.get_chain_hashes(dest_tip, (i - 1) * batch_size, batch_size);
    new_headers = server.get_headers(new_hashes);
    if new_headers are invalid { stop; }
    new_blocks = server.get_blocks(new_hashes).reverse();
    for block in new_blocks {
      new_chain_state.validate_block(block)?;
      write_block_to_storage(block);
    }
  }

  if new_chain_state.chain_quality() > state.chain_state.chain_quality() {
    state.chain_state = new_chain_state
  }
}
}

Choice of wire Technology

We don't rely on any specific wire protocol, and only require that the wire protocol allow the transfer of the high level messages in a bidirectional way.

We chose to use GRPC/Protobuf as initial technology choice:

  • Efficiency: Using Protobuf, HTTP2, binary protocol
  • Bidirectional: through HTTP2, allowing stream of data. data push on a single established connection.
  • Potential Authentication: Security / Stream atomicity towards malicious MITM
  • Simplicity: Many languages supported (code generation, wide support)
  • Language/Architecture Independent: works on everything
  • Protobuf file acts as documentation and are relatively easy to version

Connections and bidirectional subscription channels can be left open (especially for clients behind NAT), although we can cycle connections with a simple RCU-like system.

Node-to-Client communication

Client are different from the node, in the sense that they may not be reachable by other peers directly.

However we might consider non reachable clients to keep an open connections to a node to received events. TBD

  • ReceiveNext : () -> Event

Another solution would be use use libp2p which also implements NAT Traversals and already has solutions for this.

Peer-to-Peer network

This section describes the construction of the network topology between nodes participating in the protocol. It will describes the requirements necessary to propagate the most efficiently the Communication Messages to the nodes of the topology.

Definitions

  • Communication Messages: the message that are necessary to be sent through the network (node-to-node and node-to-client) as defined above;
  • Topology: defines how the peers are linked to each other;
  • Node or Peer: an instance running the protocol;
  • Link: a connection between 2 peers in the topology;

Functionalities

  • A node can join the network at any moment;
  • A node can leave the network at any moment;
  • Node will discover new nodes to connect to via gossiping: nodes will exchange information regarding other nodes;
  • Nodes will relay information to their linked nodes (neighbors);
  • A node can challenge another node utilising the VRF in order to authentify the remote node is a specific stake owner/gatherer.

Messages

  • RingGossip: NodeProfileDetails * RING_GOSSIP_MAX_SIZE;
  • VicinityGossip: NodeProfileDetails * VICINITY_GOSSIP_MAX_SIZE;
  • CyclonGossip: NodeProfileDetails * CYCLON_GOSSIP_MAX_SIZE;

A node profile contains:

  • Node’s id;
  • Node’s IP Address;
  • Node’s topics (set of what the node is known to be interested into);
  • Node’s connected IDs

Design

The requirements to join and leave the network at any moment, to discover and change the links and to relay messages are all handled by PolderCast. Implementing PolderCast provides a good support to handle churn, fast relaying and quick stabilisation of the network. The paper proposes 3 modules: Rings, Vicinity and Cyclon.

Our addition: The preferred nodes

We propose to extend the number of modules with a 4th one. This module is static and entirely defined in the config file.

This 4th module will provide the following features:

  • Connect to specific dedicated nodes that we know we can trust (we may use a VRF challenge to validate they are known stakeholder -- they participated to numerous block creations);
    • This will add a static, known inter-node communications. Allowing users to build a one to one trusted topology;
    • A direct application for this will be to build an inter-stake-pool communication layer;
  • Static / configured list of trusted parties (automatically whitelisted for quarantine)
  • Metrics measurement related to stability TBD

Reports and Quarantine

In order to facilitate the handling of unreachable nodes or of misbehaving ones we have a system of reports that handles the state of known peers.

Following such reports, at the moment based only on connectivity status, peers may move into quarantine or other less restrictive impairements.

In the current system, a peer can be in any of these 4 states:

  • Available: the peer is known to the current node and can be picked up by poldercast layers for gossip and propagation of messages. This is the state in which new peers joining the topology via gossip end up.

  • Trusted: the last handshake between the peer and this node was successfull. For these kind of nodes we are a little more forgiving with reports and failures.

  • Quarantined: the peer has (possibly several) failed handshake attempts. We will not attempt to contact it again for some time even if we receive new gossip.

  • Unknown: the peer is not known to the current node.

    Actually, due to limitations of the poldercast library, this may mean that there are some traces of the peer in the profiles maintained by the current node but it cannot be picked up by poldercast layers for gossip or propagation. For all purposes but last resort connection attempts (see next paragraph), these two cases are essentially the same.

Since a diagram is often easier to understand than a bunch of sentences, these are the transitions between states in the current implementation, with details about avoiding network partition removed (see next paragraph).

Avoid network partitions

An important property of the p2p network is resilience to outages. We must avoid creating partitions in the network as much as possible.

For this reason, we send a manual (i.e. not part of poldercast protocol) optimistic gossip message to all nodes that were reported after the report expired. If this message fails to be delivered, no further action will be taken agains that peer to avoid cycling it back and forth from quarantine indefinitely. If instead the message is delivered correctly, we successfully prevented a possible partition :).

Another measure in place is a sort of a last resort attempt: if the node did not receive any incoming gossip for the last X minutes (tweakable in the config file), we try to contact again any node that is not quarantined and for which we have any trace left in the system (this is where nodes that were artificially forgotten by the system come into play).

Part to look into:

Privacy. Possibly Dandelion tech

Adversarial models considered

Adversarial forks

We consider an adversary whose goal is to isolate from the network nodes with stake. The impact of such successful attack would prevent block creation. Such adversarial node would propose a block that may look like a fork of the blockchain. Ouroboros Genesis allows fork up to an undetermined number of blocks in the past. The targeted would then have to do a large amount of block synchronisation and validation.

  • If the fork pretend to be in an epoch known to us, we can perform some cryptographic verifications (check the VRF);
  • If the fork pretends to be in an epoch long past, we may perform a small, controlled verification of up to N blocks from the forking point to verify the validity of these blocks;
  • Once the validity verified, we can then verify the locality aliveness of the fork and apply the consensus algorithm to decide if such a fork is worth considering.
  • However, suck attack can be repeated ad nauseam by any adversarial that happened to have been elected once by the protocol to create blocks. Once elected by its stake, the node may turn adversarial, creates as many invalid blocks, and propose them to the attacked node indefinitely. How do we keep track of the rejected blocks ? How do we keep track of the blacklisted stakeholder key or pool that are known to have propose too many invalid block or attempted this attack ?
    • Rejected block have a given block hash that is unlikely to collide with valid blocks, a node can keep a bloomfilter of hashes of known rejected block hash; or of known rejected VRF key;
    • The limitation of maintaining a bloom filter is that we may need to keep an ever growing bloom filter. However, it is reasonable to assume that the consensus protocol will organise itself in a collection of stakepools that have the resources (and the incentive) to keep suck bloom filter.

Flooding attack

We consider an adversary whose goal is to disrupt or interrupt the p2p message propagation. The event propagation mechanism of the pub/sub part of the p2p network can be leverage to continuously send invalid or non desired transactions to the network. For example, in a blockchain network protocol the transactions are aimed to be sent quickly between nodes of the topology so they may be quickly added to the ledger.

  • While it is true that one can create a random amount of valid transactions, it is also possible perform a certain amount of validation and policies to prevent the transaction message forwarding from flooding the network:
    • The protocol already requires the nodes to validate the signatures and that the inputs are unspent;
    • We can add a policy not to accept transaction that may imply a double spend, i.e. in our pool of pending transaction, we can check that there is no duplicate inputs.
  • The p2p gossiping protocols is an active action where a node decides to contact another node to exchange gossip with. It is not possible to flood the network with the gossiping messages as they do not require instant propagation of the gossips.

Anonymity Against distributed adversaries

We consider an adversary whose goal is to deanonymize users by linking their transactions to their IP addresses. This model is analysed in Dandelion. PolderCast already allows us to provide some reasonable guarantees against this adversary model.

  • Node do not share their links, they share a limited number of gossips based on what a node believe the recipient node might be interested in;
  • While some links can be guessed (those of the Rings module for example), some are too arbitrary (Vicinity or Cyclon) to determined the original sender of a transaction;

Man in the middle

We consider an adversary that could intercept the communication between two nodes. Such adversary could:

  • Escalate acquired knowledge to break the node privacy (e.g. user's public keys);
  • Disrupt the communication between the two nodes;

Potentially we might use SSL/TLS with dynamic certificate generation. A node would introduce itself to the network with its certificate. The certificate is then associated to this node and would be propagated via gossiping to the network.

In relation to Ouroboros Genesis

Each participants in the protocol need:

  • Key Evolving signature (KES) secret key
  • Verifiable Random Function (VRF) secret key

Apart from the common block deserialization and hashing verification, each block requires:

  • 2 VRF verification
  • 1 KES verification.

Considering the perfect network, it allow to calculate how many sequential hops, a block can hope to reach at a maximum bound.

testing

This section describes all libraries and tools in jormungandr repo which purpose is to satifsy quality needs of jormungandr and jcli components

Jormungandr test libraries includes projects:

  • jormungandr-automation - sets of apis for automating all node calls and node sub-components (REST, GRPC, logging etc.),
  • hersir - api & cli for bootstrapping entire network of nodes with some predefined configuration. Project takes care of proper settings for all nodes as well as block0,
  • thor - testing api & cli for all wallet operations,
  • mjolnir - load tool (api & cli) for all kind of jormungandr transactions,
  • loki - api & cli for sending invalid/adversary load as well as boostraping adversary node.

jormungandr-automation

Incubator of all testing apis for the node and jcli:

build

In order to build jormungandr-automation in main project folder run:

cd testing/jormungandr-automation
cargo build

jcli testing api

Api that can be used to run jcli executable underneath and is capable to assert outcome of command. It can work with already installed jcli (using PATH variable) or custom path. For Example:

    let jcli: JCli = Default::default();
    let private_key = jcli.key().generate("Ed25519Extended");
    let public_key = jcli.key().convert_to_public_string(&private_key);

jormungandr testing api

Collection of automation modules for node interaction and configuration:

  • configuration - allows to configure node & blockchain settings,
  • explorer - explorer configuration/bootstrap & interaction module,
  • grpc - module for grpc internode connection library handling. capable of sending some RPC calls as well as bootstrap receiver instance,
  • legacy - module for loosely typed configuration. This allow to bootstrap older version of node, for example to satisfy need on cross-version testing,
  • rest - module for jormungandr REST api testing,
  • starter - module for bootstrapping node,
  • verifier - node state verifier
  • logger - api for jormungandr log handling/assertion
  • process - api for handling jormungandr process

testing

Bunch of loosely coupled utility modules, mostly for additional configuration capabilities or benchmarking:

  • benchmark - measurements framework for various purposes, for example bootstrap time or how many transactions were successfully handled by node,
  • vit - additional helpers for voting capabilities,
  • asserts - asserts extensions, tailored for node needs,
  • block0 - block0 extensions, like easier access to blockchain setting or function to download block0,
  • collector - input collector utils,
  • configuration - test configuration helper (apps paths etc.),
  • keys - create default keys,
  • observer - simple observer framework,
  • panic - panic error reporting in test code,
  • process - process extensions,
  • resource - resources manager, mostly for tls certificates used for testing,
  • storage - node storage generators,
  • time - time utils, mostly for waiting for particular block date,
  • verify - substitute of asserts in case we don't want to panic eagerly when assertion is failed.

Hersir

Hersir is a cli & api project capable of bootstrapping local jormungandr network which can be exercised by various tools.

build & install

In order to build hersir in main project folder run:

cd testing/hersir
cargo build
cargo install --path . --force

quick start

The simplest configuration is available by using command:

hersir --config res\example.yaml

it results in small network with all data dumped to current folder

configuration

Simple example:

nodes:
    - spawn_params:
        alias: passive
        leadership_mode: passive
        persistence_mode: inmemory
      trusted_peers:
        - leader
    - spawn_params:
        alias: leader
        leadership_mode: leader
        persistence_mode: inmemory

blockchain:
    discrimination: test
    consensus: bft

  • nodes sections defines each network node. We can define alias, that is then used to express relations between the nodes and if we keep everything in memory or if node can mint blocks or not.

  • blockchain section defines blockchain parameters, like what is the consensus and if we are using test or production addresses discrimination.

full list of available parameters

nodes

  • spawn_params

    • alias: string (mandatory) - reference name of the node. Example: "alias",
    • bootstrap_from_peers: bool (optional) - should node bootstrap from trusted peers. By default it is auto-evaluated: If node doesn't have any trusted peers it won't bootstrap from peers,
    • faketime: custom (optional) - inject fake time settings. For example: 
        faketime:  {
      /// Clock drift (1 = no drift, 2 = double speed)
      drift: 1,
      /// Offset from the real clock in seconds
      offset: 2,
  }
    • gossip_interval: time (optional) - node gossip interval with the rest of the network. Format: number unit. For example: 10 s,
    • jormungandr: path (optional) - path to jormungandr node executable,
    • leadership_mode: enum (optional) - node leadership mode. Possible values:
      • passive - node won't be able to produce blocks,
      • leader - node will be able to mint blocks,
    • listen_address: string (optional) - override listen address for node. Example: /ip4/127.0.0.1/tcp/10005,
    • log_level: enum (optional) - log level, Possible values: (info/warn/error/debug/trace)
    • max_bootstrap_attempts: number (optional) - maximum number of bootstrap attempt before abandon,
    • max_connections: number (optional) - max connection node will create with other nodes,
    • max_inbound_connections: number (optional) - max inbound connection that node will accept,
    • mempool: custom (optional) - mempool configuration. Example: 
      mempool:
    pool_max_entries: 100000
    log_max_entries: 100000
    • network_stuck_check: time (optional) - check interval which node use to verify blockchain advanced. Format: number unit. For example: 10 s,
    • node_key_file: path (optional) - path to node network key,
    • persistence_mode: enum (optional) - set persistence mode. Possible values:
      • inmemory - everything is kept in node memory. If node restarts, all history is gone,
      • persistence - node uses local storage to preserve current state,
    • persistent_fragment_log: path (optional) - persistent fragment log serializes every fragment node receives via REST api,
    • policy: custom (optional) - defines nodes quarantine configuration. Example: 
       policy:
    quarantine_duration: 30m
    quarantine_whitelist:
      - "/ip4/13.230.137.72/tcp/3000"
      - "/ip4/13.230.48.191/tcp/3000"
      - "/ip4/18.196.168.220/tcp/3000"
    • preferred_layer: custom (optional) - defines preferences in gossiping. Example: 
        layers:
    preferred_list:
      view_max: 20
      peers:
        - address: "/ip4/13.230.137.72/tcp/3000"
          id: e4fda5a674f0838b64cacf6d22bbae38594d7903aba2226f
        - address: "/ip4/13.230.48.191/tcp/3000"
          id: c32e4e7b9e6541ce124a4bd7a990753df4183ed65ac59e34
        - address: "/ip4/18.196.168.220/tcp/3000"
          id: 74a9949645cdb06d0358da127e897cbb0a7b92a1d9db8e70
    • public_address: String (optional)- override public address for node. Example: /ip4/127.0.0.1/tcp/10005,
    • skip_bootstrap: bool (optional) - skips node bootstrap step,
    • topics_of_interest: custom (optional) - topics of interests describe how eager node will fetch blocks or transactions: 
      topics_of_interest:
  blocks: normal # Default is normal - set to high for stakepool
  messages: low  # Default is low    - set to high for stakepool
    • verbose: bool (optional) - enable verbose mode, which prints additional information,

  • trusted_peers: List (optional) - list of trusted peers. Example:

        trusted_peers:
      - leader
      - leader_1

blockchain

  • block0_date: date (optional) - block0 date, if not provided current date would be taken,
  • block_content_max_size: number (optional) - maximum block content size in bytes,
  • committees: list (optional) - list of wallet aliases which will be committees (capable of tallying the vote),
  • consensus: enum (optional) - blockchain consensus, possible values: Bft,GenesisPraos,
  • consensus_genesis_praos_active_slot_coeff: float (optional) - Determines minimum stake required to try becoming slot leader, must be in range (0,1],
  • discrimination: enum (optional) - type of discrimination of the blockchain, if this blockchain is meant for production then use production otherwise set test,
  • external_committees: list (optional) - list of committees to be included in block0,
  • external_consensus_leader_ids: list (optional) - list of external leaders id (apart from already defined nodes),
  • external_wallets: list (optional) - list of external wallets. Example:
  external_wallets:
      - alias: Alice
        address: ca1q47vz09320mx2qcs0gspwm47lsm8sh40af305x759vvhm7qyjyluulja80r
        value: 1000000000
        tokens: {}
  • kes_update_speed: number (optional) - the speed to update the KES Key in seconds,
  • linear_fee: custom (optional) - fee calculations settings,
  • slot_duration: number (optional) - The slot duration, in seconds, is the time between the creation of 2 blocks,
  • slots_per_epoch: number (optional) - number of slots in each epoch,
  • tx_max_expiry_epochs: number (optional) - transaction ttl (expressed in number of epochs).

session

  • jormungandr: path (optional) - override path to jormungandr. By default it's taken from PATH variable,
  • root: path (optional) - override path to local storage folder. By default all related data is dumped ino TEMP folder,
  • generate_documentation: bool (optional) - generate documentation files into local storage folder,
  • mode: enum (optional) - set hersir working mode. By default it's "standard", which just prints information about correct nodes bootstrap. Possible values:
    • monitor - prints current nodes status as progress bar,
    • standard - just prints information about correct nodes bootstrap,
    • interactive - spawn helper cli, which allows to interact with nodes,
  • log: enum (optional) - log level, Possible values: (info/warn/error/debug/trace),
  • title: string (optional) - give local storage folder name instead of random one.

full list of available commands

Full list of commands is available on hersir --help command.

hersir 0.1.0

USAGE:
    hersir [FLAGS] --config <config>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information
    -v, --verbose

OPTIONS:
    -c, --config <config>

Thor

Thor is a wallet cli & wallet api project which operates on jormungandr network.

WARNING: main purpose of the wallet is testing. Do NOT use it on production.

Build & Install

In order to build hersir in main project folder run:

cd testing/hersir
cargo build
cargo install --path . --force

Quick Start

CLI

Thor can be used as a wallet cli. It is capable of sending transactions or pull data from node. The simplest usage example is available by using commands:

  • register new wallet based on secret key: thor wallets import --alias darek --password 1234 secret.file

  • connect to node rest API: thor connect https://jormungandr.iohk.io/api

  • use recently created wallet for rest of commands: thor wallets use darek

  • sync with the node regarding wallet data: thor wallets refresh

  • send transaction: thor send tx --ada 5 --address ca1q5srhkdfuxqdm6h57mj45acxcdr57cr5lhddzkrjqyl8mmw62v9qczh78cu -pin 1234

API

Thor also allows you to use it as Api to perform any wallet operations from the code:

    use thor::{Wallet, FragmentSender, FragmentSenderSetup, FragmentVerifier};

    let receiver = thor::Wallet::default();
    let mut sender = thor::Wallet::default();

    // node bootstrap
    let jormungandr = ...

    let fragment_sender = FragmentSender::from_with_setup(
        jormungandr.block0_configuration(),
        FragmentSenderSetup::no_verify(),
    );

    fragment_sender
        .send_transaction(&mut sender, &receiver, &jormungandr, 1.into())
        .unwrap();

Configuration

Thor api doesn't use any configuration files. However cli uses small cache folder on filesystem (located in: ~/.thor). The purpose of this configuration is to store wallet lists as well as secret keys guarded by pass phrase.

full list of available commands

Full list of commands is available on thor --help command.

thor 0.1.0
Command line wallet for testing Jormungandr

USAGE:
    thor <SUBCOMMAND>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

SUBCOMMANDS:
    address                 Gets address of wallet in bech32 format
    clear-tx                Clears pending transactions to confirm. In case if expiration occured
    confirm-tx              Confirms successful transaction
    connect                 Sets node rest API address. Verifies connection on set
    help                    Prints this message or the help of the given subcommand(s)
    logs                    Prints entire fragment logs from the node
    pending-transactions    Prints pending transactions (not confirmed)
    refresh                 Pulls wallet data from the node
    send                    Sends fragments to nodes
    status                  Prints wallet status (balance/spending counters/tokens)
    statuses                Prints pending or already sent fragments statuses
    wallets                 Allows to manage wallets: add/remove/select operations

Mjolnir

Mjolnir is a load cli & api project which operates on jormungandr node.

Build & Install

In order to build mjolnir in main project folder run:

cd testing/mjolnir
cargo build
cargo install --path . --force

Quick Start

CLI

Mjolnir can be used as a cli. It is capable of putting various load on jormungandr node. It has couple of different load types:

  • explorer - Explorer load
  • fragment - Fragment load
  • passive - Passive Nodes bootstrap
  • rest - Rest load

Simplest load configuration is to use rest load with below parameters:

Rest load

USAGE:
    mjolnir.exe rest [FLAGS] [OPTIONS] --duration <duration> --endpoint <endpoint>

FLAGS:
    -h, --help       Prints help information
    -m, --measure    Prints post load measurements
    -V, --version    Prints version information

OPTIONS:
    -c, --count <count>                            Number of threads [default: 3]
        --delay <delay>                            Amount of delay [milliseconds] between sync attempts [default: 50]
    -d, --duration <duration>                      Amount of delay [seconds] between sync attempts
    -e, --endpoint <endpoint>                      Address in format: http://127.0.0.1:8002/api/
    -b, --progress-bar-mode <progress-bar-mode>    Show progress bar [default: Monitor]

API

Mjolnir main purpose is to serve load api:

use jortestkit::load::{self, ConfigurationBuilder as LoadConfigurationBuilder, Monitor};
use std::time::Duration;

    //node initialization
    let mut jormungandr = ...

    let rest_client = jormungandr.rest();

    // create request generator for rest calls
    let request = mjolnir::generators::RestRequestGen::new(rest_client);

    // duration based load run (40 seconds)
    let config = LoadConfigurationBuilder::duration(Duration::from_secs(40))
        // with 5 threads
        .thread_no(5)
        // with delay between each request 0.01 s
        .step_delay(Duration::from_millis(10))
        // with monitor thread monitor status of load run each 0.1 s
        .monitor(Monitor::Progress(100))
        // with status printer which prints out status of load run each 1 s
        .status_pace(Duration::from_secs(1_000))
        .build();

    // initialize load in sync manner (duration of each request is calculated by time difference between receiving response and sending request )
    let stats = load::start_sync(request, config, "Jormungandr rest load test");

    // finally some way to assert expected correctness, like percentage of successful requests
    assert!((stats.calculate_passrate() as u32) > 95);

full list of available commands

Full list of commands is available on mjolnir --help command.

mjolnir 0.1.0
Jormungandr Load CLI toolkit

USAGE:
    mjolnir.exe [FLAGS] [SUBCOMMAND]

FLAGS:
        --full-version      display full version details (software version, source version, targets and compiler used)
    -h, --help              Prints help information
        --source-version    display the sources version, allowing to check the source's hash used to compile this
                            executable. this option is useful for scripting retrieving the logs of the version of this
                            application
    -V, --version           Prints version information

SUBCOMMANDS:
    explorer    Explorer load
    fragment    Fragment load
    help        Prints this message or the help of the given subcommand(s)
    passive     Passive Nodes bootstrap
    rest        Rest load

Loki

Loki is an adversary node implementation and api which operates on jormungandr network.

Build & Install

In order to build hersir in main project folder run:

cd testing/loki
cargo build
cargo install --path . --force

Quick Start

Loki can be used bootstrap using cli:

loki --genesis-block block0.bin --listen-address 127.0.0.1:8080 -s secret.yaml

where:

genesis-block - Path to the genesis block (the block0) of the blockchain listen-address - Specifies the address the node will listen secret - Set the secret node config (in YAML format). Example:

---
bft:
    signing_key: ed25519_sk1w2tyr7e2w26w5vxv65xf36kpvcsach8rcdmlmrhg3rjzeumjnzyqvdvwfa

Then utilizing rest interface of loki node one can send some invalid GRPC messages to rest of the network:

curl --location --request POST 'http://127.0.0.1:8080/invalid_fragment' \
--header 'Content-Type: application/json' \
--data-raw '{
    "address": "127.0.0.1:1000",
    "parent": "tip"
}'

where:

address - address of "victim" node, parent - Parent block. Possible values:

  • tip - current tip of "victim" node,
  • block0 - block0,
  • {Hash} - arbitrary parent block which hash is provided in request

Other possible operations:

  • /invalid_hash - Sends block with invalid hash,
  • /invalid_signature - Sends block with invalid by wrong leader signature,
  • /nonexistent_leader - Sends block with non-existing leader,
  • /wrong_leader - Sends block with signed with invalid leader,

API

Loki also provides API for performing adversary operations, like sending invalid fragments:

    use loki::{AdversaryFragmentSender, AdversaryFragmentSenderSetup};

    let mut sender = ...
    let receiver = ..

    // node initialization
    let jormungandr = ...

    let adversary_sender = AdversaryFragmentSender::new(
        jormungandr.genesis_block_hash(),
        jormungandr.fees(),
        BlockDate::first().next_epoch().into(),
        AdversaryFragmentSenderSetup::no_verify(),
    );

    adversary_sender
        .send_faulty_transactions_with_iteration_delay(
            10,
            &mut sender,
            &receiver,
            &jormungandr,
            Duration::from_secs(5),
        )
        .unwrap();

jormungandr-integration-tests

Integration test is a container project for all jormungandr & jcli tests. Tests are validating node correctness, stability and interaction with database/rest api. Also there are non-functional tests which verify node durability and reliability

Architecture of tests

Jormungandr tests architecture relies on test pyramid approach, where most of the effort is put into until and api level and small amount of tests on E2E. Thanks to that we can create fast and reliable tests.

Testing architecture

Before approaching Jormungandr testing we need to first remind ourselves a simplified architecture diagram for jcli & jormungandr.

Simplified architecture

Quick start

Prerequisites

In order to run test jormungandr & jcli need to be installed or prebuild.

Start tests

In order to build jormungandr-automation in main project folder run:

cd testing
cargo test

Tests categories

Test are categories based on application/layer and property under test (functional or non-functional: load, perf etc.) Below diagram is a good overview:

Test categories

How to run all functional tests

cd testing/jormungandr-integration-tests
cargo test jormungandr --features network

How to run jcli only functional tests

cd testing/jormungandr-integration-tests
cargo test jcli

How to run single node functional tests

cd testing/jormungandr-integration-tests
cargo test jormungandr

How to run single node performance tests

cd testing/jormungandr-integration-tests
cargo test jormungandr::non_functional --features sanity,non-functional

How to run single node endurance tests

cd testing/jormungandr-integration-tests
cargo test jormungandr::non_functional --features soak,non-functional

How to run network functional tests

cd testing/jormungandr-integration-tests
cargo test jormungandr::network --features network

How to run network performance tests

cd testing/jormungandr-integration-tests
cargo test jormungandr::non_functional::network --features sanity,non-functional

How to run network endurance tests

cd testing/jormungandr-integration-tests
cargo test jormungandr::non_functional::network --features soak,non-functional

Frequency

Functional tests are run on each PR. Performance and testnet integration tests are run nightly