Network overview

Jörmungandr network capabilities are split into:

1. the REST API, used for informational queries or control of the node;
2. the gRPC API for blockchain protocol exchange and participation;

Here we will only review the gRPC API as the REST API is described in another chapter: go to the REST documentation

The protocol

The protocol is based on gRPC that combines commonly used protocols like HTTP/2 and RPC. More precisely, Jörmungandr utilises.

This choice was made because gRPC is already widely supported around the world because of it’s uitilization of standard protocols HTTP/2 which makes it much easier for Proxies and Firewalls to recognise the protocol and permit the traffic.

Type of queries

The protocol allows you to send multiple types of messages between nodes:

• sync block to remote peer’s Last Block (tip).
• propose new fragments (new transactions, certificates, …): this is for the fragment propagation.
• propose new blocks: for block propagation.

There are other commands that optimise the communication and synchronization between nodes that will be documented here in the future.

Another type of messages is the Gossip message. These gossip messages allow Nodes to exchange information (gossips) about other nodes on the network, allowing for peer discovery.

Peer to peer

The peer 2 peer connections are established utilising multiple components:

• A multilayered topology (e.g. Poldercast);
• Gossiping for node discoverability;
• Subscription mechanism for event propagation;
• Security and countermeasures: (such as Topology Policy for scoring and/or blacklisting nodes);

Multilayered topology

As described in the Poldercast paper, our network topology is built on multiple layers that allow for granular control of it’s behavior. In practice this means a node will have different groups of nodes that it connects to based on different algorithms, each of these groups are a subset of the whole known list of nodes.

In short we have:

• The rings layer selects a predecessor(s) and a successor(s) for each topic (Fragment or Blocks);
• The Vicinity layer will select nodes that have similar interests;
• The Cyclon layer, will select nodes randomly.

However, we keep the option open to remove some of these layers or to add new ones, such as:

• A layer to allow privilege connections between stake pools;
• A layer for the user’s whitelist, a list of nodes the users considered trustworthy and that we could use to check in the current state of the network and verify the user’s node is not within a long running fork;

Gossiping

Gossiping is the process used for peer discovery. It allows two things:

1. For any nodes to advertise themselves as discoverable;
2. To discover new nodes via exchanging a list of nodes (gossips);

The gossips are selected by the different layers of the multilayered topology. For the Poldercast modules, the gossips are selected just as in the paper. Additional modules may select new nodes in the gossip list or may decide to not add any new information.

Subscription mechanism

Based on the multilayered topology, the node will open multiplexed and bi-directional connections (thanks to industry standard gRPC, this comes for free). These bi-directional connections are used to propagate events such as:

• Gossiping events, when 2 nodes exchange gossips for peer discovery;
• Fragment events, when a node wants to propagate a new fragment to other nodes;
• Block events, when a node wants to propagate a new block creation event

Security and countermeasures

In order to facilitate the handling of unreachable nodes or of misbehaving ones we have built a node policy tooling. Currently, we collect connectivity statuses for each node. The policy can then be tuned over the collected data to apply some parameters when connecting to a given node, as well as banning nodes from our topology.

For each node, the following data is collected:

Connection statuses:

• The failed connection attempts and when it happened;
• Latency
• Last message used per topic item (last time a fragment has been received from that node, last time a block has been received from that node…)

In the future, we may expand the polocy to include data collected at the blockchain level lile:

• Faults (e.g. trying to send an invalid block)
• Contributions in the network
• Their blockchain status (e.g. tips)

Policy

The p2p policy provides some more fine control on how to handle nodes flagged as not behaving as expected (see the list of data collected).

It currently works as a 4 levels: trusted, possible contact, quarantined, forgotten. Each gossip about a new node will create a new entry in the list of possible contact. Then the policy, based on the logged data associated to this node, may decide to put this node in quarantine for a certain amount of time.

Trusted nodes are the ones to which we were able to connect successfully. A connectivity report against those nodes will make them transition to the possible contact level, while a successful connection attempt will promote them again to trusted.

The changes from one level to another is best effort only. Applying the policy may be costly so the node applies the policy only on the node it is interested about (a gossip update or when reporting an issue against a node). This guarantees that the node does not spend too much time policing its database. And it also makes sure that only the nodes of interest are up to date. However it is possible for the node to choose, at a convenient time, to policy the whole p2p database. This is not enforced by the protocol.