Introduction

This module implements PeerStateActions, which provide the following capabilities::

synchronous promotions / demotions

• establishPeerConnection
• activatePeerConnection
• deactivatePeerConnection
• closePeerConnection

asynchronous demotions

Monitor mini-protocols and act on mini-protocol state changes done via monitorPeerConnection.

Synchronous promotions / demotions

Synchronous promotions / demotions are directly used by peerSelectionGovernor.

synchronous cold → warm transition

This transition starts with creating or reusing an inbound connection, do handshake (functionality provided by connection manager), start established and warm mini-protocols, start monitoring thread specified below.

synchronous warm → hot transition

This transition quiesce warm protocols and starts hot protocols. There is no timeout to quiesce warm mini-protocols. The tip-sample protocol which is the only planned warm protocol has some states that have a longer timeout when the remote peer has agency, but it does not transfers much data.

synchronous hot → warm transition

Within a timeout, stop hot protocols and let the warm protocols continue running. If the timeout expires the connection is closed. Note that this will impact inbound side of a duplex connection. We cannot do any better: closing is a cooperative action since we require to arrive at a well defined state of the multiplexer (no outstanding data in ingress queue). This transition must use last to finish synchronisation of all hot mini-protocols.

synchronous warm → cold transition

Shutdown established and warm protocols. As in the previous transition it must use last to finish synchronisation on established and warm protocol termination, if this synchronisation timeouts the connection is closed.

Monitoring Loop

The monitoring loop is responsible for taking an action when one of the mini-protocols either terminates or errors. Except termination of a hot protocols we shall close the connection. When one of the hot protocols terminates we trigger a synchronous hot → warm transition.

The monitoring loop is supposed to stop when the multiplexer stops.

Note that the monitoring loop must act as soon as one of the mini-protocols terminates or errors, hence the use of first to finish synchronisation.

The multiplexer guarantees that whenever one of the mini-protocols errors the connection is closed. This simplifies the actions needed to be taken by the monitoring loop.

Asynchronous demotions

asynchronous * → cold transition

This demotion is triggered whenever any of the mini-protocol errors. This does not require a further action by the monitoring loop: mux will close the connection, monitoring loop will terminate.

asynchronous hot → warm demotion

This demotion is triggered if a hot mini-protocol terminates cleanly. In this case we trigger synchronous hot → warm demotion which will halt all hot mini-protocols and will notify the peer-to-peer governor about the change.

Implementation details

PeerStateActions are build on top of ConnectionManager which provides a primitive to present us a negotiated connection (i.e. after running the handshake) and the multiplexer api which allows to start mini-protocols and track their termination via an STM interface. Each connection has associated PeerConnectionHandle which holds all the data associated with a connection.

Most important are pchMux :: Mux mode m which allows us to interact with the multiplexer and pchAppHandles. The latter contains information about each mini-protocol and its STM mini-protocol monitoring action. ahMiniProtocolResults allows us to build last-to-finish awaitAllResults and first-to-finish awaitFirstResult synchronisations that we need in synchronous transitions and monitoring loop respectively.

ahControlVar is a per-temperature TVar which holds ControlMessage. It is passed from ConnectionHandler via Handle. This variable allows us to terminate, quiesce or re-enable mini-protocols.

Bellow is a schematic illustration of function calls / threads and shared state variables. Reads done just make assertions are not included. The diagram does not include establishPeerConnection.

Legend: ─  - functions
        │░ - threads
        ━  - STM mutable variables

        ├──▶┃ - write to a TVar
        │◀──┨ - read from a TVar
        ├──▶│ - function call

        PeerStateVar        - 'pchPeerState' 'TVar'
        MiniProtocolResults - 'ahMiniProtocolResults' 'TVar'
        ControlVar          - 'ahControlVar' 'TVar'




                    ┌──────────────────────────────────────────┐
                    │ ┌────────┐                               │
                    │ │        │                               │
   ┌────────────────┴─┴─┐      │                               │
  ┌────────────────────┐│      ▼                               ▼
 ┌────────────────────┐││   ┌──────────────────────────┐     ┌─────────────────────┐
 │░░░░░░░░░░░░░░░░░░░░│││   │                          │     │                     │
 │░peerMonitoringLoop░││┘   │ deactivatePeerConnection │     │ closePeerConnection │
 │░░░░░░░░░░░░░░░░░░░░│┘    │                          │     │                     │
 └┬───────────────────┘     └┬────────────────────┬────┘     └───────┬─────────────┘
  │     ▲                    │   ▲                │              ▲ ▲ │
  │ ┌───┼────────────────────┘   │                │              │ │ │
  │ │ ┌─┼────────────────────────┼────────────────┼──────────────┘ │ │
  │ │ │ │                        │     ┌──────────┼────────────────┘ │
  │ │ │ │                        │     │          │ ┌────────────────┘
▒▒│▒│▒│▒│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│▒▒▒▒▒│▒▒▒▒▒▒▒▒▒▒│▒│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒ │ │ │ └───────────────┐        │     │          │ │                       ▒▒▒
▒ ▼ ▼ ▼                 │        │     │          ▼ ▼                       ▒ ▒▒▒
▒┏━━━━━━━━━━━━━━┓      ┏┷━━━━━━━━┷━━━━━┷┓     ┏━━━━━━━━━━━━━━━━┓            ▒ ▒ ▒
▒┃              ┃┓     ┃                ┃┓    ┃                ┃┓           ▒ ▒ ▒
▒┃ PeerStateVar ┃┃┓    ┃  MiniProtocol  ┃┃┓   ┃  ControlVar    ┃┃┓          ▒ ▒ ▒
▒┃              ┃┃┃    ┃     Results    ┃┃┃   ┃  - established ┃┃┃          ▒ ▒ ▒
▒┃              ┃┃┃    ┃  - established ┃┃┃   ┃  - warm        ┃┃┃          ▒ ▒ ▒
▒┗━━━━━━━━━━━━━━┛┃┃    ┃  - warm        ┃┃┃   ┃  - hot         ┃┃┃          ▒ ▒ ▒
▒ ┗━━━━━━━━━━━━━━┛┃    ┃  - hot         ┃┃┃   ┃                ┃┃┃          ▒ ▒ ▒
▒  ┗━━━━━━━━━━━━━━┛    ┃                ┃┃┃   ┃                ┃┃┃          ▒ ▒ ▒
▒  ▲                   ┗━━━━━━━━━━━━━━━━┛┃┃   ┗━━━━━━━━━━━━━━━━┛┃┃          ▒ ▒ ▒
▒  │                    ┗━━━━━━━━━━━━━━━━┛┃    ┗━━━━━━━━━━━━━━━━┛┃          ▒ ▒ ▒
▒  │                     ┗━━━━━━━━━━━━━━━━┛     ┗━━━━━━━━━━━━━━━━┛          ▒ ▒ ▒
▒  │                                             ▲                          ▒ ▒ ▒
▒  │                   PeerConnectionHandles     │                          ▒ ▒ ▒
▒▒▒│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ ▒ ▒
 ▒ │                                             │                            ▒ ▒
 ▒▒│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ ▒
  ▒│                                             │                              ▒
  ▒│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
   │                   ┌─────────────────────────┘
 ┌─┴───────────────────┴──┐
 │                        │
 │ activatePeerConnection │
 │                        │
 └────────────────────────┘

Notes:

All three upper boxes: peerMonitoringLoop, deactivatePeerConnection and closePeerConnection are reading ahMiniProtocolResults via the last-to-finish awaitAllResults synchronisation.

All of the thin boxes are writing to pchPeerState variable; which is read by 'monitorPeerConnection. Also all of them writing to ahControlVar: peerMonitoringLoop does that through a call to deactivePeerConnection or closePeerConnection.