Skip to main content

3 posts tagged with "network"

View All Tags

· 3 min read
Marcin Szamotulski

Network Quarterly Update

2023-01 - 2023-03

Main achievements

Gradual dynamic P2P release on mainnet

We released two version of cardano-node with dynamic P2P capabilities:

  • 1.35.6
    • we found and fixed a bug in exception handling in peer-state-actionspull-4357
    • we found and fixed a busy loop when demoting a peer from hot to warm pull-4385
  • 1.35.7
    • includes interoperability in the legacy non-p2p network stack pull-4467
  • we fixed a busy loop of demotion & promotions: warm -> hot -> warm[pull-4485] /it will be included in cardano-node-8.0.0 release/.

Currently there are more than 200 P2P relays on mainnet.

Peer Sharing

We implemented /peer sharing/ pull-4019 which will be available as an experimental feature in one of the future cardano-node releases.

We implemented /light peer sharing/, e.g. adding inbound connections to the set of known peers of the outbound governor, which allows to bootstrap relays not registered on chain. This complements peer sharing. The pull-4277 is in late review stages.

Eclipse Evasion

We finalised design of eclipse evasion and we started implementing it. We have an initial implementation (not merged). We are in the process of extending our test suite to cover new implementation details: issue-3886, pull-4462.

Cardano Network Service Assurance

Galois has been making progress on Cardano Network Service Assurance project.

  • In cardano-node, they have developed a datapoint abstraction that creates a queue of (existing) log events, they now have two such datapoints (of log events) implemented.

  • They have developed a datapoint client executable that can connect to a node which serves the "new tracing".

  • They have been exploring approaches for the consolidation and analysis of datapoint data to extract actionable network health status.


  • We made it possible to configure accepted connections limit pull-4902.

Testing improvements

  • We fixed a bug in network simulation implementation of TCP simultaneous open pull-4265.

  • We introduced header-body split in the diffusion simulation pull-4419 (in review).

  • We introduced initiator only nodes in the diffusion simulation pull-4280.

  • We fixed a connection-manager test failure issue-4370.

Technical Debt

  • We refactored Snocket interface decoupling it from the multiplexer pull-4260. This simplified some aspects of the KES agent implementation.

  • We introduces a record for CBOR codecs which are used for various data structures by mini-protocol codecs pull-4430.


  • We explained some limitations of CDDL in our technical report pull-4351.


  • We fixed implementation of MVar's pull-70.


  • We published a new version of nothunks library to Hackage.

Next steps

  • Finish implementation & testing of eclipse evasion issue-3886.
  • Optimise connectivity to peers behind firewall issue-4381.
  • Finish the work on enabling block production dynamically to allow using P2P on block producers issue-3159.
  • If time permits we would like also to reserve some time for finishing publication of io-sim to Hackage.

· 5 min read
Marcin Szamotulski

Network Quarterly Update

2022-11 - 2023-01


The primary goal of the networking team was to focus on the single relay release of P2P. We fixed a number of small late bugs, and concluded QA & performance testing. Although it was discovered a regression in performance of block production when P2P is enabled, relaying with P2P performs better comparing to a non p2p. We concluded that this is not a blocker for the Single Relay Release which is planned shortly.

Peer sharing has gone through review and final review is just being done right now. After merging it will still be disabled (hidden behind a flag) as it's not safe without eclipse evasion. We started implementing light peer sharing (i.e. include inbound peers into known peer set of the outbound governor).

We started a detailed eclipse evasion design, it will continue in the next quarter.

We also made a major revision of package structure of the network packages. We ended up with a very clean dependency graph (pr #4155).

Armando Santos delivered a talk at the ODOPIS 2022 conference on principles of distributed systems in Brussels. The slides are available here.

Neil Davies gave an invited seminar on DeltaQ at Université Catholique de Louvain.

We also found and fixed a few of bugs:

  • a bug in keep alive mini-protocol which resulted in warm to cold transitions to be always executed through a timeout path rather than do a clean demotion ([pr #4168]).

  • fixed an assetion failure in the outbound governor (issue #4177)

Next steps

We will work towards the next release of P2P for block producer nodes. This includes:

  • analysing performance regression for BP nodes when using P2P
  • finish the work on controlling the block forger through node kernel (pr #3800)
  • address issue #3907 and write a script to analyse deployment of P2P relays

We would like also to push forward eclipse evasion. Although most of the work has be done already the release of io-sim on Hackage will happen in the next quarter.

We would also like to address chain-sync timeout issue recently diagnosed by Karl Knutsson.

If time permits we would also like to address some technical debt, especially:


The performance regression for block producer with P2P needs to be investigated in the near future. This is blocker for the release of P2P on BP nodes.

Detailed log

Contributions to Ouroboros-Network

  • We added TraceDemoteLocalAsynchronous, which enables notification of critical issues for SPOs
  • We fixed cardano-ping compatibility with NodeToNodeV_10 (P2P, pr #4165)
  • We fixed a bug in demotion peers to cold which affected P2P nodes (commit-61058aa5c2)
  • Karl Knutsson enhanced SendFetchRequest (commit-bb1c3dddee), open-source contribution)
  • We turned SizeInBytes into a newtype.
  • We extended,, added CODE_OF_CONDUCT.
  • We fixed DNS test failure issue #4191
  • We fixed a simulation bug found in issue #4258
  • [pr #4168]
  • issue #4177

Contributions to Cardano-Node

  • We maintained the Single Relay Release pr #4612, (e.g. fixing CI issues, Rebasing it when necessary, publishing packages to Cardano Haskell Packages);
  • We enhanced JSON serialisation / deserialisation of NodeToNodeVersion and NodeToClientVersion;

Contributions to IOSim

  • We started to use Cardano Haskell Packages for IOSim (pr #48)
  • We updated change log files
  • We added support of ghc-9.4 (pr #50)

We also addressed the following issues in pr #57 in order to prepare the package for publication on Hackage:

  • refactored io-classes timers API (issue #46);
  • created a new package si-timers which exposes an interface using SI units and is safe on 32-bit systems (issue #59);
  • added monad transformers instances for classes defined in io-classes (issue #58);
  • created io-classes-mtl package which includes (experimental) instances for monad transformers;
  • provide MonadMonotonicTimeNSec in io-classes and MonadMonotonicTime in si-timers (so that io-classes follow the base package);
  • added registerCancellableDelay in si-timers (which allowed us to hide fancy timer api and clean io-classes)
  • added support for js_HOST_ARCH (the new GHC JS backend)

Note the pr #57 contains almost 40 commits, and was a major step forward for io-sim ecosystem. We also prepared a draft pr #4281 which updates ouroboros-network.

Other changes for release on Hackage:

  • Refactored test suite (pr #47)
  • Updated documentation, cabal files, CONTRIBUTING, SECURITY documents, etc in pr #60, currently under review.

· 4 min read
Marcin Szamotulski

Network Quarterly Update

2022-09 - 2022-11

Summary of most important improvements

During this quarter the networking team delivered low level specification of peer sharing & eclipse evasion. We held a session with the consensus & the scientists; we got a positive feedback on the design.

Further we focused on implementation of peer sharing. We produced a detail design and an early implementation.

We prepared the P2P Single Relay Release (cardano-node-1.35.5). It includes over 130 patches of network stack improvements over the previous version 1.35.4, which were accomplished over a longer period of time. Among them are both bug fixes and UX improvements for stake pool operators like simplified format of the topology file, or improvements in the logged messages:

We also provide better integration with systemd (socket activation improvements) or improvements in the networking stack:

  • exit policies,
  • peer metrics improvements,
  • DNS TTL improvements (which make it harder to misconfigure the system, an issue discovered by the performance & monitoring team),
  • do not trigger inbound idle timeout for node-to-client connections (pr #3844), an issue reported to us by Matthias Benkort from Cardano Foundation.

Duncan has been making progress with the input endorsers demo. His simulation provides a useful animated visualisation and live quantification of behaviour of the modeled design.

We also improved our e2e diffusion simulation by implementing header-body split, similar to what the real implementation does.

We also made some advances towards our future goals of P2P release for block producer nodes (pr #3800 - in review) & for Daedalus users (pr #3690 - merged).

Detailed log

  • We expanded diffusion simulation with block-fetch protocol bringing it closer to the production system.

  • We addressed some additional technical depth in diffusion simulation

  • We slightly improved documentation & CI of io-sim and typed-protocols repositories for open-source contributors.

  • We closed a number of issues towards publishing io-sim on Hackage (only two essential issues are left open).

  • We pushed a branch of typed-protocols which captures one of the developer UX problems in the API which we need to solve.

  • We identified and fixed an issue related to systemd sockets.

  • We identified and fixed an issue in consensus initialisation not giving feedback on early errors.

  • We deployed RT View, identified a number of issues which were communicated to the performance & monitoring team.

  • We finished high level & detailed design of peer sharing, very early implementation of peer sharing is done (note that peer sharing cannot be safely deployed without eclipse evasion & genesis).

  • We finished high level design of eclipse evasion, and started working on a detailed design.

  • We were assigned the role of release engineer for 1.35.5 release (the P2P single relay release); we prepared a cardano-node for 1.35.5 release which contains more than 130 patches of just network stack improvements done over last few months.

  • We diagnosed and fixed an tricky bug in the peer state actions (a component which sits between outbound governor and connection manager). That bug was introduced earlier this year and never released. It was caught by the QA testing framework. We expanded our diffusion simulation to cover such case and also mitigated a chance for reintroducing such a bug in future.

  • We identified and quite likely mitigated a misconfiguration in the benchmarking cluster (next benchmarking run will confirm our hypothesis).

  • We simplified the format of p2p topology file, we got positive feedback from SPOs.

  • We raised severities of some of the logging messages, which is an important improvement for SPOs, exchanges and other users of the system.

  • We worked on input endorsers simulation which gives both animated and quantified live feedback on network operation, using a simplified model of a TCP/IP network.

Next quarter

  • Release the Single Relay P2P Release 1.35.5.

  • Carry on with Peer Sharing (review, testing).

  • Deliver a talk at Conference on Principles of Distributed Systems 2022 in Brussels, Belgium.

  • Present Detailed Design of Eclipse Evasion and start implementation phase.

  • Work on P2P Block Producer release.

  • Carry on with publishing of io-sim on Hackage.