During the past two weeks, the consensus team started documenting the
implementation of the UTxO HD feature and continued developing tests for it. As
part of our work on UTxO HD, we improved the Haskell support for LMDB. We also
spent time working on the LSM tree prototype, and designed a parameter tuning
algorithm for it. Regarding our work on Genesis, our investigation of the
"plateaus" pointed at the
TICKF slowdown on era boundaries as culprit. This
led us to developing a caching strategy that will not only remove the
aforementioned "plateaus", but can help alleviating the growing block production
delay on epoch switch. We also helped reviewing the block forge credential
hotswap feature, which is intended for use in the adoption of P2P.
We also worked on paying technical debt and fostering collaboration. In
particular, we improved the
io-sim framework, which is crucial for testing and
simulating Cardano components. We also removed thunks that appeared on era
translations, and improved our diffusion pipelining feature. We are working on a
presentation for explaining Praos and Genesis.
High-level status report
- Finish the UTxO HD prototype: in progress.
- We added documentation for this feature.
- We developed the second version of the mempool tests.
- We fixed benchmarks that were inflating the speedup we observed in the
anti-diff implementation of sequences of differences. Speedups are now in the
range of [3.33, 4.75], which remain significant.
- We continued improving Haskell LMDB support.
- We finished implementing a "parameter tuning algorithm" for the LSM tree
prototype. This enables us to run experiments to check the correctness of
- Genesis: in progress.
- Work investigating the "plateaus" in the
ChainSync jumping prototype
pointed to the
TICKF slowdown on era boundaries as culprit.
- Tech debt:
- We improved the capabilities of our
io-sim library, which is crucial for
testing and simulating Cardano components.
- We removed thunks from epoch translations in the ledger.
- We added Linux CI support for
- We got pending diffusion pipelining improvements merged.
- Fostering collaboration:
- We are working on a explanation of Praos and Genesis protocols.
- Investigation of CSJ "plateaus" led us to developing a caching strategy for
TICKF that will not only remove these "plateaus", but can help alleviating
the growing block production delay on epoch switch.
- We reviewed the block forge credential hotswapping feature which is intended
for use in the adoption of P2P.
Finish the UTxO HD prototype
We merged PR #4060, which adds a report documenting the UTxO HD
feature, and puts emphasis in explaining how the mempool works in combination
with UTxO HD.
We opened a draft PR with the second iteration of the property tests for the
We fixed the
Arbitrary instances for keys and values in
(#4143). The problem was that we were testing with mostly small
values, which artificially boosted the performance gains we saw on benhcmarks.
Speedups are now in the range of [3.33, 4.75] across the different
Backing store property tests
We focused on incorporating feedback on the monadic cursor API PR (#1).
This required us to make small tweaks to
quickcheck-lockstep to test the new
API. We also updated the backing store property tests to use the new version of
the monadic cursor API.
LSM tree implementation
We worked on the LSM tree prototype. In particular: finished implementing a
"parameter tuning algorithm" that adapts the LSM tree design based on factors
- machine specs,
- and characteristics of the data being stored.
We are now running experiments to gather results and cross-reference them with
existing experimental results from the LSM tree paper to see if the algorithm is
Benchmarking the CSJ prototype
We focused on investigating the "plateaus" in the
ChainSync tip, which turned
out to be due to the
TICKF bug which we previously were only aware of in the
context of the long forging times near epoch boundaries. For the most drastic
patch by @nfrisby to speed up
TICKF, full sync is speeding up by 7%.
The following plot shows that by caching the
ChainSync tip and the
VolatileDB tip progress at the same rate.
The plot below shows the speedup observed by caching the
TICKF rule wrt the
After addressing the PR comments, we merged PR #16, which implements
MonadCatch instance for
STM. This extends the capability of our
library, which is crucial for testing and simulating Cardano components PR #16
closed #1461. This new feature was published as version
We continued with our work fixing the
NoThunk errors required for enabling
nightly tests, with the help of
TVarInvariant checks in
nothunks libraries. We proposed fixes in
cardano-ledger that took care of
thunks that appeared in era translations (#3143). The fixes will be
integrated back into consensus when
cardano-ledger approves and publish the
changes introduced in
We added CI support for
lmdb-simple (#2). We currently test the build on
a Linux environment only.
We got pending diffusion pipelining PRs (#3857, #3860,
#3856) merged, after rebasing and addressing feedback.
@nfrisby finished a visualisation tool and outlined scripts for the Praos and
Genesis explanation presentations. The idea is to produce a video that gives an
overview of these protocols.
We started working on caching the computation of the
(#4054), since this was blocking our benchmarking work for
Genesis. In addition, this issue has the Cardano community quite
concerned, so we are hoping the work done in caching the
computation of the
TICKF rule can help alleviating the growing block
production delay on epoch switch.
We reviewed the block forge credential hotswapping PR #3800 from the
networking team, which is intended for use in the adoption of P2P.