ouroboros-consensus-0.1.0.2: Consensus layer for the Ouroboros blockchain protocol
Safe HaskellNone
LanguageHaskell2010

Ouroboros.Consensus.Storage.LedgerDB.OnDisk

Synopsis

Opening the database

data InitFailure blk Source #

Constructors

InitFailureRead ReadIncrementalErr

We failed to deserialise the snapshot

This can happen due to data corruption in the ledger DB.

InitFailureTooRecent (RealPoint blk)

This snapshot is too recent (ahead of the tip of the chain)

InitFailureGenesis

This snapshot was of the ledger state at genesis, even though we never take snapshots at genesis, so this is unexpected.

Instances

Instances details
StandardHash blk ⇒ Eq (InitFailure blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

Methods

(==)InitFailure blk → InitFailure blk → Bool Source #

(/=)InitFailure blk → InitFailure blk → Bool Source #

StandardHash blk ⇒ Show (InitFailure blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

Generic (InitFailure blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

Associated Types

type Rep (InitFailure blk) ∷ TypeType Source #

Methods

fromInitFailure blk → Rep (InitFailure blk) x Source #

toRep (InitFailure blk) x → InitFailure blk Source #

type Rep (InitFailure blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

type Rep (InitFailure blk) = D1 ('MetaData "InitFailure" "Ouroboros.Consensus.Storage.LedgerDB.OnDisk" "ouroboros-consensus-0.1.0.2-inplace" 'False) (C1 ('MetaCons "InitFailureRead" 'PrefixI 'False) (S1 ('MetaSel ('NothingMaybe Symbol) 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 ReadIncrementalErr)) :+: (C1 ('MetaCons "InitFailureTooRecent" 'PrefixI 'False) (S1 ('MetaSel ('NothingMaybe Symbol) 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 (RealPoint blk))) :+: C1 ('MetaCons "InitFailureGenesis" 'PrefixI 'False) (U1TypeType)))

data InitLog blk Source #

Initialization log

The initialization log records which snapshots from disk were considered, in which order, and why some snapshots were rejected. It is primarily useful for monitoring purposes.

Constructors

InitFromGenesis

Defaulted to initialization from genesis

NOTE: Unless the blockchain is near genesis, we should see this only if data corrupted occurred.

InitFromSnapshot DiskSnapshot (RealPoint blk)

Used a snapshot corresponding to the specified tip

InitFailure DiskSnapshot (InitFailure blk) (InitLog blk)

Initialization skipped a snapshot

We record the reason why it was skipped.

NOTE: We should only see this if data corrupted occurred.

Instances

Instances details
StandardHash blk ⇒ Eq (InitLog blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

Methods

(==)InitLog blk → InitLog blk → Bool Source #

(/=)InitLog blk → InitLog blk → Bool Source #

StandardHash blk ⇒ Show (InitLog blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

Methods

showsPrecIntInitLog blk → ShowS Source #

showInitLog blk → String Source #

showList ∷ [InitLog blk] → ShowS Source #

Generic (InitLog blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

Associated Types

type Rep (InitLog blk) ∷ TypeType Source #

Methods

fromInitLog blk → Rep (InitLog blk) x Source #

toRep (InitLog blk) x → InitLog blk Source #

type Rep (InitLog blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

initLedgerDB Source #

Arguments

∷ ∀ m blk. (IOLike m, LedgerSupportsProtocol blk, InspectLedger blk, HasCallStack) 
⇒ Tracer m (ReplayGoal blk → TraceReplayEvent blk) 
→ Tracer m (TraceEvent blk) 
SomeHasFS m 
→ (∀ s. Decoder s (ExtLedgerState blk)) 
→ (∀ s. Decoder s (HeaderHash blk)) 
LedgerDbCfg (ExtLedgerState blk) 
→ m (ExtLedgerState blk)

Genesis ledger state

StreamAPI m blk 
→ m (InitLog blk, LedgerDB' blk, Word64) 

Initialize the ledger DB from the most recent snapshot on disk

If no such snapshot can be found, use the genesis ledger DB. Returns the initialized DB as well as the block reference corresponding to the snapshot we found on disk (the latter primarily for testing/monitoring purposes).

We do not catch any exceptions thrown during streaming; should any be thrown, it is the responsibility of the ChainDB to catch these and trigger (further) validation. We only discard snapshots if

  • We cannot deserialise them, or
  • they are ahead of the chain

It is possible that the Ledger DB will not be able to roll back k blocks after initialization if the chain has been truncated (data corruption).

We do not attempt to use multiple ledger states from disk to construct the ledger DB. Instead we load only a single ledger state from disk, and compute all subsequent ones. This is important, because the ledger states obtained in this way will (hopefully) share much of their memory footprint with their predecessors.

Instantiate in-memory to blk

Abstraction over the stream API

data NextBlock blk Source #

Next block returned during streaming

Constructors

NoMoreBlocks 
NextBlock blk 

data StreamAPI m blk Source #

Stream blocks from the immutable DB

When we initialize the ledger DB, we try to find a snapshot close to the tip of the immutable DB, and then stream blocks from the immutable DB to its tip to bring the ledger up to date with the tip of the immutable DB.

In CPS form to enable the use of withXYZ style iterator init functions.

Constructors

StreamAPI 

Fields

Read from disk

readSnapshot ∷ ∀ m blk. IOLike m ⇒ SomeHasFS m → (∀ s. Decoder s (ExtLedgerState blk)) → (∀ s. Decoder s (HeaderHash blk)) → DiskSnapshotExceptT ReadIncrementalErr m (ExtLedgerState blk) Source #

Read snapshot from disk

Write to disk

takeSnapshot ∷ ∀ m blk. (MonadThrow m, IsLedger (LedgerState blk)) ⇒ Tracer m (TraceEvent blk) → SomeHasFS m → (ExtLedgerState blk → Encoding) → LedgerDB' blk → m (Maybe (DiskSnapshot, RealPoint blk)) Source #

Take a snapshot of the oldest ledger state in the ledger DB

We write the oldest ledger state to disk because the intention is to only write ledger states to disk that we know to be immutable. Primarily for testing purposes, takeSnapshot returns the block reference corresponding to the snapshot that we wrote.

If a snapshot with the same number already exists on disk or if the tip is at genesis, no snapshot is taken.

Note that an EBB can have the same slot number and thus snapshot number as the block after it. This doesn't matter. The one block difference in the ledger state doesn't warrant an additional snapshot. The number in the name of the snapshot is only indicative, we don't rely on it being correct.

NOTE: This is a lower-level API that takes a snapshot independent from whether this snapshot corresponds to a state that is more than k back.

TODO: Should we delete the file if an error occurs during writing?

trimSnapshotsMonad m ⇒ Tracer m (TraceEvent r) → SomeHasFS m → DiskPolicy → m [DiskSnapshot] Source #

Trim the number of on disk snapshots so that at most onDiskNumSnapshots snapshots are stored on disk. The oldest snapshots are deleted.

The deleted snapshots are returned.

writeSnapshot ∷ ∀ m blk. MonadThrow m ⇒ SomeHasFS m → (ExtLedgerState blk → Encoding) → DiskSnapshotExtLedgerState blk → m () Source #

Write snapshot to disk

Low-level API (primarily exposed for testing)

deleteSnapshotHasCallStackSomeHasFS m → DiskSnapshot → m () Source #

Delete snapshot from disk

opaque

data DiskSnapshot Source #

Constructors

DiskSnapshot 

Fields

  • dsNumberWord64

    Snapshots are numbered. We will try the snapshots with the highest number first.

    When creating a snapshot, we use the slot number of the ledger state it corresponds to as the snapshot number. This gives an indication of how recent the snapshot is.

    Note that the snapshot names are only indicative, we don't rely on the snapshot number matching the slot number of the corresponding ledger state. We only use the snapshots numbers to determine the order in which we try them.

  • dsSuffixMaybe String

    Snapshots can optionally have a suffix, separated by the snapshot number with an underscore, e.g., 4492799_last_Byron. This suffix acts as metadata for the operator of the node. Snapshots with a suffix will not be trimmed.

Instances

Instances details
Eq DiskSnapshot Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

Ord DiskSnapshot Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

Show DiskSnapshot Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

Generic DiskSnapshot Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

Associated Types

type Rep DiskSnapshotTypeType Source #

type Rep DiskSnapshot Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

type Rep DiskSnapshot = D1 ('MetaData "DiskSnapshot" "Ouroboros.Consensus.Storage.LedgerDB.OnDisk" "ouroboros-consensus-0.1.0.2-inplace" 'False) (C1 ('MetaCons "DiskSnapshot" 'PrefixI 'True) (S1 ('MetaSel ('Just "dsNumber") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 Word64) :*: S1 ('MetaSel ('Just "dsSuffix") 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 (Maybe String))))

Trace events

newtype ReplayGoal blk Source #

Which point the replay is expected to end at

Constructors

ReplayGoal (Point blk) 

Instances

Instances details
StandardHash blk ⇒ Eq (ReplayGoal blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

Methods

(==)ReplayGoal blk → ReplayGoal blk → Bool Source #

(/=)ReplayGoal blk → ReplayGoal blk → Bool Source #

StandardHash blk ⇒ Show (ReplayGoal blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

Methods

showsPrecIntReplayGoal blk → ShowS Source #

showReplayGoal blk → String Source #

showList ∷ [ReplayGoal blk] → ShowS Source #

newtype ReplayStart blk Source #

Which point the replay started from

Constructors

ReplayStart (Point blk) 

Instances

Instances details
StandardHash blk ⇒ Eq (ReplayStart blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

Methods

(==)ReplayStart blk → ReplayStart blk → Bool Source #

(/=)ReplayStart blk → ReplayStart blk → Bool Source #

StandardHash blk ⇒ Show (ReplayStart blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

data TraceEvent blk Source #

Constructors

InvalidSnapshot DiskSnapshot (InitFailure blk)

An on disk snapshot was skipped because it was invalid.

TookSnapshot DiskSnapshot (RealPoint blk)

A snapshot was written to disk.

DeletedSnapshot DiskSnapshot

An old or invalid on-disk snapshot was deleted

Instances

Instances details
StandardHash blk ⇒ Eq (TraceEvent blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

Methods

(==)TraceEvent blk → TraceEvent blk → Bool Source #

(/=)TraceEvent blk → TraceEvent blk → Bool Source #

StandardHash blk ⇒ Show (TraceEvent blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

Methods

showsPrecIntTraceEvent blk → ShowS Source #

showTraceEvent blk → String Source #

showList ∷ [TraceEvent blk] → ShowS Source #

Generic (TraceEvent blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

Associated Types

type Rep (TraceEvent blk) ∷ TypeType Source #

Methods

fromTraceEvent blk → Rep (TraceEvent blk) x Source #

toRep (TraceEvent blk) x → TraceEvent blk Source #

type Rep (TraceEvent blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

data TraceReplayEvent blk Source #

Events traced while replaying blocks against the ledger to bring it up to date w.r.t. the tip of the ImmutableDB during initialisation. As this process takes a while, we trace events to inform higher layers of our progress.

Constructors

ReplayFromGenesis (ReplayGoal blk)

There were no LedgerDB snapshots on disk, so we're replaying all blocks starting from Genesis against the initial ledger.

ReplayFromSnapshot 

Fields

  • DiskSnapshot
     
  • (RealPoint blk)
     
  • (ReplayStart blk)

    the block at which this replay started

  • (ReplayGoal blk)

    the block at the tip of the ImmutableDB | We replayed the given block (reference) on the genesis snapshot during the initialisation of the LedgerDB. Used during ImmutableDB replay.

ReplayedBlock 

Fields

Instances

Instances details
(StandardHash blk, InspectLedger blk) ⇒ Eq (TraceReplayEvent blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

(StandardHash blk, InspectLedger blk) ⇒ Show (TraceReplayEvent blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

Generic (TraceReplayEvent blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

Associated Types

type Rep (TraceReplayEvent blk) ∷ TypeType Source #

type Rep (TraceReplayEvent blk) Source # 
Instance details

Defined in Ouroboros.Consensus.Storage.LedgerDB.OnDisk

decorateReplayTracerWithGoal Source #

Arguments

Point blk

Tip of the ImmutableDB

→ Tracer m (TraceReplayEvent blk) 
→ Tracer m (ReplayGoal blk → TraceReplayEvent blk) 

Add the tip of the Immutable DB to the trace event

Between the tip of the immutable DB and the point of the starting block, the node could (if it so desired) easily compute a "percentage complete".