| Safe Haskell | Safe-Inferred |
|---|---|
| Language | Haskell2010 |
Convex.TestingInterface
Synopsis
- class (Show state, Eq state, Show (Action state), ToJSON state) => TestingInterface state where
- data Action state
- initialize :: MonadIO m => TestingMonadT m state
- arbitraryAction :: state -> Gen (Action state)
- precondition :: state -> Action state -> Bool
- perform :: MonadIO m => state -> Action state -> TestingMonadT m state
- validate :: MonadIO m => state -> TestingMonadT m Bool
- monitoring :: state -> Action state -> Property -> Property
- discardNegativeTestForUserExceptions :: Bool
- data ModelState state
- class TestingInterface state => ThreatModelsFor state where
- threatModels :: [ThreatModel ()]
- expectedVulnerabilities :: [ThreatModel ()]
- propRunActions :: forall state. ThreatModelsFor state => String -> TestTree
- propRunActionsWithOptions :: forall state. ThreatModelsFor state => String -> RunOptions -> TestTree
- data RunOptions = RunOptions {}
- defaultRunOptions :: RunOptions
- genAction :: (TestingInterface state, Monad m) => state -> PropertyM m (Maybe (Action state))
- runActions :: (TestingInterface state, MonadIO m) => RunOptions -> Int -> state -> TestingMonadT (PropertyM m) state
- data TraceRecorder = TraceRecorder {}
- newtype TestingMonadT m a = TestingMonadT {
- unTestingMonadT :: ExceptT (BalanceTxError ConwayEra) (MockchainT ConwayEra m) a
- runTestingMonadT :: NodeParams ConwayEra -> TestingMonadT m a -> m (Either (BalanceTxError ConwayEra) a, MockChainState ConwayEra)
- mockchainSucceedsWithOptions :: Options ConwayEra -> TestingMonadT IO a -> Assertion
- mockchainFailsWithOptions :: Options ConwayEra -> TestingMonadT IO a -> (BalanceTxError ConwayEra -> Assertion) -> Assertion
- data Options era = Options {
- params :: NodeParams era
- coverageRef :: Maybe (IORef CoverageData)
- defaultOptions :: Options ConwayEra
- modifyTransactionLimits :: Options ConwayEra -> Word32 -> Options ConwayEra
- withCoverage :: CoverageConfig -> (Options ConwayEra -> RunOptions -> IO ()) -> IO ()
- data CoverageConfig = CoverageConfig {
- coverageIndices :: [CoverageIndex]
- coverageReport :: CoverageReport -> IO ()
- printCoverageReport :: CoverageReport -> IO ()
- writeCoverageReport :: FilePath -> CoverageReport -> IO ()
- silentCoverageReport :: CoverageReport -> IO ()
- printCoverageJSON :: CoverageReport -> IO ()
- writeCoverageJSON :: FilePath -> CoverageReport -> IO ()
- printCoverageJSONPretty :: CoverageReport -> IO ()
- writeCoverageJSONPretty :: FilePath -> CoverageReport -> IO ()
- data CoverageSummary = CoverageSummary {
- csCovered :: [JsonCovered]
- csUncovered :: [JsonAnnotation]
- csIgnored :: [JsonAnnotation]
- coverageSummary :: CoverageReport -> CoverageSummary
- data Gen a
- class Arbitrary a where
- frequency :: HasCallStack => [(Int, Gen a)] -> Gen a
- oneof :: HasCallStack => [Gen a] -> Gen a
- elements :: HasCallStack => [a] -> Gen a
- data TestTree
Testing interface
class (Show state, Eq state, Show (Action state), ToJSON state) => TestingInterface state where Source #
A testing interface defines the state and behavior of one or more smart contracts.
The type parameter state represents the model's view of the world. It should
track all relevant information needed to validate that the contract is behaving
correctly.
Minimal complete definition: Action, initialize, arbitraryAction, perform
Minimal complete definition
Associated Types
Actions that can be performed on the contract. This is typically a data type with one constructor per contract operation.
Methods
initialize :: MonadIO m => TestingMonadT m state Source #
The initial state of the model, before any actions are performed.
arbitraryAction :: state -> Gen (Action state) Source #
Generate a random action given the current state. The generated action should be appropriate for the current state.
precondition :: state -> Action state -> Bool Source #
Precondition that must hold before an action can be executed.
Return False to indicate that an action is not valid in the current state.
Default: all actions are always valid.
perform :: MonadIO m => state -> Action state -> TestingMonadT m state Source #
Perform the action on the real blockchain (mockchain). This should execute the actual transaction(s) that implement the action. The current model state is provided to allow access to tracked blockchain state. The returned state should reflect the expected effect of the action on the contract state.
validate :: MonadIO m => state -> TestingMonadT m Bool Source #
Validate that the blockchain state matches the model state. Default: no validation (always succeeds).
monitoring :: state -> Action state -> Property -> Property Source #
Called after each successful action to wrap the enclosing QuickCheck property.
This hook runs only after perform and validate succeed. Use it for
property-level checks, labels, and counterexamples that should be attached to
valid state transitions.
Default: no additional checks.
discardNegativeTestForUserExceptions :: Bool Source #
Whether to discard (skip) test cases where the invalid action fails due to a user-level error (e.g., off-chain balancing failure) rather than an on-chain validator rejection during negative testing.
When True, negative tests that throw user exceptions are discarded
(via QuickCheck's discard), so only on-chain rejections count as
successful negative tests.
When False (the default), user exceptions also cause the test case
to be discarded — meaning both off-chain and on-chain failures are
treated the same way.
Override this in your TestingInterface instance if you need finer
control over which failure modes are accepted in negative testing.
data ModelState state Source #
Opaque wrapper for model state
Instances
| Show state => Show (ModelState state) Source # | |
Defined in Convex.TestingInterface Methods showsPrec :: Int -> ModelState state -> ShowS # show :: ModelState state -> String # showList :: [ModelState state] -> ShowS # | |
| Eq state => Eq (ModelState state) Source # | |
Defined in Convex.TestingInterface Methods (==) :: ModelState state -> ModelState state -> Bool # (/=) :: ModelState state -> ModelState state -> Bool # | |
class TestingInterface state => ThreatModelsFor state where Source #
Minimal complete definition
Nothing
Methods
threatModels :: [ThreatModel ()] Source #
Threat models to run against the transactions. Each threat model will be evaluated against the transaction generated by a succesful test run with the UTxO state captured before each transaction executed. Default: the list of all threat models that don't take parameters.
expectedVulnerabilities :: [ThreatModel ()] Source #
Threat models that are expected to find vulnerabilities.
These are run like threatModels but with inverted pass/fail semantics:
- OK when a vulnerability IS detected
- FAIL when a vulnerability is NOT detected
Output is quiet — no verbose transaction dumps. Default: empty, backward compatible.
Running Tests
propRunActions :: forall state. ThreatModelsFor state => String -> TestTree Source #
Main property for testing a testing interface. Generates random action sequences and checks that the implementation matches the model.
propRunActionsWithOptions :: forall state. ThreatModelsFor state => String -> RunOptions -> TestTree Source #
Run testing interface tests with custom options
data RunOptions Source #
Options for running property tests
Constructors
| RunOptions | |
Fields
| |
genAction :: (TestingInterface state, Monad m) => state -> PropertyM m (Maybe (Action state)) Source #
Generate a valid actions
runActions :: (TestingInterface state, MonadIO m) => RunOptions -> Int -> state -> TestingMonadT (PropertyM m) state Source #
Generate a number of actions (with a given maximum) and run them.
Trace recording
data TraceRecorder Source #
Callback for recording iteration traces as pre-serialized JSON. Arguments: group name, category ("positive"/"negative"), pre-serialized trace JSON. Default is a no-op (zero overhead when streaming is not active).
When trEnabled returns True, test bodies use the expensive traced code
path (building IterationTrace values with UTxO snapshots, transaction
summaries, and JSON serialisation). When it returns False (the IsOption
default), the cheap runActions path is used instead, avoiding all that
work.
trEnabled is an IO action so that the decision can be deferred until the
streaming reporter has parsed --no-trace and written the shared IORef.
Constructors
| TraceRecorder | |
Instances
| IsOption TraceRecorder | |
Defined in Convex.Tasty.Streaming.TMSummary Methods defaultValue :: TraceRecorder Source # parseValue :: String -> Maybe TraceRecorder Source # optionName :: Tagged TraceRecorder String Source # optionHelp :: Tagged TraceRecorder String Source # | |
The Testing Monad
newtype TestingMonadT m a Source #
Tests run in the mockchain monad extended with balancing error handling.
Leaving handling of balancing errors to the testing interface is important because the errors can contain data for code coverage.
Constructors
| TestingMonadT | |
Fields
| |
Instances
runTestingMonadT :: NodeParams ConwayEra -> TestingMonadT m a -> m (Either (BalanceTxError ConwayEra) a, MockChainState ConwayEra) Source #
mockchainSucceedsWithOptions :: Options ConwayEra -> TestingMonadT IO a -> Assertion Source #
Run the TestingMonadT action with the given options and fail if there is an error
mockchainFailsWithOptions :: Options ConwayEra -> TestingMonadT IO a -> (BalanceTxError ConwayEra -> Assertion) -> Assertion Source #
Run the TestingMonadT action with the given options, fail if it
succeeds, and handle the error appropriately.
Options for running the testing monad.
Constructors
| Options | |
Fields
| |
modifyTransactionLimits :: Options ConwayEra -> Word32 -> Options ConwayEra Source #
Modify the maximum transaction size in the protocol parameters of the given options
Coverage helpers
withCoverage :: CoverageConfig -> (Options ConwayEra -> RunOptions -> IO ()) -> IO () Source #
Run a test suite with Plutus script coverage collection.
Creates the coverage IORef, wires it into Options and RunOptions,
runs the user's action, and on exit produces a CoverageReport from the
accumulated data.
The report is generated when the inner action throws an ExitCode exception
(which is how tasty's defaultMain signals completion). The
original exception is re-thrown after the report action runs.
main :: IO ()
main = withCoverage config $ \opts runOpts ->
defaultMain $ testGroup "my tests"
[ testCase "t1" (mockchainSucceedsWithOptions opts myTest)
, myPropertyTests runOpts
]
where
config = CoverageConfig
{ coverageIndices = [myScriptCovIdx]
, coverageReport = printCoverageReport
}
data CoverageConfig Source #
Configuration for coverage collection and reporting.
Use with withCoverage to set up coverage tracking for your test suite.
Constructors
| CoverageConfig | |
Fields
| |
printCoverageReport :: CoverageReport -> IO () Source #
Print a coverage report to stdout using prettyprinter.
writeCoverageReport :: FilePath -> CoverageReport -> IO () Source #
Write a coverage report to a file.
silentCoverageReport :: CoverageReport -> IO () Source #
Collect coverage data but discard the report.
printCoverageJSON :: CoverageReport -> IO () Source #
Print a coverage report as compact JSON to stdout.
writeCoverageJSON :: FilePath -> CoverageReport -> IO () Source #
Write a coverage report as compact JSON to a file.
printCoverageJSONPretty :: CoverageReport -> IO () Source #
Print a coverage report as pretty-printed JSON to stdout.
writeCoverageJSONPretty :: FilePath -> CoverageReport -> IO () Source #
Write a coverage report as pretty-printed JSON to a file.
data CoverageSummary Source #
Minimal coverage summary matching what Pretty.pretty shows.
Constructors
| CoverageSummary | |
Fields
| |
Instances
| ToJSON CoverageSummary Source # | |
Defined in Convex.TestingInterface Methods toJSON :: CoverageSummary -> Value Source # toEncoding :: CoverageSummary -> Encoding Source # toJSONList :: [CoverageSummary] -> Value Source # toEncodingList :: [CoverageSummary] -> Encoding Source # omitField :: CoverageSummary -> Bool Source # | |
| Generic CoverageSummary Source # | |
Defined in Convex.TestingInterface Associated Types type Rep CoverageSummary :: Type -> Type # Methods from :: CoverageSummary -> Rep CoverageSummary x # to :: Rep CoverageSummary x -> CoverageSummary # | |
| type Rep CoverageSummary Source # | |
Defined in Convex.TestingInterface | |
coverageSummary :: CoverageReport -> CoverageSummary Source #
Convert a CoverageReport to a compact summary (same info as Pretty.pretty shows).
Re-exports from QuickCheck
A generator for values of type a.
The third-party packages
QuickCheck-GenT
and
quickcheck-transformer
provide monad transformer versions of Gen.
class Arbitrary a where Source #
Random generation and shrinking of values.
QuickCheck provides Arbitrary instances for most types in base,
except those which incur extra dependencies.
For a wider range of Arbitrary instances see the
quickcheck-instances
package.
Minimal complete definition
Methods
A generator for values of the given type.
It is worth spending time thinking about what sort of test data
you want - good generators are often the difference between
finding bugs and not finding them. You can use sample,
label and classify to check the quality of your test data.
There is no generic arbitrary implementation included because we don't
know how to make a high-quality one. If you want one, consider using the
testing-feat or
generic-random packages.
The QuickCheck manual goes into detail on how to write good generators. Make sure to look at it, especially if your type is recursive!
Produces a (possibly) empty list of all the possible immediate shrinks of the given value.
The default implementation returns the empty list, so will not try to
shrink the value. If your data type has no special invariants, you can
enable shrinking by defining shrink = , but by customising
the behaviour of genericShrinkshrink you can often get simpler counterexamples.
Most implementations of shrink should try at least three things:
- Shrink a term to any of its immediate subterms.
You can use
subtermsto do this. - Recursively apply
shrinkto all immediate subterms. You can userecursivelyShrinkto do this. - Type-specific shrinkings such as replacing a constructor by a simpler constructor.
For example, suppose we have the following implementation of binary trees:
data Tree a = Nil | Branch a (Tree a) (Tree a)
We can then define shrink as follows:
shrink Nil = [] shrink (Branch x l r) = -- shrink Branch to Nil [Nil] ++ -- shrink to subterms [l, r] ++ -- recursively shrink subterms [Branch x' l' r' | (x', l', r') <- shrink (x, l, r)]
There are a couple of subtleties here:
- QuickCheck tries the shrinking candidates in the order they
appear in the list, so we put more aggressive shrinking steps
(such as replacing the whole tree by
Nil) before smaller ones (such as recursively shrinking the subtrees). - It is tempting to write the last line as
[Branch x' l' r' | x' <- shrink x, l' <- shrink l, r' <- shrink r]but this is the wrong thing! It will force QuickCheck to shrinkx,landrin tandem, and shrinking will stop once one of the three is fully shrunk.
There is a fair bit of boilerplate in the code above.
We can avoid it with the help of some generic functions.
The function genericShrink tries shrinking a term to all of its
subterms and, failing that, recursively shrinks the subterms.
Using it, we can define shrink as:
shrink x = shrinkToNil x ++ genericShrink x
where
shrinkToNil Nil = []
shrinkToNil (Branch _ l r) = [Nil]genericShrink is a combination of subterms, which shrinks
a term to any of its subterms, and recursivelyShrink, which shrinks
all subterms of a term. These may be useful if you need a bit more
control over shrinking than genericShrink gives you.
A final gotcha: we cannot define shrink as simply
as this shrinks shrink x = Nil:genericShrink xNil to Nil, and shrinking will go into an
infinite loop.
If all this leaves you bewildered, you might try to begin with,
after deriving shrink = genericShrinkGeneric for your type. However, if your data type has any
special invariants, you will need to check that genericShrink can't break those invariants.
Instances
frequency :: HasCallStack => [(Int, Gen a)] -> Gen a Source #
Chooses one of the given generators, with a weighted random distribution. The input list must be non-empty.
oneof :: HasCallStack => [Gen a] -> Gen a Source #
Randomly uses one of the given generators. The input list must be non-empty.
elements :: HasCallStack => [a] -> Gen a Source #
Generates one of the given values. The input list must be non-empty.
Re-exports from Tasty
The main data structure defining a test suite.
It consists of individual test cases and properties, organized in named groups which form a tree-like hierarchy.
There is no generic way to create a test case. Instead, every test
provider (tasty-hunit, tasty-smallcheck etc.) provides a function to
turn a test case into a TestTree.
Groups can be created using testGroup.
Since: tasty-0.1