Remove Integration Monad #6346

Jimbo4350 · 2025-10-21T17:42:45Z

Description

Remove Integration monad from functions that we also want to use outside of property tests.
Introduce annotated exceptions (via liftIOAnnotated) where the Integration monad has been removed in order to preserve debugging information.

┃ hprop_dump_config :: H.Property
    38 ┃ hprop_dump_config = integrationRetryWorkspace 2 "dump-config-files" $ \tmpDir -> H.runWithDefaultWatchdog_ $ do
       ┃ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       ┃ │ ━━━ Exception (AnnotatedException SomeException) ━━━
       ┃ │ ! AnnotatedException !
       ┃ │ Underlying exception type: IOException
       ┃ │ 
       ┃ │ unsupported operation (Operation is not supported)
       ┃ │ 
       ┃ │ CallStack (from HasCallStack):
       ┃ │   liftIOCallStack, called at src/Testnet/Components/Configuration.hs:186:24 in cardano-testnet-10.0.1-inplace:Testnet.Components.Configuration
       ┃ │   createSPOGenesisAndFiles, called at src/Testnet/Start/Cardano.hs:133:8 in cardano-testnet-10.0.1-inplace:Testnet.Start.Cardano
       ┃ │   createTestnetEnv, called at test/cardano-testnet-test/Cardano/Testnet/Test/DumpConfig.hs:48:23 in cardano-testnet-10.0.1-inplace-cardano-testnet-test:Cardano.Testnet.Test.DumpConfig
       ┃ │   testCase, called at src/Hedgehog/Extras/Test/TestWatchdog.hs:166:66 in hedgehog-extras-0.8.0.0-26dc8622927009e7df9ba3764930f3c64dfd8a2e158af1686ffeb231b1a5219b:Hedgehog.Extras.Test.TestWatchdog
       ┃ │   a given constraint, called at src/Hedgehog/Extras/Test/TestWatchdog.hs:159:26 in hedgehog-extras-0.8.0.0-26dc8622927009e7df9ba3764930f3c64dfd8a2e158af1686ffeb231b1a5219b:Hedgehog.Extras.Test.TestWatchdog
       ┃ │   testCase, called at src/Hedgehog/Extras/Test/TestWatchdog.hs:144:11 in hedgehog-extras-0.8.0.0-26dc8622927009e7df9ba3764930f3c64dfd8a2e158af1686ffeb231b1a5219b:Hedgehog.Extras.Test.TestWatchdog
       ┃ │   runWithWatchdog, called at src/Hedgehog/Extras/Test/TestWatchdog.hs:159:26 in hedgehog-extras-0.8.0.0-26dc8622927009e7df9ba3764930f3c64dfd8a2e158af1686ffeb231b1a5219b:Hedgehog.Extras.Test.TestWatchdog
       ┃ │   runWithDefaultWatchdog, called at src/Hedgehog/Extras/Test/TestWatchdog.hs:166:36 in hedgehog-extras-0.8.0.0-26dc8622927009e7df9ba3764930f3c64dfd8a2e158af1686ffeb231b1a5219b:Hedgehog.Extras.Test.TestWatchdog
       ┃ │   runWithDefaultWatchdog_, called at test/cardano-testnet-test/Cardano/Testnet/Test/DumpConfig.hs:38:82 in cardano-testnet-10.0.1-inplace-cardano-testnet-test:Cardano.Testnet.Test.DumpConfig

Checklist

Commit sequence broadly makes sense and commits have useful messages
New tests are added if needed and existing tests are updated. These may include:
- golden tests
- property tests
- roundtrip tests
- integration tests
  See Runnings tests for more details
Any changes are noted in the CHANGELOG.md for affected package
The version bounds in .cabal files are updated
CI passes. See note on CI. The following CI checks are required:
- Code is linted with hlint. See .github/workflows/check-hlint.yml to get the hlint version
- Code is formatted with stylish-haskell. See .github/workflows/stylish-haskell.yml to get the stylish-haskell version
- Code builds on Linux, MacOS and Windows for ghc-9.6 and ghc-9.12
Self-reviewed the diff

Note on CI

If your PR is from a fork, the necessary CI jobs won't trigger automatically for security reasons.
You will need to get someone with write privileges. Please contact IOG node developers to do this
for you.

cardano-testnet/src/Testnet/Orphans.hs

cardano-testnet/src/Testnet/Process/RunIO.hs

carbolymer · 2025-11-04T16:35:25Z

cardano-testnet/src/Testnet/Process/RunIO.hs

+
+liftIOAnnotated :: (HasCallStack, MonadIO m) => IO a -> m a 
+liftIOAnnotated action = GHC.withFrozenCallStack $
+  liftIO $ action `catch` (\(e :: SomeException) -> throwM $ exceptionWithCallStack e)


This is catch from Prelude right? So this should work with watchdog exceptions I think. Could you add a comment here that we're catching async exceptions here as well intentionally?

carbolymer · 2025-11-04T17:10:19Z

cardano-testnet/src/Testnet/Process/RunIO.hs

I'm not a fan of copying code downstream. Have you given a thought how we could reconcile your changes with hedgehog-extras, so possibly we could avoid the duplication?

My main issue with that is having to fix issues in both places, because we're using similar code in cardano-cli as well.

I'll open a PR.

carbolymer · 2025-11-04T17:22:34Z

cardano-testnet/src/Testnet/Runtime.hs

    -- The port number if it is obtained using 'H.randomPort', it is firstly bound to and then closed. The closing
    -- and release in the operating system is done asynchronously and can be slow. Here we wait until the port
-    -- is out of CLOSING state.
-    H.note_ $ "Waiting for port " <> show port <> " to be available before starting node"


This was a useful trace. I don't remember if we weren't experiencing hangups long time ago at this place. In general it would be nice to have a tracer available here so we could emit traces about the progress of the node startup.

cardano-testnet/src/Testnet/Runtime.hs

cardano-testnet/src/Testnet/Start/Cardano.hs

cardano-testnet/src/Testnet/Start/Types.hs

cardano-testnet/test/cardano-testnet-test/Cardano/Testnet/Test/SanityCheck.hs

carbolymer · 2025-11-05T12:05:27Z

cardano-testnet/test/cardano-testnet-test/Cardano/Testnet/Test/SanityCheck.hs

+        (_,asyncA) <- asyncRegister_ (threadDelay 10_000_000)
+        let tId = asyncThreadId asyncA
+        return (s,tId)
+     afterForkedThread <- getCurrentTime


FYI You can use https://hackage-content.haskell.org/package/hedgehog-extras-0.10.1.0/docs/Hedgehog-Extras-Test-TestWatchdog.html to verify if code paths were reached.

We use this function to mainly lift the two functions we want to use outside of a property testing context: - createTestnetEnv - cardanoTestnet

createEnvOptions This removes the unnecessary Integration monad

Add Testnet.Process.RunIO which exposes execCli functions without the MonadTest constraint

This differs from hedgehog-extras's asyncRegister function in that the thread is linked before it is cancelled.

cardanoTestnet

…teLogging

Removed handleException changes (testing)

Jimbo4350 force-pushed the jordan/remove-integration-monad branch 3 times, most recently from b99bb37 to 0bf6d79 Compare October 22, 2025 13:55

Jimbo4350 force-pushed the jordan/remove-integration-monad branch 19 times, most recently from 0f61855 to d7b60ca Compare November 3, 2025 15:26

Jimbo4350 marked this pull request as ready for review November 4, 2025 12:08

Jimbo4350 requested a review from a team as a code owner November 4, 2025 12:08

Jimbo4350 changed the title ~~Remove Integration Monad~~ Do not merge: Remove Integration Monad Nov 4, 2025

carbolymer reviewed Nov 4, 2025

View reviewed changes

cardano-testnet/src/Testnet/Orphans.hs Outdated Show resolved Hide resolved

carbolymer reviewed Nov 4, 2025

View reviewed changes

carbolymer requested changes Nov 4, 2025

View reviewed changes

cardano-testnet/src/Testnet/Runtime.hs Outdated Show resolved Hide resolved

carbolymer reviewed Nov 4, 2025

View reviewed changes

cardano-testnet/src/Testnet/Runtime.hs Show resolved Hide resolved

carbolymer reviewed Nov 5, 2025

View reviewed changes

Jimbo4350 force-pushed the jordan/remove-integration-monad branch 2 times, most recently from 2a6399b to 7853b82 Compare December 2, 2025 15:24

Jimbo4350 and others added 19 commits December 3, 2025 10:32

Implement liftToIntegration

617e4ad

We use this function to mainly lift the two functions we want to use outside of a property testing context: - createTestnetEnv - cardanoTestnet

Implement mkConfig and mkConfigAbs

e9e86e5

Remove runTestnet usage in runCardanoOptions and testnetRoutine from

9815306

createEnvOptions This removes the unnecessary Integration monad

Implement liftIOAnnotated which annotates exceptions with a callstack

353d7c7

Add Testnet.Process.RunIO which exposes execCli functions without the MonadTest constraint

Remove testnetRoutine

8ab1a90

Implement asyncRegister_

232a2d8

This differs from hedgehog-extras's asyncRegister function in that the thread is linked before it is cancelled.

Add hprop_asyncRegister_sanity_check

58257fb

Remove Integration monad return type from createTestnetEnv and

22edf8d

cardanoTestnet

Remove MonadTest constraint from createSPOGenesisAndFiles

296392f

Remove MonadTest constraint from startNode and startLedgerNewEpochSta…

c79c9a3

…teLogging

Remove MonadTest constraints from assertion functions

3dd6838

Propagate liftToIntegration and liftIOAnnotated

542d034

Fix build issues

082ed23

Review changes

3b60f2e

Replace liftIO with liftIOAnnotated

e1cf677

Fix infinite loop

7587cd3

Fix liftToIntegration to report the exception location to Hedgehog

ea904de

Rebase changes

450a650

Remove unnecessary function

8f27471

Jimbo4350 force-pushed the jordan/remove-integration-monad branch 2 times, most recently from eeedd74 to 20a7376 Compare December 3, 2025 21:32

Update createDirectoryIfMissingNew

25825b0

Jimbo4350 force-pushed the jordan/remove-integration-monad branch from 20a7376 to 25825b0 Compare December 3, 2025 21:38

Jimbo4350 added 2 commits December 4, 2025 10:23

Create log directory if it does not exist

28af3f0

Final fix

c2484e1

Jimbo4350 force-pushed the jordan/remove-integration-monad branch from 19dff12 to 6f63358 Compare December 4, 2025 14:30

Remove createDirectoryIfMissingNew changes (passes CI)

23a9869

Removed handleException changes (testing)

Jimbo4350 force-pushed the jordan/remove-integration-monad branch from 6f63358 to 23a9869 Compare December 4, 2025 15:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Remove Integration Monad #6346

Remove Integration Monad #6346

Jimbo4350 commented Oct 21, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

carbolymer Nov 4, 2025

Uh oh!

carbolymer Nov 4, 2025

Uh oh!

Jimbo4350 Nov 5, 2025

Uh oh!

carbolymer Nov 4, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

carbolymer Nov 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Remove Integration Monad #6346

Are you sure you want to change the base?

Remove Integration Monad #6346

Conversation

Jimbo4350 commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Note on CI

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

carbolymer Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

carbolymer Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

Jimbo4350 Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

carbolymer Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

carbolymer Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Jimbo4350 commented Oct 21, 2025 •

edited

Loading