Skip to content

Conversation

@stringintech
Copy link
Owner

@stringintech stringintech commented Dec 4, 2025

This PR introduces stateful test suites and adds a new suite for testing chain operations.

Key additions:

  • Stateful suite support: Introduces a stateful flag for test suites where tests depend on each other's success. If any test fails in a stateful suite, all subsequent tests are automatically skipped and marked as failed. Handler processes are respawned after stateful suites to prevent state leaks between suites.
  • Object reference protocol: Implements ref field and introduces registry pattern for persisting objects (contexts, blocks, etc.) across requests, enabling complex multi-step test scenarios.
  • Chain operations test suite: New chain.json demonstrates stateful testing with block processing, height tracking, and chain reorganization verification. The block data used in this suite can be deterministically regenerated by this functional test.
  • Method reference documentation: Restructures handler-spec.md with detailed method reference for all supported operations.
  • Verbose mode: Adds -v and -vv flags to print complete request chains that can be piped directly to a handler for manual debugging and reproduction - Add verbose modes for easier handler debugging #6.

@stringintech
Copy link
Owner Author

Before implementing the object reference pattern with the ref field, I initially considered this approach that relied on implicit global state in the handler.

Key differences:

Old approach (implicit state):

  • Chainstate manager created once, all subsequent tests assume it exists
  • Empty params: {} for operations on the global state
  • Multiple blocks processed in batches via arrays
{
  "id": "assert_chain_height_zero",
  "method": "btck_chain_get_height",
  "params": {},  // Assumes global chainstate exists
  "expected": {"success": true, "result": {"height": 0}}
}

New approach (explicit references):

  • Each object gets a named reference (e.g., $context_ref, $chainstate_manager_ref)
  • Tests explicitly pass which objects to operate on
  • More granular, one operation per test
{
  "id": "chain#5",
  "method": "btck_chain_get_height",
  "params": {"chain": "$chain_ref"},  // Explicit reference
  "expected": {"result": 0}
}

The reference pattern should provide more flexibility: handlers can manage multiple concurrent contexts/chainstates, dependencies between tests are explicit, and test scenarios are more composable.

Note on option objects: I intentionally avoid exposing option objects (like btck_ContextOptions, btck_ChainstateManagerOptions) as references in the protocol. Each language binding can implement options idiomatically (builder pattern in Rust, functional options in Go, etc.) without being constrained by the test protocol. Tests pass configuration directly in params rather than creating/manipulating option objects through the protocol.

@stringintech
Copy link
Owner Author

Tested v0.0.3-alpha.1 locally with go-bitcoinkernel and will push a commit soon.

Copy link
Contributor

@stickies-v stickies-v left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK. This seems like a really elegant way to chain (hah) operations together, and wrt explicitness: I agree that the explicit approach is much better than the implicit one.

Working on other things atm but will test this out as soon as I get the chance (and having #6 addressed would make testing a lot easier).

Without this check, the code would proceed to call Normalize() on a null result, causing an unclear failure that doesn't explain the actual problem.
Sort test suites alphabetically based on their json file names before execution.
Introduces stateful test suites where tests depend on each other and execute sequentially. If any test fails in a stateful suite, all subsequent tests are automatically skipped and marked as failed. Handler processes are closed after stateful suites to force re-spawn and prevent state leaks to the remaining tests.
@stringintech
Copy link
Owner Author

Rebased to include changes in #12 - ID no longer required in response. Also added support for the requested feature in #6.

Tested pre-release v0.0.3-alpha.2 in stringintech/go-bitcoinkernel#12 (comment).

@stringintech stringintech linked an issue Dec 8, 2025 that may be closed by this pull request
Introduces the object reference and registry pattern for persisting objects across requests. Test cases now include a ref field directly in the request when the expected result is a reference, allowing handlers to store and reuse objects like context, block, and chain throughout the tests.

Extends response validation unit tests to test scenarios where reference is expected in response.
Adds chain.json test suite demonstrating stateful chain operations including block processing, height tracking, and verifying state after a reorg scenario.

Documents the ref field and registry pattern in handler-spec.md.

Restructures handler-spec.md with a method reference documenting supported operations (context management, chainstate manager, chain operations, block operations, and script verification) with their parameters, return values, and error conditions.
Add -v and -vv flags to print complete request chains that can be piped directly to a handler for manual debugging and reproduction.

- Introduces DependencyTracker to build transitive dependency chains from ref usage
- Switches to pflag for better CLI flag support
- Adds unit tests for dependency tracking logic
@stringintech
Copy link
Owner Author

Addressed suggestion by @stickies-v in #11 (comment). Also addressed some nits and extended response validation unit tests to test scenarios where a ref object is required as result.

Tested pre-release v0.0.3-alpha.3 in stringintech/go-bitcoinkernel#12 (comment).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add verbose modes for easier handler debugging

4 participants