-
Notifications
You must be signed in to change notification settings - Fork 3
Add support for stateful test suites and add chain tests #11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
Before implementing the object reference pattern with the Key differences: Old approach (implicit state):
{
"id": "assert_chain_height_zero",
"method": "btck_chain_get_height",
"params": {}, // Assumes global chainstate exists
"expected": {"success": true, "result": {"height": 0}}
}New approach (explicit references):
{
"id": "chain#5",
"method": "btck_chain_get_height",
"params": {"chain": "$chain_ref"}, // Explicit reference
"expected": {"result": 0}
}The reference pattern should provide more flexibility: handlers can manage multiple concurrent contexts/chainstates, dependencies between tests are explicit, and test scenarios are more composable. Note on option objects: I intentionally avoid exposing option objects (like |
|
Tested |
stickies-v
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Concept ACK. This seems like a really elegant way to chain (hah) operations together, and wrt explicitness: I agree that the explicit approach is much better than the implicit one.
Working on other things atm but will test this out as soon as I get the chance (and having #6 addressed would make testing a lot easier).
Without this check, the code would proceed to call Normalize() on a null result, causing an unclear failure that doesn't explain the actual problem.
Sort test suites alphabetically based on their json file names before execution.
Introduces stateful test suites where tests depend on each other and execute sequentially. If any test fails in a stateful suite, all subsequent tests are automatically skipped and marked as failed. Handler processes are closed after stateful suites to force re-spawn and prevent state leaks to the remaining tests.
ad723e8 to
39bc313
Compare
|
Rebased to include changes in #12 - ID no longer required in response. Also added support for the requested feature in #6. Tested pre-release |
Introduces the object reference and registry pattern for persisting objects across requests. Test cases now include a ref field directly in the request when the expected result is a reference, allowing handlers to store and reuse objects like context, block, and chain throughout the tests. Extends response validation unit tests to test scenarios where reference is expected in response.
Adds chain.json test suite demonstrating stateful chain operations including block processing, height tracking, and verifying state after a reorg scenario. Documents the ref field and registry pattern in handler-spec.md. Restructures handler-spec.md with a method reference documenting supported operations (context management, chainstate manager, chain operations, block operations, and script verification) with their parameters, return values, and error conditions.
Add -v and -vv flags to print complete request chains that can be piped directly to a handler for manual debugging and reproduction. - Introduces DependencyTracker to build transitive dependency chains from ref usage - Switches to pflag for better CLI flag support - Adds unit tests for dependency tracking logic
39bc313 to
9558afd
Compare
|
Addressed suggestion by @stickies-v in #11 (comment). Also addressed some nits and extended response validation unit tests to test scenarios where a ref object is required as result. Tested pre-release |
This PR introduces stateful test suites and adds a new suite for testing chain operations.
Key additions:
chain.jsondemonstrates stateful testing with block processing, height tracking, and chain reorganization verification. The block data used in this suite can be deterministically regenerated by this functional test.handler-spec.mdwith detailed method reference for all supported operations.-vand-vvflags to print complete request chains that can be piped directly to a handler for manual debugging and reproduction - Add verbose modes for easier handler debugging #6.