Understanding and debugging Nimbus EVM JSON tests

Introduction

The Nimbus JSON tests are taken from the official Ethereum test suite. To find out more about the contents of this test suite, please refer to the official documentation.

You can run all tests with nimble test. Based on our config, Nimble will write binaries to build/ - you can do this manually also, as in the following examples:

Run example:

mkdir -p build
nim c -o:build/decompile_smart_contract -r examples/decompile_smart_contract.nim

Run Ethereum JSON-based general state tests:

mkdir -p build
nim c -o:build/test_generalstate_json -r tests/test_generalstate_json.nim

Build and run an individual test:

nim c -o:build/test_generalstate_json -r tests/test_generalstate_json.nim tests/fixtures/GeneralStateTests/stAttackTest/ContractCreationSpam.json

VMTests

JSON structure

A JSON EVM test has the following structure (example is add0.json).

{
    "add0" : {
        "_info" : {
            "comment" : "",
            "filledwith" : "cpp-1.3.0+commit.6e0ce939.Linux.g++",
            "lllcversion" : "Version: 0.4.18-develop.2017.9.25+commit.a72237f2.Linux.g++",
            "source" : "src/VMTestsFiller/vmArithmeticTest/add0Filler.json",
            "sourceHash" : "dcc7fc8aebdc2d7334440cfe6c63172941b4164c1ba8c32897318ca0cdfb7a1c"
        },
        "callcreates" : [
        ],
        "env" : {
            "currentCoinbase" : "0x2adc25665018aa1fe0e6bc666dac8fc2697ff9ba",
            "currentDifficulty" : "0x0100",
            "currentGasLimit" : "0x0f4240",
            "currentNumber" : "0x00",
            "currentTimestamp" : "0x01"
        },
        "exec" : {
            "address" : "0x0f572e5295c57f15886f9b263e2f6d2d6c7b5ec6",
            "caller" : "0xcd1722f2947def4cf144679da39c4c32bdc35681",
            "code" : "0x7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01600055",
            "data" : "0x",
            "gas" : "0x0186a0",
            "gasPrice" : "0x5af3107a4000",
            "origin" : "0xcd1722f2947def4cf144679da39c4c32bdc35681",
            "value" : "0x0de0b6b3a7640000"
        },
        "gas" : "0x013874",
        "logs" : "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
        "out" : "0x",
        "post" : {
            "0x0f572e5295c57f15886f9b263e2f6d2d6c7b5ec6" : {
                "balance" : "0x0de0b6b3a7640000",
                "code" : "0x7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01600055",
                "nonce" : "0x00",
                "storage" : {
                    "0x00" : "0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe"
                }
            }
        },
        "pre" : {
            "0x0f572e5295c57f15886f9b263e2f6d2d6c7b5ec6" : {
                "balance" : "0x0de0b6b3a7640000",
                "code" : "0x7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01600055",
                "nonce" : "0x00",
                "storage" : {
                }
            }
        }
    }
}

Important: As of July 2018, VMTests always use Homestead fork for opcode implementations (notably CALL) and gas prices. We should not rely on currentNumber.

The important fields are:

• exec section:

  • code: the code being executed in isolation. (In production this is part of a transaction).
  • gas: usually 0x0186a0 (100000) the starting gas

• gas field: the remaining gas after code execution. Here 0x013874 (79988)

• pre section: the state before execution of the code by the VM

  • storage: if the account already stores data that we can retrieve with SLOAD or overwrite/reset with SSTORE

• post section: the state after execution of the code by the VM. If there is no post section, the code is supposed to throw an EVM exception. After an EVM exception, gas is consumed but state is reverted.

  Implementation-wise computation.isError returns true and an error message is available in computation.error field (implementation as of July 2018)

Decompiling bytecode.

You can decompile EVM bytecode using Etherscan: https://etherscan.io/opcode-tool Pasting 0x6003600202600055 gives:

[1] PUSH1 0x03 
[3] PUSH1 0x02 
[4] MUL 
[6] PUSH1 0x00 
[7] SSTORE 

Alternatively you can use the following:

import ../nimbus/vm/code_stream, strformat

var c = newCodeStreamFromUnescaped("0x6003600202600055")

let opcodes = c.decompile()
for op in opcodes:
  echo &"[{op[0]}]\t{op[1]}\t{op[2]}"

# [1]     PUSH1   0x03
# [3]     PUSH1   0x02
# [4]     MUL
# [6]     PUSH1   0x00
# [7]     SSTORE
# [-1]    STOP

The number in bracket refers to the position after reading the opcode and its arguments (i.e. the value of the program counter).

add0 (0x7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff01600055) decompiles to:

[32] PUSH32 0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff 
[65] PUSH32 0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff 
[66] ADD 
[68] PUSH1 0x00 
[69] SSTORE