Make Tesseract compatible with Sinter #88

draganaurosgrbic · 2025-08-11T12:11:34Z

Address issue #55

oscarhiggott

Thanks for this. It's important to add some python tests that test that this class works with sinter, for example see here for examples: https://github.com/quantumlib/Stim/blob/main/glue/sample/src/sinter/_decoding/_decoding_test.py.

src/tesseract_sinter_compat.pybind.cc

src/BUILD

…rary, bind tesseract decoding config inside Sinter API

draganaurosgrbic · 2025-08-11T23:04:40Z

Thanks for this. It's important to add some python tests that test that this class works with sinter, for example see here for examples: https://github.com/quantumlib/Stim/blob/main/glue/sample/src/sinter/_decoding/_decoding_test.py.

Thank you. Done.

oscarhiggott

This is looking very nice, but one current issue is that the sinter decoder is not pickleable. E.g. if I run:

from tesseract_decoder.tesseract_sinter_compat import TesseractSinterDecoder

import stim
import sinter
import matplotlib.pyplot as plt


# Generates surface code circuit tasks using Stim's circuit generation.
def generate_example_tasks():
    for p in [0.001]:
        for d in [3]:
            yield sinter.Task(
                circuit=stim.Circuit.generated(
                    rounds=2,
                    distance=d,
                    before_round_data_depolarization=p,
                    code_task=f'surface_code:rotated_memory_x',
                ),
                json_metadata={
                    'p': p,
                    'd': d,
                },
            )


def main():
    # Collect the samples (takes a few minutes).
    samples = sinter.collect(
        num_workers=4,
        max_shots=1_000_000,
        max_errors=1000,
        tasks=generate_example_tasks(),
        custom_decoders={"tesseract": TesseractSinterDecoder()},
        decoders=['tesseract'],
    )

    # Print samples as CSV data.
    print(sinter.CSV_HEADER)
    for sample in samples:
        print(sample.to_csv_line())

    # Render a matplotlib plot of the data.
    fig, ax = plt.subplots(1, 1)
    sinter.plot_error_rate(
        ax=ax,
        stats=samples,
        group_func=lambda stat: f"Rotated Surface Code d={stat.json_metadata['d']}",
        x_func=lambda stat: stat.json_metadata['p'],
    )
    ax.loglog()
    ax.set_ylim(1e-5, 1)
    ax.grid()
    ax.set_title('Logical Error Rate vs Physical Error Rate')
    ax.set_ylabel('Logical Error Probability (per shot)')
    ax.set_xlabel('Physical Error Rate')
    ax.legend()

    # Save to file and also open in a window.
    fig.savefig('plot.png')
    plt.show()


# NOTE: This is actually necessary! If the code inside 'main()' was at the
# module level, the multiprocessing children spawned by sinter.collect would
# also attempt to run that code.
if __name__ == '__main__':
    main()

I get:

Traceback (most recent call last):
  File "/usr/local/google/home/oscarhiggott/.virtualenvs/tesseract/lib/python3.12/site-packages/sinter/_collection/_collection.py", line 222, in iter_collect
    manager.start_workers()
  File "/usr/local/google/home/oscarhiggott/.virtualenvs/tesseract/lib/python3.12/site-packages/sinter/_collection/_collection_manager.py", line 183, in start_workers
    worker_state.process.start()
  File "/usr/lib/python3.12/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
                  ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/multiprocessing/context.py", line 289, in _Popen
    return Popen(process_obj)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/usr/lib/python3.12/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/usr/lib/python3.12/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/usr/lib/python3.12/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle 'tesseract_decoder.tesseract_sinter_compat.TesseractSinterDecoder' object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/google/home/oscarhiggott/Documents/software/tesseract_examples/sinter_testing.py", line 67, in <module>
    main()
  File "/usr/local/google/home/oscarhiggott/Documents/software/tesseract_examples/sinter_testing.py", line 28, in main
    samples = sinter.collect(
              ^^^^^^^^^^^^^^^
  File "/usr/local/google/home/oscarhiggott/.virtualenvs/tesseract/lib/python3.12/site-packages/sinter/_collection/_collection.py", line 404, in collect
    for progress in iter_collect(
                    ^^^^^^^^^^^^^
  File "/usr/local/google/home/oscarhiggott/.virtualenvs/tesseract/lib/python3.12/site-packages/sinter/_collection/_collection.py", line 198, in iter_collect
    with CollectionManager(
         ^^^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/oscarhiggott/.virtualenvs/tesseract/lib/python3.12/site-packages/sinter/_collection/_collection_manager.py", line 124, in __exit__
    self.hard_stop()
  File "/usr/local/google/home/oscarhiggott/.virtualenvs/tesseract/lib/python3.12/site-packages/sinter/_collection/_collection_manager.py", line 269, in hard_stop
    w.kill()
  File "/usr/lib/python3.12/multiprocessing/process.py", line 140, in kill
    self._popen.kill()
    ^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'kill'

This should be fixable using the pybind11 pickling support.

It would also be nice if we add an example such as the above (once it's working) into the documentation or readme somewhere (maybe the python readme).

draganaurosgrbic · 2025-08-12T21:08:33Z

@oscarhiggott Thank you so much for the serialization reference! I have updated the code and the Sinter Integration decoder is now also serializable. I also added a section about the Sinter Integration inside the README.md for the Python interface.

oscarhiggott · 2025-08-13T19:39:43Z

If I try to run an example in the python readme:

import sinter
import stim
from sinter._decoding._decoding import sample_decode

from tesseract_decoder import tesseract_sinter_compat as tesseract_module
import tesseract_decoder

# Define a function that returns a dictionary mapping a decoder name to its
# Sinter-compatible decoder object.
def get_tesseract_decoder_for_sinter():
    return {"tesseract": tesseract_module.TesseractSinterDecoder()}

# Create a repetition code circuit to test the decoder.
circuit = stim.Circuit.generated(
    'repetition_code:memory',
    distance=3,
    rounds=3,
    after_clifford_depolarization=0.01
)

# Use sinter.collect to run the decoding task.
results, = sinter.collect(
    num_workers=1,
    tasks=[sinter.Task(circuit=circuit)],
    decoders=["tesseract"],
    max_shots=1000,
    custom_decoders=get_tesseract_decoder_for_sinter(),
)

# Print a summary of the decoding results.
print("Basic Repetition Code Decoding Results:")
print(f"Shots run: {results.shots}")
print(f"Observed errors: {results.errors}")
print(f"Logical error rate: {results.errors / results.shots}")

I get this exception:

Traceback (most recent call last):
  File "/usr/local/google/home/oscarhiggott/.virtualenvs/tesseract/lib/python3.12/site-packages/sinter/_collection/_collection.py", line 222, in iter_collect
    manager.start_workers()
  File "/usr/local/google/home/oscarhiggott/.virtualenvs/tesseract/lib/python3.12/site-packages/sinter/_collection/_collection_manager.py", line 183, in start_workers
    worker_state.process.start()
  File "/usr/lib/python3.12/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
                  ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/multiprocessing/context.py", line 289, in _Popen
    return Popen(process_obj)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/usr/lib/python3.12/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/usr/lib/python3.12/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/usr/lib/python3.12/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle 'tesseract_decoder.tesseract_sinter_compat.TesseractSinterDecoder' object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/google/home/oscarhiggott/Documents/software/tesseract_examples/sinter_testing_2.py", line 22, in <module>
    results, = sinter.collect(
               ^^^^^^^^^^^^^^^
  File "/usr/local/google/home/oscarhiggott/.virtualenvs/tesseract/lib/python3.12/site-packages/sinter/_collection/_collection.py", line 404, in collect
    for progress in iter_collect(
                    ^^^^^^^^^^^^^
  File "/usr/local/google/home/oscarhiggott/.virtualenvs/tesseract/lib/python3.12/site-packages/sinter/_collection/_collection.py", line 198, in iter_collect
    with CollectionManager(
         ^^^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/oscarhiggott/.virtualenvs/tesseract/lib/python3.12/site-packages/sinter/_collection/_collection_manager.py", line 124, in __exit__
    self.hard_stop()
  File "/usr/local/google/home/oscarhiggott/.virtualenvs/tesseract/lib/python3.12/site-packages/sinter/_collection/_collection_manager.py", line 269, in hard_stop
    w.kill()
  File "/usr/lib/python3.12/multiprocessing/process.py", line 140, in kill
    self._popen.kill()
    ^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'kill'

Any ideas why?

I installed the package from this branch by first changing DEFAULT_PYTHON_VERSION = "3.11" to my python version (3.12) in MODULE.bazel and then running:

bazel build --define TARGET_VERSION="py312" --define VERSION="0.1.1"  :tesseract_decoder_wheel

and then:

pip install --force-reinstall bazel-bin/tesseract_decoder-0.1.1-py312-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

src/py/README.md

draganaurosgrbic · 2025-08-13T19:47:12Z

@oscarhiggott I had added that exactly same test that you told me that fails in the CI. It passed when I ran it. Do you have an idea why it failed when you run it?? Did you pull my latest changes? You can wait for the CI to complete and see if that test fails in the CI.

draganaurosgrbic · 2025-08-13T19:49:48Z

@oscarhiggott the test passes in the CI. As I mentioned before, you need to run tests with bazel test src/...

oscarhiggott · 2025-08-13T20:59:26Z

@oscarhiggott I had added that exactly same test that you told me that fails in the CI. It passed when I ran it. Do you have an idea why it failed when you run it?? Did you pull my latest changes? You can wait for the CI to complete and see if that test fails in the CI.

Hi @draganaurosgrbic, I'm not sure what the bug is, I haven't had the chance to take a look into it. But the tests pass fine. There's something that the tests don't catch that still causes sinter not to work. I would recommend trying to reproduce the issue I found using the method I gave to install the wheel. The tests pass for me too (using the latest version) but the sinter does not work with the code snippet I showed.

oscarhiggott · 2025-08-13T21:01:30Z

Though actually running the same code with pymatching also fails so maybe there's a bug in the example itself

draganaurosgrbic · 2025-08-13T21:08:44Z

Though actually running the same code with pymatching also fails so maybe there's a bug in the example itself

Hmm, let me take a look at the example then

oscarhiggott · 2025-08-13T21:10:15Z

Ah I realize what it is, I hadn't put the code in an if __name__ == "__main__" block, but that's necessary here.

This works for me:

import sinter
import stim
from sinter._decoding._decoding import sample_decode

from tesseract_decoder import tesseract_sinter_compat as tesseract_module
import tesseract_decoder

# Define a function that returns a dictionary mapping a decoder name to its
# Sinter-compatible decoder object.
def get_tesseract_decoder_for_sinter():
    return {"tesseract": tesseract_module.TesseractSinterDecoder()}


if __name__ == "__main__":
    # Create a repetition code circuit to test the decoder.
    circuit = stim.Circuit.generated(
        "color_code:memory_xyz",
        distance=3,
        rounds=3,
        after_clifford_depolarization=0.002
    )

    # Use sinter.collect to run the decoding task.
    results, = sinter.collect(
        num_workers=1,
        tasks=[sinter.Task(circuit=circuit)],
        decoders=["tesseract"],
        max_shots=1000,
        custom_decoders=get_tesseract_decoder_for_sinter(),
    )

    # Print a summary of the decoding results.
    print("Basic Repetition Code Decoding Results:")
    print(f"Shots run: {results.shots}")
    print(f"Observed errors: {results.errors}")
    print(f"Logical error rate: {results.errors / results.shots}")

Works very nicely, TYSM!

oscarhiggott · 2025-08-13T21:10:42Z

Would be good if the examples were updated to work (be in if __name__ == "__main__"). Would also be nice if the python readme was just directly in the top level docs directory (e.g. docs/python_documentation.md or similar).

draganaurosgrbic · 2025-08-13T21:12:21Z

Would be good if the examples were updated to work (be in if __name__ == "__main__"). Would also be nice if the python readme was just directly in the top level docs directory (e.g. python_documentation.md or similar)

I didn't make a decision where to place the Python documentation, but I can move it, if you find it more appropriate

oscarhiggott · 2025-08-13T21:14:50Z

Would be good if the examples were updated to work (be in if __name__ == "__main__"). Would also be nice if the python readme was just directly in the top level docs directory (e.g. python_documentation.md or similar)

I didn't make a decision where to place the Python documentation, but I can move it, if you find it more appropriate

I do think having it in docs as docs/python_documentation.md or similar would be best as users will look for documentation in the docs folder first

draganaurosgrbic · 2025-08-13T21:15:28Z

Would be good if the examples were updated to work (be in if __name__ == "__main__"). Would also be nice if the python readme was just directly in the top level docs directory (e.g. python_documentation.md or similar)

I didn't make a decision where to place the Python documentation, but I can move it, if you find it more appropriate

I do think having it in docs as docs/python_documentation.md or similar would be best as users will look for documentation in the docs folder first

Do you want me to do it in this PR? Or a separate? I am fine with both

oscarhiggott · 2025-08-13T21:18:02Z

Good point another PR is best to keep the diff small, can wait until this is merged. I'll take another look through this PR now.

draganaurosgrbic · 2025-08-13T21:18:54Z

Good point another PR is best to keep the diff small, can wait until this is merged. I'll take another look through this PR now.

Thank you so much!!!

oscarhiggott · 2025-08-13T22:03:18Z

One nice thing to have would be a test along these lines (this is a sketch, would need to replace config_arg1 etc). Do you think that's doable in this PR or shall we create an issue for it?


@pytest.mark.parametrize("config_arg_1,config_arg_2, etc", [("ex1_arg1", "ex1_arg2"), ("ex2_arg1", "ex2_arg2")])
def test_decode_shots_bit_packed_vs_decode_batch(config_arg_1, config_arg2, etc):
    circuit = stim.Circuit.generated(
        "color_code:memory_xyz",
        distance=3,
        rounds=3,
        after_clifford_depolarization=0.02
    )
    dem = circuit.detector_error_model()
    
    config = tesseract_decoder.tesseract.TesseractConfig(config_arg_1, config_arg2)

    sinter_decoder = tesseract_module.TesseractSinterDecoder(config=config)
    compiled_sinter_decoder = sinter_decoder.compile_decoder_for_dem(dem=dem)
    
    decoder = config.compile_decoder()
    
    bitpacked_shots, obs = circuit.compile_detector_sampler().sample(shots=1000, separate_observables=True, bit_packed=True)
    
    unpacked_shots = np.unpackbits(bitpacked_shots, bitorder='little', axis=1)
    
    predictions_sinter = compiled_sinter_decoder.decode_shots_bit_packed(
        bit_packed_detection_event_data=bitpacked_shots)
    
    predictions_decode_batch = decoder.decode_batch(unpacked_shots)
    
    assert np.array_equal(predictions_sinter, predictions_decode_batch)

draganaurosgrbic · 2025-08-13T22:32:52Z

@oscarhiggott Please, check my last commit. It has a new test.

src/tesseract_sinter_compat.pybind.h

draganaurosgrbic · 2025-08-14T16:59:11Z

@oscarhiggott please write me all the issues you have today, so I can address them by tomorrow (tomorrow is my last day). If you have anything else you want me to change, please add by today.

oscarhiggott

This is excellent, thank you so much! Feel free to merge once the remaining comments are addressed.

…ctor

…eract-decoder into sinter-interface

draganaurosgrbic added 4 commits August 9, 2025 17:29

Python interface for Sinter API

72750f8

Minor

75068c9

Minor change

1f34dd9

Merge branch 'main' into sinter-interface

0dd64ba

draganaurosgrbic requested a review from oscarhiggott August 11, 2025 12:17

draganaurosgrbic added 2 commits August 11, 2025 10:05

Merge branch 'main' into sinter-interface

1807fc1

Merge branch 'main' into sinter-interface

655fccd

oscarhiggott reviewed Aug 11, 2025

View reviewed changes

src/tesseract_sinter_compat.pybind.cc Outdated Show resolved Hide resolved

src/BUILD Outdated Show resolved Hide resolved

Test integration with Sinter, include Sinter API inside tesseract lib…

d4ab871

…rary, bind tesseract decoding config inside Sinter API

draganaurosgrbic added 3 commits August 12, 2025 10:42

Merge branch 'main' into sinter-interface

afa98e2

Merge branch 'main' into sinter-interface

aac74b9

Merge branch 'main' into sinter-interface

f2285e4

oscarhiggott reviewed Aug 12, 2025

View reviewed changes

draganaurosgrbic added 2 commits August 12, 2025 13:32

Make sinter decoder pickable

9c4a6f1

Documentation for Sinter integration

e88f9c7

Merge branch 'main' into sinter-interface

3b8f406

Add Oscar's test

839d4a9

oscarhiggott reviewed Aug 13, 2025

View reviewed changes

src/py/README.md Show resolved Hide resolved

Remove print statements inside sinter test

4449c43

draganaurosgrbic added 2 commits August 13, 2025 14:47

Add custom decoders dict

cd395c9

Update sinter tests with custom detectors dict

f37f370

Another sinter test

1c9f5aa

draganaurosgrbic and others added 3 commits August 13, 2025 15:34

Minor change

fe8a486

Merge branch 'main' into sinter-interface

bc8c691

format and unpack observables in test

cbf8942

oscarhiggott reviewed Aug 14, 2025

View reviewed changes

src/tesseract_sinter_compat.pybind.h Show resolved Hide resolved

oscarhiggott approved these changes Aug 14, 2025

View reviewed changes

draganaurosgrbic added 3 commits August 14, 2025 15:18

Merge branch 'main' into sinter-interface

524fd50

Make DEM optional argument when contructing config and compiling dete…

b078bb4

…ctor

Merge branch 'sinter-interface' of https://github.com/quantumlib/tess…

8522389

…eract-decoder into sinter-interface

draganaurosgrbic merged commit ad65b82 into main Aug 14, 2025
4 checks passed

draganaurosgrbic deleted the sinter-interface branch August 14, 2025 23:14

draganaurosgrbic mentioned this pull request Aug 14, 2025

make the python decoder compatible with Sinter #55

Closed

NoureldinYosri mentioned this pull request Sep 5, 2025

Build for py13 #122

Merged

NoureldinYosri mentioned this pull request Oct 20, 2025

drop macos-13 from workflows #146

Merged

Make Tesseract compatible with Sinter #88

Make Tesseract compatible with Sinter #88

Uh oh!

Conversation

draganaurosgrbic commented Aug 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

oscarhiggott left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

draganaurosgrbic commented Aug 11, 2025

Uh oh!

oscarhiggott left a comment

Choose a reason for hiding this comment

Uh oh!

draganaurosgrbic commented Aug 12, 2025

Uh oh!

oscarhiggott commented Aug 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

draganaurosgrbic commented Aug 13, 2025

Uh oh!

draganaurosgrbic commented Aug 13, 2025

Uh oh!

oscarhiggott commented Aug 13, 2025

Uh oh!

oscarhiggott commented Aug 13, 2025

Uh oh!

draganaurosgrbic commented Aug 13, 2025

Uh oh!

oscarhiggott commented Aug 13, 2025

Uh oh!

oscarhiggott commented Aug 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

draganaurosgrbic commented Aug 13, 2025

Uh oh!

oscarhiggott commented Aug 13, 2025

Uh oh!

draganaurosgrbic commented Aug 13, 2025

Uh oh!

oscarhiggott commented Aug 13, 2025

Uh oh!

draganaurosgrbic commented Aug 13, 2025

Uh oh!

oscarhiggott commented Aug 13, 2025

Uh oh!

draganaurosgrbic commented Aug 13, 2025

Uh oh!

Uh oh!

draganaurosgrbic commented Aug 14, 2025

Uh oh!

oscarhiggott left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

draganaurosgrbic commented Aug 11, 2025 •

edited

Loading

oscarhiggott commented Aug 13, 2025 •

edited

Loading

oscarhiggott commented Aug 13, 2025 •

edited

Loading