You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Saving your simulation results to file is important for reproducility, as it allows others to verify your findings and generate consistent (or new) figures and analyses, even if they can't re-run your simulation.
450
+
This practice is transparent, providing a clear record of what you found, and it is valuable for you as well, ensuring you always know exactly what results you obtained, and can regenerate your own tables and figures from the results.
451
+
452
+
However, there are two key things to keep in mind:
453
+
454
+
**1. Number of files.** Running many scenarios or replications can easily lead to an explosion of output files. Do not save each scenario or run as a separate file unless there is a specific need. Instead, combine all results into a **single file with columns marking scenario and replication IDs**.
455
+
456
+
**2. Avoid large file sizes.** Be strategic about what you save. For short tests and debugging, saving detailed ("patient-level") results makes sense. But for full-scale runs with many replications, those files can become unmanageably large. Generally, **save summary outputs** to file for analysis (e.g., means from each run), not massive raw datasets. If you absolutely need to save or share large files, use compressed formats (e.g., `csv.gz`). Also, keep in mind practical size limits for version control: for example, GitHub's individual file size limit is 100 MB.
<!-- Hide as no python-content or r-content blocks -->
6
+
<style>
7
+
#quarto-announcement {
8
+
display: none!important;
9
+
}
10
+
</style>
11
+
12
+
<br>
13
+
14
+
::: {.pale-blue}
15
+
16
+
**Well done, you made it to the end!** 😁
17
+
18
+
This book has shared with you the knowledge and tools to create simulation models in Python or R as part of a reproducible analytical pipeline - models that others can reproduce, trust, understand, and build upon.
19
+
20
+
:::
21
+
22
+
## What's next?
23
+
24
+
<divclass="h3-tight"></div>
25
+
26
+
### Explore the example models
27
+
28
+
These example models demonstrate many of the practices covered in this book. They may seem daunting at first - there's a lot going on - but having worked through the book, you're in a good position to understand them.
29
+
30
+
Remember, these are **examples**, not prescriptions. They're not perfect, and there's no single "right way" to build reproducible models. They simply show one approach to implementing the principles you've learned.
31
+
32
+
**Nurse visit simulation:**
33
+
34
+
{{< include ../../html/pydesrapmms.html >}}
35
+
36
+
{{< include ../../html/rdesrapmms.html >}}
37
+
38
+
**Stroke pathway simulation:**
39
+
40
+
{{< include ../../html/pydesrapstroke.html >}}
41
+
42
+
{{< include ../../html/rdesrapstroke.html >}}
43
+
44
+
### Make your own model
45
+
46
+
The best way to solidify what you've learned is to apply it. When planning you model, remember that a good simulation starts with **conceptual modelling**. As defined in Robinson (2007):
47
+
48
+
> "The conceptual model is a non-software specific description of the simulation model that is to be developed, describing the objectives, inputs, outputs, content, assumptions and simplifications of the model."
49
+
50
+
Some good resources on conceptual modelling include:
51
+
52
+
* Robinson, Stewart. 2007. "Chapter 5: Conceptual Modelling." In Simulation: The Practice of Model Development and Use, 63–75. John Wiley & Sons.
53
+
* Robinson, Stewart. 2007. "Chapter 6: Developing the Conceptual Model." In Simulation: The Practice of Model Development and Use, 77–93. John Wiley & Sons.
54
+
55
+
This book focused on building simple model structures to help you establish robust foundations and reproducible workflows. However, real-world simulation models often involve additional features and complexities, such as reneging, balking, priority classes, resource scheduling, branching, blocking, or more detailed patients pathways.
56
+
57
+
For inspiration on implementing a wider range of features, the [HSMA "little book of DES"](https://des.hsma.co.uk/) is an excellent resource. Its examples use Python, and the set up or structure may differ from those in this book, but the simulation principles apply whatever language you use. Focus on understanding the logic - how features are implemented and why - then adapt those ideas for the language, structure, or workflow that best fits your own model.
58
+
59
+
### Review your existing models
60
+
61
+
Already have simulation models in development or completed? Now's a great time to audit them against the practices you've learned.
62
+
63
+
Use the checklists linked below to identify what you've already achieved (woohoo!) and what's missing. Then revisit relevant sections of the book to fill the gaps. Even small improvements - adding seeds, externalising parameters, or documenting dependencies - can significantly enhance your model's reproducibility.
64
+
65
+
## Checklists
66
+
67
+
Download checklists to **audit existing models** or **guide development of existing models**.
68
+
69
+
You can see examples of completed checklists in the nurse visit and stroke simulation example model repositories.
{{< downloadthis conclusion_resources/nhs_levels_of_rap.md dname="nhs_levels_of_rap" label="Download the NHS Levels of RAP maturity framework" type="primary" >}}
74
+
75
+
Also, don't forget about the handy **verification and validation** checklist:
76
+
77
+
{{< downloadthis ../verification_validation/verification_validation_resources/verification_validation_checklist.md dname="verification_validation_checklist" label="Download the verification and validation checklist" type="primary" >}}
78
+
79
+
## Acknowledgements
80
+
81
+
This book builds upon the generous work of many contributors to the open-source and simulation communities. We are particularly grateful for:
82
+
83
+
* The **SimPy** and **simmer** development teams for creating and maintaining excellent open-source DES libraries.
84
+
* The **NHS RAP Community of Practice** for their maturity framework.
85
+
* The **HSMA Programme** (Health Service Modelling Associates) for their [little book of DES](https://des.hsma.co.uk/).
86
+
* All the **researchers and practitioners who have openly shared their simulation models**, enabling the research that informed this book.
87
+
***Contributors and reviewers** who have provided feedback to improve this resource
88
+
89
+
Full references and citations appear throughout the book where specific resources are discussed.
90
+
91
+
### Please cite this book
92
+
93
+
The code in this book is licensed under an **MIT License**, and the text is under **CC-BY-SA**, making it free to use, modify and share. However, we kindly ask/require that you **cite or acknowledge** this work when you use it.
94
+
95
+
Suggested citation:
96
+
97
+
> Heather, A., Monks, T., Mustafee, N., & Harper, A. (2025). DES RAP Book: Reproducible Discrete-Event Simulation in Python and R. https://github.com/pythonhealthdatascience/des_rap_book. https://doi.org/10.5281/zenodo.17094155.
98
+
99
+
## Find out more about STARS
100
+
101
+
This book is part of the **STARS (Sharing Tools and Artefacts for Reusable and Reproducible Simulations)** project, supported by the Medical Research Council [grant number MR/Z503915/1].
102
+
103
+
104
+
105
+
STARS tackles the challenges of sharing, reusing, and reproducing discrete event simulation (DES) models in healthcare. Our goal is to create open resources using the two most popular open-source languages for DES: Python and R. As part of this project, you'll find tutorials, code examples, and tools to help researchers and practitioners develop, validate, and share DES models more effectively.
<!-- TODO: Add link to STARS summary website once created -->
112
+
113
+
## Well done! It's a journey, not a race
114
+
115
+
Whether you're new to reproducible simulation or deepening your existing practice, you've taken important steps toward building more trustworthy, transparent models.
116
+
117
+
Implementing these practices requires time and iteration. **Perfection is not required immediately - or ever**. Many of us are crunched for time, juggling multiple projects and deadlines. Finding space to improve workflows can feel impossible.
118
+
119
+
When time is limited, focus on what matters most for your specific project. Go through the checklists or flip back through the book to identify key steps that would make the biggest difference. **Every small change moves your work forward.**
120
+
121
+
**Don't be afraid to share your model**, even if it doesn't have all the bells and whistles. You benefit from others' shared work; others benefit from yours. Shared models spark conversations, enable collaboration, and push the field forward. Each model shared raises the bar for transparency and makes it easier for the next person to do the same.
122
+
123
+
<br>
124
+
125
+
*See the next page for details on giving feedback and contributing to this resource.*
0 commit comments