Skip to content

Commit a4a50d8

Browse files
committed
feat(conclusion): add nhs + stars checklists
1 parent 0bd761a commit a4a50d8

File tree

3 files changed

+94
-0
lines changed

3 files changed

+94
-0
lines changed

pages/further_info/conclusion.qmd

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,8 @@ Use the checklists linked below to identify what you've already achieved (woohoo
6666

6767
Download checklists to **audit existing models** or **guide development of existing models**.
6868

69+
You can see examples of completed checklists in the nurse visit and stroke simulation example model repositories.
70+
6971
{{< downloadthis conclusion_resources/stars_reproducibility_recommendations.md dname="stars_reproducibility_recommendations" label="Download the STARS reproducibility recommendations" type="primary" >}}
7072

7173
{{< downloadthis conclusion_resources/nhs_levels_of_rap.md dname="nhs_levels_of_rap" label="Download the NHS Levels of RAP maturity framework" type="primary" >}}
Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# NHS Levels of RAP Maturity Framework
2+
3+
The following framework has been directly copied from the RAP Community of Practice repository/website: [NHS RAP Levels of RAP Framework](https://nhsdigital.github.io/rap-community-of-practice/introduction_to_RAP/levels_of_RAP/).
4+
5+
This framework is maintained by the NHS RAP Community of Practice and is © 2024 Crown Copyright (NHS England), shared by them under the terms of the [Open Government 3.0 licence](https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/).
6+
7+
The specific version of the framework copied below is that from commit [2549256](https://github.com/NHSDigital/rap-community-of-practice/commit/2549256498886d6d7ea4cdb736e2a2864c8bb461) (9th September 2024).
8+
9+
## 🥉 Baseline
10+
11+
RAP fundamentals offering resilience against future change.
12+
13+
| Criteria | Completion | Further details |
14+
| - | - | - |
15+
| Data produced by code in an open-source language (e.g., Python, R, SQL). | | |
16+
| Code is version controlled (see [Git basics](https://nhsdigital.github.io/rap-community-of-practice/training_resources/git/introduction-to-git/) and [using Git collaboratively](https://nhsdigital.github.io/rap-community-of-practice/training_resources/git/using-git-collaboratively/) guides). | | |
17+
| Repository includes a README.md file (or equivalent) that clearly details steps a user must follow to reproduce the code (use [NHS Open Source Policy section on Readmes](https://github.com/nhsx/open-source-policy/blob/main/open-source-policy.md#b-readmes) as a guide). | | |
18+
| Code has been [peer reviewed](https://nhsdigital.github.io/rap-community-of-practice/implementing_RAP/workflow/code-review/). | | |
19+
| Code is [published in the open](https://nhsdigital.github.io/rap-community-of-practice/implementing_RAP/publishing_code/how-to-publish-your-code-in-the-open/) and linked to & from accompanying publication (if relevant). | | |
20+
21+
## 🥈 Silver
22+
23+
Implementing best practice by following good analytical and software engineering standards.
24+
25+
Meeting all of the above requirements, plus:
26+
27+
| Criteria | Completion | Further details |
28+
| - | - | - |
29+
| Outputs are produced by code with minimal manual intervention. | | |
30+
| Code is well-documented including user guidance, explanation of code structure & methodology and [docstrings](https://nhsdigital.github.io/rap-community-of-practice/training_resources/python/python-functions/#documentation) for functions. | | |
31+
| Code is well-organised following [standard directory format](https://nhsdigital.github.io/rap-community-of-practice/training_resources/python/project-structure-and-packaging/). | | |
32+
| [Reusable functions](https://nhsdigital.github.io/rap-community-of-practice/training_resources/python/python-functions/) and/or classes are used where appropriate. | | |
33+
| Code adheres to agreed coding standards (e.g PEP8, [style guide for Pyspark](https://nhsdigital.github.io/rap-community-of-practice/training_resources/pyspark/pyspark-style-guide/)). | | |
34+
| Pipeline includes a testing framework ([unit tests](https://nhsdigital.github.io/rap-community-of-practice/training_resources/python/unit-testing/), [back tests](https://nhsdigital.github.io/rap-community-of-practice/training_resources/python/backtesting/)). | | |
35+
| Repository includes dependency information (e.g. [requirements.txt](https://pip.pypa.io/en/stable/user_guide/#requirements-files), [PipFile](https://github.com/pypa/pipfile/blob/main/README.rst), [environment.yml](https://nhsdigital.github.io/rap-community-of-practice/training_resources/python/virtual-environments/conda/)). | | |
36+
| [Logs](https://nhsdigital.github.io/rap-community-of-practice/training_resources/python/logging-and-error-handling/) are automatically recorded by the pipeline to ensure outputs are as expected. | | |
37+
| Data is handled and output in a [Tidy data format](https://medium.com/@kimrodrikwa/untidy-data-a90b6e3ebe4c). | | |
38+
39+
## 🥇 Gold
40+
41+
Analysis as a product to further elevate your analytical work and enhance its reusability to the public.
42+
43+
Meeting all of the above requirements, plus:
44+
45+
| Criteria | Completion | Further details |
46+
| - | - | - |
47+
| Code is fully [packaged](https://packaging.python.org/en/latest/). | | |
48+
| Repository automatically runs tests etc. via CI/CD or a different integration/deployment tool e.g. [GitHub Actions](https://docs.github.com/en/actions). | | |
49+
| Process runs based on event-based triggers (e.g., new data in database) or on a schedule. | | |
50+
| Changes to the RAP are clearly signposted. E.g. a changelog in the package, releases etc. (See gov.uk info on [Semantic Versioning](https://github.com/alphagov/govuk-frontend/blob/main/docs/contributing/versioning.md)). | | |
Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# STARS Reproducibility Recommendations
2+
3+
As part of the project STARS (Sharing Tools and Artefacts for Reproducible Simulations), a series of computational reproducibility assessments were conducted and described in:
4+
5+
> Heather, A., Monks, T., Harper, A., Mustafee, N., & Mayne, A. (2025). On the reproducibility of discrete-event simulation studies in health research: an empirical study using open models. Journal of Simulation. https://doi.org/10.1080/17477778.2025.2552177.
6+
7+
From these, several recommendations were shared to support reproducibility of healthcare discrete-event simulation (DES) models. These are copied below. Those marked with a star (⭐) were identified as having the greatest impact in the paper.
8+
9+
## Recommendations to support reproduction
10+
11+
| Recommendation | Completion | Further details |
12+
| - | - | - |
13+
| **Set-up** |
14+
| Share code with an open licence (⭐) | | |
15+
| Link publication to a specific version of the code | | |
16+
| List dependencies and versions | | |
17+
| **Running the model** |
18+
| Provide code for all scenarios and sensitivity analyses (⭐) | | |
19+
| Ensure model parameters are correct (⭐) | | |
20+
| Control randomness | | |
21+
| **Outputs** |
22+
| Include code to calculate all required model outputs (⭐) | | |
23+
| Include code to generate the tables, figures, and other reported results (⭐) | | |
24+
25+
## Recommendations to support troubleshooting and reuse
26+
27+
| Recommendation | Completion | Further details |
28+
| - | - | - |
29+
| **Design** |
30+
| Separate model code from applications | | |
31+
| Avoid hard-coded parameters | | |
32+
| Minimise code duplication | | |
33+
| **Clarity** |
34+
| Comment sufficiently | | |
35+
| Ensure clarity and consistency in the model results tables | | |
36+
| Include run instructions | | |
37+
| State run times and machine specifications | | |
38+
| **Functionality** |
39+
| Optimise model run time | | |
40+
| Save outputs to a file | | |
41+
| Avoid excessive output files | | |
42+
| Address large file sizes | | |

0 commit comments

Comments
 (0)