-
Notifications
You must be signed in to change notification settings - Fork 3
Add blog post on Epiverse community contribution #395
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
6d3d345
6ceaab6
e839e08
2cc2e67
3bbe130
906b8fc
f5acabc
bd2a7cd
47d6936
acbae9c
e4024b7
6743d36
52e5fa7
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| @misc{davisImperfectToolContact2020, | ||
| title = {An Imperfect Tool: Contact Tracing Could Provide Valuable Reductions in {{COVID-19}} Transmission If Good Adherence Can Be Achieved and Maintained}, | ||
| shorttitle = {An Imperfect Tool}, | ||
| author = {Davis, Emma L. and Lucas, Tim C. D. and Borlase, Anna and Pollington, Timothy M. and Abbott, Sam and Ayabina, Diepreye and Crellen, Thomas and Hellewell, Joel and Pi, Li and {CMMID COVID-19 working group} and Medley, Graham F. and Hollingsworth, T. D{\'e}irdre and Klepac, Petra}, | ||
| year = {2020}, | ||
| month = jun, | ||
| doi = {10.1101/2020.06.09.20124008}, | ||
| urldate = {2025-08-13}, | ||
| abstract = {Abstract Emerging evidence suggests that contact tracing has had limited success in the UK in reducing the R number across the COVID-19 pandemic. We investigate potential pitfalls and areas for improvement by extending an existing branching process contact tracing model, adding diagnostic testing and refining parameter estimates. Our results demonstrate that reporting and adherence are the most important predictors of programme impact but tracing coverage and speed plus diagnostic sensitivity also play an important role. We conclude that well-implemented contact tracing could bring small but potentially important benefits to controlling and preventing outbreaks, providing up to a 15\% reduction in R , and reaffirm that contact tracing is not currently appropriate as the sole control measure.}, | ||
| langid = {english}, | ||
| file = {/Users/lshjl15/Zotero/storage/AHPVUDTS/Davis et al. - 2020 - An imperfect tool contact tracing could provide v.pdf} | ||
| } | ||
|
|
||
| @article{firthUsingRealworldNetwork2020, | ||
| title = {Using a Real-World Network to Model Localized {{COVID-19}} Control Strategies}, | ||
| author = {Firth, Josh A. and Hellewell, Joel and Klepac, Petra and Kissler, Stephen and {CMMID COVID-19 Working Group} and Jit, Mark and Atkins, Katherine E. and Clifford, Samuel and {Villabona-Arenas}, C. Julian and Meakin, Sophie R. and Diamond, Charlie and Bosse, Nikos I. and Munday, James D. and Prem, Kiesha and Foss, Anna M. and Nightingale, Emily S. and Zandvoort, Kevin Van and Davies, Nicholas G. and Gibbs, Hamish P. and Medley, Graham and Gimma, Amy and Flasche, Stefan and Simons, David and Auzenbergs, Megan and Russell, Timothy W. and Quilty, Billy J. and Rees, Eleanor M. and Leclerc, Quentin J. and Edmunds, W. John and Funk, Sebastian and Houben, Rein M. G. J. and Knight, Gwenan M. and Abbott, Sam and Sun, Fiona Yueqian and Lowe, Rachel and Tully, Damien C. and Procter, Simon R. and Jarvis, Christopher I. and Endo, Akira and O'Reilly, Kathleen and Emery, Jon C. and Jombart, Thibaut and Rosello, Alicia and Deol, Arminder K. and Quaife, Matthew and Hu{\'e}, St{\'e}phane and Liu, Yang and Eggo, Rosalind M. and Pearson, Carl A. B. and Kucharski, Adam J. and Spurgin, Lewis G.}, | ||
| year = {2020}, | ||
| month = oct, | ||
| journal = {Nature Medicine}, | ||
| volume = {26}, | ||
| number = {10}, | ||
| pages = {1616--1622}, | ||
| issn = {1078-8956, 1546-170X}, | ||
| doi = {10.1038/s41591-020-1036-8}, | ||
| urldate = {2025-08-13}, | ||
| langid = {english}, | ||
| file = {/Users/lshjl15/Zotero/storage/EKVXH6KU/Firth et al. - 2020 - Using a real-world network to model localized COVI.pdf} | ||
| } | ||
|
|
||
| @article{hellewellFeasibilityControllingCOVID192020, | ||
| title = {Feasibility of Controlling {{COVID-19}} Outbreaks by Isolation of Cases and Contacts}, | ||
| author = {Hellewell, Joel and Abbott, Sam and Gimma, Amy and Bosse, Nikos I and Jarvis, Christopher I and Russell, Timothy W and Munday, James D and Kucharski, Adam J and Edmunds, W John and Funk, Sebastian and Eggo, Rosalind M and Sun, Fiona and Flasche, Stefan and Quilty, Billy J and Davies, Nicholas and Liu, Yang and Clifford, Samuel and Klepac, Petra and Jit, Mark and Diamond, Charlie and Gibbs, Hamish and Van Zandvoort, Kevin}, | ||
| year = {2020}, | ||
| month = apr, | ||
| journal = {The Lancet Global Health}, | ||
| volume = {8}, | ||
| number = {4}, | ||
| pages = {e488-e496}, | ||
| issn = {2214109X}, | ||
| doi = {10.1016/S2214-109X(20)30074-7}, | ||
| urldate = {2025-04-09}, | ||
| langid = {english}, | ||
| file = {/Users/lshjl15/Zotero/storage/NG9MCA47/Hellewell et al. - 2020 - Feasibility of controlling COVID-19 outbreaks by i.pdf} | ||
| } | ||
|
|
||
| @article{kucharskiCOVID19ResponseIllustrates2020, | ||
| title = {The {{COVID-19}} Response Illustrates That Traditional Academic Reward Structures and Metrics Do Not Reflect Crucial Contributions to Modern Science}, | ||
| author = {Kucharski, Adam J. and Funk, Sebastian and Eggo, Rosalind M.}, | ||
| year = {2020}, | ||
| month = oct, | ||
| journal = {PLOS Biology}, | ||
| volume = {18}, | ||
| number = {10}, | ||
| pages = {e3000913}, | ||
| publisher = {Public Library of Science (PLoS)}, | ||
| issn = {1545-7885}, | ||
| doi = {10.1371/journal.pbio.3000913}, | ||
| urldate = {2025-07-22}, | ||
| copyright = {http://creativecommons.org/licenses/by/4.0/}, | ||
| langid = {english}, | ||
| file = {/Users/lshjl15/Zotero/storage/ZB4LABCU/Kucharski et al. - 2020 - The COVID-19 response illustrates that traditional.pdf} | ||
| } | ||
|
|
||
| @article{kucharskiEffectivenessRingVaccination2016, | ||
| title = {Effectiveness of {{Ring Vaccination}} as {{Control Strategy}} for {{Ebola Virus Disease}}}, | ||
| author = {Kucharski, Adam J. and Eggo, Rosalind M. and Watson, Conall H. and Camacho, Anton and Funk, Sebastian and Edmunds, W. John}, | ||
| year = {2016}, | ||
| month = jan, | ||
| journal = {Emerging Infectious Diseases}, | ||
| volume = {22}, | ||
| number = {1}, | ||
| pages = {105--108}, | ||
| issn = {1080-6040, 1080-6059}, | ||
| doi = {10.3201/eid2201.151410}, | ||
| urldate = {2023-05-23}, | ||
| file = {/Users/lshjl15/Zotero/storage/98HLKB9J/Kucharski et al. - 2016 - Effectiveness of Ring Vaccination as Control Strat.pdf} | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,147 @@ | ||
| --- | ||
| title: "Epiverse community engagement and software sustainability for research software" | ||
| author: | ||
| - name: "Joshua W. Lambert" | ||
| orcid: "0000-0001-5218-3046" | ||
| date: "2025-08-25" | ||
| categories: [open-source, R, R package, epidemiology, community, Epiverse, DOI] | ||
| bibliography: index.bib | ||
| format: | ||
| html: | ||
| toc: true | ||
| --- | ||
|
|
||
| Software that is developed for research or by researchers can be difficult to maintain given the incentive and funding structures in academia. This remains true for epidemiology, with a large volume of software written during the COVID-19 pandemic, much of which is now abandonware[^1]. This does not mean that the software developed to understand the COVID-19 pandemic was bad or does not have utility in understanding future epidemics and pandemics, but just that the capacity to maintain and further develop these tools is not available now the pandemic is no [longer considered an acute public health emergency](https://www.who.int/news/item/05-05-2023-statement-on-the-fifteenth-meeting-of-the-international-health-regulations-(2005)-emergency-committee-regarding-the-coronavirus-disease-(covid-19)-pandemic). | ||
|
|
||
| These issues around software sustainability and the academic structures that hinder software longevity were raised by @kucharskiCOVID19ResponseIllustrates2020 and were one of the leading reasons for the [Epiverse-TRACE initiative](https://epiverse-trace.github.io/). Alongside the developing novel software (R packages), Epiverse also has a commitment to support the community of package developers in epidemiology and outbreak analytics. The initiative also tries to improve community collaboration and contribution friendliness of open-source software. | ||
|
|
||
| This blog post highlights some recent work by Epiverse software engineers to collaborate on research software, or researchware, to help improve an R package that was initially written in the early days of the COVID-19 pandemic (January 2020 - May 2020) to assess the effectiveness of isolation and contact tracing effectiveness [@hellewellFeasibilityControllingCOVID192020]. It built on code written for the 2014-2016 West Africa Ebola outbreak to provide insights into ring vaccination [@kucharskiEffectivenessRingVaccination2016]. These applications and the general nature of the questions the package addresses suggest that it could be of great help in future infectious disease outbreaks, but has lacked developer resources without pandemic-related priorities. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think it's fair to mention that the work that package was used to develop received a lot of attention, i,e., citations, and the package was repurposed a few times for other high impact work, i,e., covidhm and others.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I see this is later mentioned in the next section. I wonder if the two sections can be combined then to make it easier to be sold on the importance of this exercise. |
||
|
|
||
| ## The R package | ||
|
|
||
| The R package in question is [{ringbp}](https://github.com/epiforecasts/ringbp). The package has two pieces of functionality: 1) to simulate an infectious disease outbreak using a branching process model with non-pharmaceutical interventions; and 2) to calculate the proportion of simulated outbreaks that are contained (i.e. do not cause a large sustained human-to-human epidemic). The utility of the package's general model framework has been shown by serving as a template for other epidemiological research such as [post-exposure prophylaxis](https://sophiemeakin.github.io/pepbp/), [network effects on control](https://github.com/lgs85/covidhm) [@firthUsingRealworldNetwork2020] and the [impact of self-reporting and isolation adherence](https://github.com/timcdlucas/ringbp) [@davisImperfectToolContact2020]. | ||
joshwlambert marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ## The problem | ||
|
|
||
| It is understandable that because {ringbp} was written in haste to produce insights to inform pandemic response it did not adhere to all software best practices. Usability, documentation, testing, code style and (computational) performance could be improved. Certain aspects of model code, like parameterisations, were hard-coded, not providing users the full flexibility that the model could allow. | ||
joshwlambert marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ## Epiverse contribution | ||
|
|
||
| In the recent months Epiverse has collaborated with {ringbp} developers Seb Funk (also a member of Epiverse) and Carl Pearson (external collaborator), based at the London School of Hygiene and Tropical Medicine and University of North Carolina, respectively, to try and improve the R package, both internally and from the user-experience. The following sections will give brief summaries of some of the collaborative developments. | ||
|
|
||
| ### User interface | ||
|
|
||
| The user experience (API) of the package has been refactored. The main simulation function `scenario_sim()` remains, but its arguments have been modularised to better group model parameters and control arguments. This also makes the package easier to develop further without necessarily introducing many breaking changes and prevents the number of top-level function arguments from expanding. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. would be good here to give an example of before/after function syntax
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. yep, show-don't-tell
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added old versus new
joshwlambert marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| :::: {.columns} | ||
|
|
||
| ::: {.column width="47.5%"} | ||
|
|
||
| ### Old | ||
|
|
||
| ``` r | ||
| scenario_sim( | ||
| n.sim = 5, | ||
| num.initial.cases = 5, | ||
| cap_max_days = 365, | ||
| cap_cases = 2000, | ||
| r0isolated = 0, | ||
| r0community = 2.5, | ||
| disp.iso = 1, | ||
| disp.com = 0.16, | ||
| k = 0.7, | ||
| delay_shape = 2.5, | ||
| delay_scale = 5, | ||
| prop.asym = 0, | ||
| prop.ascertain = 0 | ||
| ) | ||
| ``` | ||
|
|
||
| ::: | ||
|
|
||
| ::: {.column width="5%"} | ||
| <!-- empty column to create gap --> | ||
| ::: | ||
|
|
||
| ::: {.column width="47.5%"} | ||
|
|
||
| ### New | ||
|
|
||
| ``` r | ||
| scenario_sim( | ||
| n = 5, | ||
| initial_cases = 5, | ||
| offspring = offspring_opts( | ||
| community = \(n) rnbinom(n = n, mu = 2.5, size = 0.16), | ||
| isolated = \(n) rnbinom(n = n, mu = 0, size = 1), | ||
| asymptomatic = \(n) rnbinom(n = n, mu = 2.5, size = 0.16) | ||
| ), | ||
| delays = delay_opts( | ||
| incubation_period = \(n) rweibull(n = n, shape = 2.32, scale = 6.49), | ||
| onset_to_isolation = \(n) rweibull(n = n, shape = 2.5, scale = 5) | ||
| ), | ||
| event_probs = event_prob_opts( | ||
| asymptomatic = 0, | ||
| presymptomatic_transmission = 0.3, | ||
| symptomatic_ascertained = 0 | ||
| ), | ||
| interventions = intervention_opts(quarantine = TRUE), | ||
| sim = sim_opts( | ||
| cap_max_days = 365, | ||
| cap_cases = 2000 | ||
| ) | ||
| ) | ||
| ``` | ||
|
|
||
| ::: | ||
|
|
||
| :::: | ||
|
|
||
| The new API gives the user more control over the model's parameterisation. The incubation period is now specified by the user instead of being set to an estimate for COVID-19. The way offspring and delay distribution functions are specified also means that any distributional or non-parametric form can be supplied, relaxing the assumption that the onset-to-isolation has to be a Weibull distribution. | ||
|
|
||
| Users can now specify the proportion of presymptomatic transmission rather than having to understand the skew normal parameterisation used by the simulation model, making it easier to get started with the package for new users. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. again I would put an example here which will really help people who don't know the package to follow
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Again, I think the comparison code chunks with the old and new |
||
|
|
||
| Lastly on user-facing changes, the naming and style of function arguments has been standardised for consistent use of [snakecase](https://en.wikipedia.org/wiki/Snake_case) style and abbreviations. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. mention that this was previously a mix of stuff? (perhaps in "the problem" above)
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The problem section mentions code style and here we mention standardisation so I would say it's implicit that there was a mix of styles previously. I'll leave as is, as I don't want the details to obfuscate the overarching message, I guess that if anyone is interested they can see the package changelog or version history (linked at the end of this post).
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can't the "Problem" section have various sections/paragraphs that describe the problems so that they're all in one place? Problems could be introduced as topic sentences in various paragraphs for each kind of problem. |
||
|
|
||
| ### Documentation | ||
|
|
||
| Function documentation already used {roxygen2}, but did not make use of inheritance or comprehensively document the function output or usage. We used `@inheritParams` from {roxygen2} to deduplicate, added `@return` documentation to all functions. We also improved the function argument documentation by following a structure of: `<type>: description`, for example: | ||
|
|
||
| ```r | ||
| @param sim a `list` with class `<ringbp_sim_opts>`: the simulation control | ||
| options for the \pkg{ringbp} model, returned by [sim_opts()] | ||
| ``` | ||
|
|
||
| Exported functions now have informative examples (`@examples`) to showcase how the functions should be used. Function examples now always run (removing `\dontrun{}`) to catch any errors. | ||
|
|
||
| The [{roxyglobals}](https://github.com/anthonynorth/roxyglobals) package has been added to automate the management of global variables with the use of the `@autoglobal` tag. | ||
|
|
||
| Vignettes are useful long-form package documentation. Thus far we've added one vignette to the package and plan to add more where relevant. | ||
|
|
||
| ### Bug fixes | ||
|
|
||
| Perhaps more important that any of the software best practices and user interface is the correctness of the code. In our developments we've uncovered a few bugs in the previous version of {ringbp}. Errors in the timing of quarantining infected individuals, sampling from the onset-to-isolation distribution, and calculating the generation time from the incubation period have all been identified and fixed. | ||
|
|
||
| ### Testing | ||
|
|
||
| - simulation correctness regression (snapshot) testing | ||
joshwlambert marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ### Miscellaneous | ||
|
|
||
| There are various other changes in {ringbp} from our work. Examples include: input checking, not specifying erroneous function defaults, updating the package website, and functions that return `data.table` objects no longer [returning silently](https://cran.r-project.org/web/packages/data.table/vignettes/datatable-faq.html#sec:why-do-i-have-to-type-dt-sometimes-twice-after-using-to-print-the-result-to-console). Mentioned in the introduction, model performance has been incrementally improved, but we've not focused on this aspect, and the package will benefit from time spent focusing on this in the future; especially if the set and complexity of non-pharmaceutical interventions in the model expands. | ||
|
|
||
| ## Conclusion | ||
|
|
||
| The {ringbp} R package implements a simple but informative model for infectious disease transmission and interventions. When originally written it included many well-developed aspects, but the time constraints of real-time outbreak response meant several improvements were possible. | ||
|
|
||
| Epiverse-TRACE has the opportunity to not only develop new tooling for pandemic preparedness and response, but to contribute to the ecosystem of open-source software in infectious disease epidemiology. We hope that by covering the collaborative developments of {ringbp}, it can illustrate the benefits of bringing software up to date with best practices, and make tools available, accessible and robust when a new epidemic or pandemic occurs, in turn hopefully removing the need for redeveloping similar software in the future. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do you think there is scope for perhaps a reflection on sustainability / maintenance, which you mentioned extensively in the motivating intro? How do we ensure this is not again abandoned for 5 years (assuming it would be a problem if it was)? Also perhaps there is scope for a final statement that we're hoping that this will make the package more useful for others and will re-visit the analysis for the paper (with ref to the cmmid ringbp repo) in the first instance?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. On the first item - maybe also lean into all the other contributor-friendliness-items you've added (contributor statement, code of contact, making issue templates, tag clean up, etc) [or will add prior to releasing blog]? And plan for CRAN release?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I did consider this but initially opted again as I don't think it would be a positive message. Given the current lifecycle stage of Epiverse-TRACE I'm not sure we can make any grand statements about long term improvements in the landscape for OSS for research. Specifically on "How to do we ensure this is not again abandoned for 5 years", I think this is outside the scope of the post and I'm not sure I have an adequate answer. If you disagree, or there is something specific you'd like the post to cover please suggest and I'd be happy to incorporate.
We could add this to the post but it's thus far not been added to the {ringbp} repository so would be similar to the Testing section of the post which requires some more work on the package before publishing. IIRC there hasn't been any discussion of code of conduct, issue or PR templates; the contributing document has been raised in epiforecasts/ringbp#109.
This has been raised in epiforecasts/ringbp#128 and is the plan. It would be nice to publish this post in tandem with the {ringbp} release.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think there's a reasonably-but-not-overly optimistic statement here - that the flexibility, refactoring, expanded documentation, etc make it easier for other people to reuse it, which is the gateway for external contributors to engage with the project. That doesn't assure resources on it, but enables them. I'd also argue that folding it into a larger effort (tho I guess Epiverse-TRACE is wrapping?), instead of just an isolated package, improves awareness.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks, that's a good point that I'd largely overlooked. I've added a paragraph to the Conclusion to hopefully address this point in 6743d36. Let me know if you think it needs reworking.
The project is moving into Phase II with a focus on implementation with training and maintenance of tools, so will have a reduced software engineering capacity. |
||
|
|
||
| Enhancing the accessibility of software for users and developers by improving its documentation and user interface will hopefully provide a gateway for more external contributors to engage with the project. In the public health landscape of temporal surges in capacity and priorities, better enabling community contributions to open-source software should aid software sustainability. | ||
|
|
||
| All of the changes discussed in this blog post can be found in the [{ringbp} news](https://epiforecasts.io/ringbp/news/index.html). For details of developments see the [pull request history of {ringbp} on GitHub](https://github.com/epiforecasts/ringbp/pulls?q=is%3Apr+is%3Aclosed). | ||
|
|
||
| ## Acknowledgements | ||
|
|
||
| Thanks to Seb Funk and Carl Pearson for helpful feedback when drafting this post and for their collaboration on the {ringbp} project. | ||
|
|
||
| [^1]: [Defined by Cambridge Dictionary](https://dictionary.cambridge.org/dictionary/english/abandonware) as: "software that is no longer produced or supported by the company that originally made it". | ||
Uh oh!
There was an error while loading. Please reload this page.