diff --git a/README.md b/README.md index b15bc75..85a9294 100644 --- a/README.md +++ b/README.md @@ -1,136 +1,26 @@ -# Awesome Volunteer Computing [![Awesome](https://awesome.re/badge.svg)](https://awesome.re) +# About This Project +In today’s era of rapid AI and LLM (Large Language Model) development, we see groundbreaking models released almost daily. However, one critical issue is often overlooked: the massive computational resources required to train these models. -**volunteer computing** β€” the practice of donating idle computing power to help solve real-world scientific and humanitarian problems. -Early pioneers like **[GIMPS](https://www.mersenne.org/)** and **[distributed.net](https://www.distributed.net/)** paved the way for modern platforms that aggregate these resources into powerful virtual supercomputers, enabling research otherwise impossible. -> Below is the curated list of awesome platforms, projects, and tools for volunteer computing ---- +## The Problem +Training foundational models like GPT, LLaMA, or DeepSeek requires access to enormous GPU clustersβ€”something only a handful of tech giants and well-funded startups can afford. The capital expenditure (CAPEX) involved is extremely high, creating a significant barrier to entry. This results in: + * A monopoly on foundational model development + * Limited accessibility for researchers, hobbyists, and small organizations + * A lack of transparency around model training data and practices -## πŸ“š Contents +## What Can Be Done? +While renting cloud GPU clusters is one path, it's financially impractical for most. An alternative solution is volunteer computingβ€”a model where people contribute their idle computing power for a shared, community-driven cause. -- [Platforms & Infrastructure](#platforms--infrastructure) - - [Hybrid or Multi-Platform](#hybrid-or-multi-platform) - - [Desktop-Native](#desktop-native) - - [Mobile-Based](#mobile-based) - - [Browser-Based](#browser-based) -- [For Developers](#for-developers) -- [Volunteer Projects](#volunteer-projects) -- [Research & Papers](#research--papers) -- [Communities & Forums](#communities--forums) -- [Related Concepts](#related-concepts) -- [Articles & Videos](#articles--videos) -- [Contributing](#contributing) +Volunteer computing is not new. Projects like Folding@Home have proven that distributed efforts can achieve massive scale. +By applying this approach to AI, we can build a community-powered foundational modelβ€”one that is: + * Free + * Open + * Ethical + * Privacy-conscious ---- -## βš™οΈ Platforms & Infrastructure - -Volunteer computing systems people can join or build on top of. - -### 🧬 Hybrid or Multi-Platform - -- 🌸 **[Petals](https://petals.dev/)** – Peer-to-peer network that runs large language models (LLMs) like BLOOM across distributed GPUs and contributors’ machines. - -### πŸ’» Desktop-Native - -- 🧩 **[BOINC](https://boinc.berkeley.edu/)** – The gold standard in open-source volunteer computing, powering dozens of scientific research projects. -- πŸ–₯️ **[GridRepublic](https://www.gridrepublic.org/)** – A user-friendly frontend for managing BOINC projects and participation. -- πŸ’‘ **[Charity Engine](https://www.charityengine.com/)** – Turns donated computing power into charitable donations or rewards for users. -- πŸ§ͺ **[XtremWeb](http://www.xtremweb.net/)** – Research platform for global distributed computing used by academic institutions. - -### πŸ“± Mobile-Based - -- πŸ“² **[DreamLab](https://www.vodafone.com/vodafone-foundation/focus-areas/dreamlab-app)** – Vodafone Foundation’s mobile app that powers medical and nutritional research while your phone charges. - -### 🌐 Browser-Based - -- πŸ•ΈοΈ **[distri.js.org](https://distri.js.org/)** – A minimalist JavaScript framework enabling distributed computing inside any modern web browser using Web Workers. -- 🐼 **[Pando](https://github.com/pando-project/pando)** – A personal volunteer computing platform for the Web, allowing easy setup for personal or shared computations via JavaScript. ([See Paper](https://arxiv.org/vc/arxiv/papers/1804/1804.01482v1.pdf)) - ---- - -## πŸ› οΈ For Developers - -Libraries, SDKs, and tools for building or contributing to volunteer computing systems. - -- 🧰 **[BOINC Development Resources](https://boinc.berkeley.edu/trac/wiki/SoftwareDevelopment)** – Comprehensive guide covering the SDK, APIs, server/client setup, and creating BOINC-compatible applications. -- 🌳 **[webrtc-tree-overlay](https://github.com/elavoie/webrtc-tree-overlay)** – A library for building scalable WebRTC P2P mesh networks in a tree overlay structure, useful for browser-based distributed applications. -- πŸ”§ **[Volunteer Grid Toolkit (VGT)](https://sourceforge.net/projects/vgt/)** – A toolkit for experimenting with volunteer grid computing concepts. *Note: Check project activity/relevance.* -- πŸ”— **[GridCoin Developer Information](https://wiki.gridcoin.us/Developer-Information)** – Resources for integrating blockchain-based incentives (GridCoin) with distributed computing (primarily BOINC). - ---- - -## πŸ§ͺ Volunteer Projects - -Scientific and research projects powered by public computing contributions. - -- πŸ”¬ **[Folding@home](https://foldingathome.org/)** – Simulates protein folding to better understand diseases like cancer and Alzheimer’s. -- 🧬 **[Rosetta@home](https://boinc.bakerlab.org/)** – Predicts protein structures and supports synthetic biology innovation. -- 🌱 **[World Community Grid](https://www.worldcommunitygrid.org/)** – Runs humanitarian research on health, sustainability, and climate change. -- 🌌 **[Einstein@Home](https://einsteinathome.org/)** – Searches for gravitational waves and pulsars in massive astrophysical datasets. -- πŸͺ **[Asteroids@Home](http://asteroidsathome.net/)** – Reconstructs asteroid shapes based on brightness curve analysis. -- πŸ‘½ **[SETI@home](https://setiathome.berkeley.edu/)** – Based at UC Berkeley, a scientific experiment to Search for Extraterrestrial Intelligence (SETI) -- πŸ’₯ **[LHC@home](https://lhcathome.web.cern.ch/)** – Simulates CERN’s Large Hadron Collider experiments to advance particle physics. - -#### πŸ“˜ [Wikipedia: List of Volunteer Computing Projects](https://en.wikipedia.org/wiki/List_of_volunteer_computing_projects) – A comprehensive and categorized list of past and present volunteer computing projects. - ---- - -## πŸ“š Research & Papers - -Academic insights and deep technical dives into distributed volunteer computing. - -- πŸ“„ **[BOINC: A Platform for Volunteer Computing](https://arxiv.org/pdf/1903.01699)** (D. Anderson, 2019) – Describes the architecture, features, and implementation of the widely used BOINC middleware. -- πŸ“ˆ **[The Computational and Storage Potential of Volunteer Computing](https://www.researchgate.net/publication/4241887_The_Computational_and_Storage_Potential_of_Volunteer_Computing)** (D. Anderson & G. Fedak, 2006) – Analyzes measurements from ~330k hosts to assess the capacity of volunteer computing beyond CPU power. -- βš™οΈ **[Research on the Task Assignment Problem with Maximum Benefits in Volunteer Computing Platforms](https://www.mdpi.com/2073-8994/12/5/862)** (L. Wang et al., 2020) – Explores task scheduling strategies for maximizing benefits in VCPs considering deadlines. -- πŸ‘₯ **[Retaining volunteers in volunteer computing projects](https://www.researchgate.net/publication/45461278_Retaining_volunteers_in_volunteer_computing_projects)** (D. Nov, O. Arazy, D. Anderson, 2011) – Investigates factors influencing volunteer retention and contribution levels. -- πŸ’‘ **[Personal Volunteer Computing](https://arxiv.org/vc/arxiv/papers/1804/1804.01482v1.pdf)** (S. Cahon et al., 2018) – Proposes a more personal, web-based approach to volunteer computing and introduces the Pando platform. -- ❀️ **[Volunteer computing: A model of the factors determining contribution to community-based scientific research](https://dl.acm.org/doi/abs/10.1145/1772690.1772766)** (O. Nov, D. Anderson, O. Arazy, 2010) – Develops and tests a model of user motivation based on SETI@home survey data and activity logs. - ---- - -## πŸ‘₯ Communities & Forums - -Places to ask questions, meet contributors, or follow development. - -- πŸ’¬ [BOINC Message Boards](https://boinc.berkeley.edu/dev/forum_index.php) -- πŸ“’ [Reddit: r/BOINC](https://www.reddit.com/r/BOINC/) -- πŸ—¨οΈ [BOINC Discord](https://discord.gg/boinc) -- *(Project-specific forums, like Folding@home's, also exist)* - ---- - -## πŸ“Ί Articles & Videos - -Resources to learn more about the real-world impact and concepts of volunteer computing. - -- πŸ“° [What is Volunteer Computing? – TechRadar](https://www.techradar.com/news/what-is-volunteer-computing) – General overview article. -- 🌐 [Volunteer computing - Wikipedia](https://en.wikipedia.org/wiki/Volunteer_computing) – Comprehensive encyclopedia article. -- CERN Explainer: [What is volunteer computing? | LHC@home](https://lhcathome.web.cern.ch/about/volunteer-computing) – Explanation from the perspective of a major project. -- Khan Academy: [Citizen science (article)](https://www.khanacademy.org/computing/computers-and-internet/xcae6f4a7ff015e7d:computing-innovations/xcae6f4a7ff015e7d:crowdsourcing-innovations/a/citizen-science) – Discusses volunteer computing as a form of citizen science. - ---- - -## πŸ”— Related Concepts - -Areas closely related to or overlapping with volunteer computing. - -- **Citizen Science:** Public participation in scientific research, often involving data collection or analysis. Volunteer computing is sometimes considered a sub-field. - - [Zooniverse](https://www.zooniverse.org/): Popular platform for web-based citizen science projects (classification, transcription). - - [SciStarter](https://scistarter.org/): Hub for discovering citizen science projects. -- **Distributed Computing:** General term for systems where components are located on different networked computers, communicating and coordinating actions. -- **Grid Computing:** Often involves more formal resource sharing agreements and infrastructure, typically within or between organizations, compared to the public volunteer model. - ---- - -## 🀝 Contributing - -Found a project we missed? Want to improve descriptions or links? Contributions welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for how to get started. - ---- - -## πŸ“„ License - -[CC0 1.0 Universal (Public Domain Dedication)](LICENSE) - -> Inspired by [awesome](https://awesome.re), the community-driven standard for curated knowledge. +## The Vision +* A truly open-source model built by the community, for the community +* Transparent training datasetsβ€”chosen and vetted collectively +* Models that can be fine-tuned and deployed without fear of hidden limitations or licensing issues +* Respect for privacy and ethical data use baked into the training process \ No newline at end of file diff --git a/tools.md b/tools.md new file mode 100644 index 0000000..718ffe9 --- /dev/null +++ b/tools.md @@ -0,0 +1,136 @@ +# Awesome Volunteer Computing [![Awesome](https://awesome.re/badge.svg)](https://awesome.re) + +**volunteer computing** β€” the practice of donating idle computing power to help solve real-world scientific and humanitarian problems. + +Early pioneers like **[GIMPS](https://www.mersenne.org/)** and **[distributed.net](https://www.distributed.net/)** paved the way for modern platforms that aggregate these resources into powerful virtual supercomputers, enabling research otherwise impossible. +> Below is the curated list of awesome platforms, projects, and tools for volunteer computing +--- + +## πŸ“š Contents and tools + +- [Platforms & Infrastructure](#platforms--infrastructure) + - [Hybrid or Multi-Platform](#hybrid-or-multi-platform) + - [Desktop-Native](#desktop-native) + - [Mobile-Based](#mobile-based) + - [Browser-Based](#browser-based) +- [For Developers](#for-developers) +- [Volunteer Projects](#volunteer-projects) +- [Research & Papers](#research--papers) +- [Communities & Forums](#communities--forums) +- [Related Concepts](#related-concepts) +- [Articles & Videos](#articles--videos) +- [Contributing](#contributing) + +--- + +## βš™οΈ Platforms & Infrastructure + +Volunteer computing systems people can join or build on top of. + +### 🧬 Hybrid or Multi-Platform + +- 🌸 **[Petals](https://petals.dev/)** – Peer-to-peer network that runs large language models (LLMs) like BLOOM across distributed GPUs and contributors’ machines. + +### πŸ’» Desktop-Native + +- 🧩 **[BOINC](https://boinc.berkeley.edu/)** – The gold standard in open-source volunteer computing, powering dozens of scientific research projects. +- πŸ–₯️ **[GridRepublic](https://www.gridrepublic.org/)** – A user-friendly frontend for managing BOINC projects and participation. +- πŸ’‘ **[Charity Engine](https://www.charityengine.com/)** – Turns donated computing power into charitable donations or rewards for users. +- πŸ§ͺ **[XtremWeb](http://www.xtremweb.net/)** – Research platform for global distributed computing used by academic institutions. + +### πŸ“± Mobile-Based + +- πŸ“² **[DreamLab](https://www.vodafone.com/vodafone-foundation/focus-areas/dreamlab-app)** – Vodafone Foundation’s mobile app that powers medical and nutritional research while your phone charges. + +### 🌐 Browser-Based + +- πŸ•ΈοΈ **[distri.js.org](https://distri.js.org/)** – A minimalist JavaScript framework enabling distributed computing inside any modern web browser using Web Workers. +- 🐼 **[Pando](https://github.com/pando-project/pando)** – A personal volunteer computing platform for the Web, allowing easy setup for personal or shared computations via JavaScript. ([See Paper](https://arxiv.org/vc/arxiv/papers/1804/1804.01482v1.pdf)) + +--- + +## πŸ› οΈ For Developers + +Libraries, SDKs, and tools for building or contributing to volunteer computing systems. + +- 🧰 **[BOINC Development Resources](https://boinc.berkeley.edu/trac/wiki/SoftwareDevelopment)** – Comprehensive guide covering the SDK, APIs, server/client setup, and creating BOINC-compatible applications. +- 🌳 **[webrtc-tree-overlay](https://github.com/elavoie/webrtc-tree-overlay)** – A library for building scalable WebRTC P2P mesh networks in a tree overlay structure, useful for browser-based distributed applications. +- πŸ”§ **[Volunteer Grid Toolkit (VGT)](https://sourceforge.net/projects/vgt/)** – A toolkit for experimenting with volunteer grid computing concepts. *Note: Check project activity/relevance.* +- πŸ”— **[GridCoin Developer Information](https://wiki.gridcoin.us/Developer-Information)** – Resources for integrating blockchain-based incentives (GridCoin) with distributed computing (primarily BOINC). + +--- + +## πŸ§ͺ Volunteer Projects + +Scientific and research projects powered by public computing contributions. + +- πŸ”¬ **[Folding@home](https://foldingathome.org/)** – Simulates protein folding to better understand diseases like cancer and Alzheimer’s. +- 🧬 **[Rosetta@home](https://boinc.bakerlab.org/)** – Predicts protein structures and supports synthetic biology innovation. +- 🌱 **[World Community Grid](https://www.worldcommunitygrid.org/)** – Runs humanitarian research on health, sustainability, and climate change. +- 🌌 **[Einstein@Home](https://einsteinathome.org/)** – Searches for gravitational waves and pulsars in massive astrophysical datasets. +- πŸͺ **[Asteroids@Home](http://asteroidsathome.net/)** – Reconstructs asteroid shapes based on brightness curve analysis. +- πŸ‘½ **[SETI@home](https://setiathome.berkeley.edu/)** – Based at UC Berkeley, a scientific experiment to Search for Extraterrestrial Intelligence (SETI) +- πŸ’₯ **[LHC@home](https://lhcathome.web.cern.ch/)** – Simulates CERN’s Large Hadron Collider experiments to advance particle physics. + +#### πŸ“˜ [Wikipedia: List of Volunteer Computing Projects](https://en.wikipedia.org/wiki/List_of_volunteer_computing_projects) – A comprehensive and categorized list of past and present volunteer computing projects. + +--- + +## πŸ“š Research & Papers + +Academic insights and deep technical dives into distributed volunteer computing. + +- πŸ“„ **[BOINC: A Platform for Volunteer Computing](https://arxiv.org/pdf/1903.01699)** (D. Anderson, 2019) – Describes the architecture, features, and implementation of the widely used BOINC middleware. +- πŸ“ˆ **[The Computational and Storage Potential of Volunteer Computing](https://www.researchgate.net/publication/4241887_The_Computational_and_Storage_Potential_of_Volunteer_Computing)** (D. Anderson & G. Fedak, 2006) – Analyzes measurements from ~330k hosts to assess the capacity of volunteer computing beyond CPU power. +- βš™οΈ **[Research on the Task Assignment Problem with Maximum Benefits in Volunteer Computing Platforms](https://www.mdpi.com/2073-8994/12/5/862)** (L. Wang et al., 2020) – Explores task scheduling strategies for maximizing benefits in VCPs considering deadlines. +- πŸ‘₯ **[Retaining volunteers in volunteer computing projects](https://www.researchgate.net/publication/45461278_Retaining_volunteers_in_volunteer_computing_projects)** (D. Nov, O. Arazy, D. Anderson, 2011) – Investigates factors influencing volunteer retention and contribution levels. +- πŸ’‘ **[Personal Volunteer Computing](https://arxiv.org/vc/arxiv/papers/1804/1804.01482v1.pdf)** (S. Cahon et al., 2018) – Proposes a more personal, web-based approach to volunteer computing and introduces the Pando platform. +- ❀️ **[Volunteer computing: A model of the factors determining contribution to community-based scientific research](https://dl.acm.org/doi/abs/10.1145/1772690.1772766)** (O. Nov, D. Anderson, O. Arazy, 2010) – Develops and tests a model of user motivation based on SETI@home survey data and activity logs. + +--- + +## πŸ‘₯ Communities & Forums + +Places to ask questions, meet contributors, or follow development. + +- πŸ’¬ [BOINC Message Boards](https://boinc.berkeley.edu/dev/forum_index.php) +- πŸ“’ [Reddit: r/BOINC](https://www.reddit.com/r/BOINC/) +- πŸ—¨οΈ [BOINC Discord](https://discord.gg/boinc) +- *(Project-specific forums, like Folding@home's, also exist)* + +--- + +## πŸ“Ί Articles & Videos + +Resources to learn more about the real-world impact and concepts of volunteer computing. + +- πŸ“° [What is Volunteer Computing? – TechRadar](https://www.techradar.com/news/what-is-volunteer-computing) – General overview article. +- 🌐 [Volunteer computing - Wikipedia](https://en.wikipedia.org/wiki/Volunteer_computing) – Comprehensive encyclopedia article. +- CERN Explainer: [What is volunteer computing? | LHC@home](https://lhcathome.web.cern.ch/about/volunteer-computing) – Explanation from the perspective of a major project. +- Khan Academy: [Citizen science (article)](https://www.khanacademy.org/computing/computers-and-internet/xcae6f4a7ff015e7d:computing-innovations/xcae6f4a7ff015e7d:crowdsourcing-innovations/a/citizen-science) – Discusses volunteer computing as a form of citizen science. + +--- + +## πŸ”— Related Concepts + +Areas closely related to or overlapping with volunteer computing. + +- **Citizen Science:** Public participation in scientific research, often involving data collection or analysis. Volunteer computing is sometimes considered a sub-field. + - [Zooniverse](https://www.zooniverse.org/): Popular platform for web-based citizen science projects (classification, transcription). + - [SciStarter](https://scistarter.org/): Hub for discovering citizen science projects. +- **Distributed Computing:** General term for systems where components are located on different networked computers, communicating and coordinating actions. +- **Grid Computing:** Often involves more formal resource sharing agreements and infrastructure, typically within or between organizations, compared to the public volunteer model. + +--- + +## 🀝 Contributing + +Found a project we missed? Want to improve descriptions or links? Contributions welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for how to get started. + +--- + +## πŸ“„ License + +[CC0 1.0 Universal (Public Domain Dedication)](LICENSE) + +> Inspired by [awesome](https://awesome.re), the community-driven standard for curated knowledge.