Skip to content

Commit 4269792

Browse files
committed
Create celery.md
1 parent 77442c6 commit 4269792

File tree

1 file changed

+162
-0
lines changed

1 file changed

+162
-0
lines changed

docs/celery.md

Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
## Celery docs
2+
3+
### Celery introduction
4+
Celery is a tool that helps you manage tasks in the background, independent of the main application's workflow. Celery is a tool that helps you run time-consuming tasks without slowing down your application or making it unresponsive. Celery allows you to distribute tasks/jobs across several threads.
5+
6+
**Rocket Django** offers the `Async task manager` feature, which makes use of Celery to run scripts in the background without affecting the main application's performance. This is done by placing the scripts in a task queue, which Celery then manages and executes. This allows users to run time-consuming tasks without having to wait for them to finish before continuing with other tasks.
7+
8+
### Requirements
9+
Celery needs a way to send and receive messages, so you'll need to install a message broker like RabbitMQ or Redis.
10+
11+
**Rocket Django** is pre-configured to utilize Redis as its message broker, but you have the flexibility to switch to your preferred broker if needed.
12+
13+
### Installation and setup
14+
- Install the dependencies
15+
```bash
16+
$ pip install -r requirements.txt
17+
$ npm install
18+
```
19+
20+
- Copy the `env.sample` file and name it `.env`. You can do this using the following command in your command line
21+
```bash
22+
$ cp env.sample .env
23+
```
24+
25+
- Inside the `.env file`, create the variable `CELERY_BROKER`. If you are using Redis running on your computer, the value of `CELERY_BROKER` should be:
26+
```sh
27+
#.env
28+
...
29+
CELERY_BROKER="redis://localhost:6379"
30+
```
31+
32+
- To tell Celery where to find your Django settings, add `DJANGO_SETTINGS_MODULE` to your environmental variables. You can do this by opening a terminal window and running the following command:
33+
```bash
34+
$ export DJANGO_SETTINGS_MODULE=core.settings
35+
```
36+
37+
- The Celery configuration file, named `celery.py`, defines how Celery will operate within your Django project. It is located in your `home` directory.
38+
```py
39+
# home/celery.py
40+
...
41+
if os.environ.get('DJANGO_SETTINGS_MODULE'):
42+
43+
app = Celery('core')
44+
45+
# - namespace='CELERY' means all celery-related configuration keys should have a `CELERY_` prefix.
46+
app.config_from_object('django.conf:settings', namespace='CELERY')
47+
48+
# Load task modules from all registered Django apps.
49+
app.autodiscover_tasks()
50+
```
51+
52+
- The configuration options for celery can be found in `core/settings.py`. More configuration options can be found in Celery's [documentation](https://docs.celeryq.dev/en/stable/userguide/configuration.html)
53+
```py
54+
CELERY_SCRIPTS_DIR = os.path.join(BASE_DIR, "tasks_scripts" )
55+
56+
CELERY_LOGS_URL = "/tasks_logs/"
57+
CELERY_LOGS_DIR = os.path.join(BASE_DIR, "tasks_logs" )
58+
59+
CELERY_BROKER_URL = os.environ.get("CELERY_BROKER", "redis://localhost:6379")
60+
CELERY_RESULT_BACKEND = os.environ.get("CELERY_BROKER", "redis://localhost:6379")
61+
62+
CELERY_TASK_TRACK_STARTED = True
63+
CELERY_TASK_TIME_LIMIT = 30 * 60
64+
CELERY_CACHE_BACKEND = "django-cache"
65+
CELERY_RESULT_BACKEND = "django-db"
66+
CELERY_RESULT_EXTENDED = True
67+
CELERY_RESULT_EXPIRES = 60*60*24*30 # Results expire after 1 month
68+
CELERY_ACCEPT_CONTENT = ["json"]
69+
CELERY_TASK_SERIALIZER = 'json'
70+
CELERY_RESULT_SERIALIZER = 'json'
71+
```
72+
73+
To create your scripts, head over to the `tasks_scripts` folder within the base directory. Script files saved in this location can be executed using the `Async task manager` feature.
74+
75+
- In the root folder of the application, create a folder called `task_logs`. You can do that from the terminal using the command:
76+
```bash
77+
$ mkdir tasks_logs
78+
```
79+
80+
- Run the celery command from the terminal
81+
```bash
82+
$ celery -A home worker -l info -B
83+
```
84+
85+
- Run node server to allow the use of tailwind on another terminal
86+
```bash
87+
$ npm run dev
88+
```
89+
90+
- Run the Django server on a different terminal
91+
```bash
92+
$ python manage.py runserver
93+
```
94+
95+
Visit https://localhost:8000 to view the application.
96+
97+
![Rocket Django - Styled with Tailwind-Flowbite AppSeed](https://github.com/app-generator/dummy/assets/57325382/409d6211-d1ed-4be0-8fca-d5ea58693481)
98+
99+
- Under the App menu in the sidebar, you will see a new route called `Tasks`.
100+
101+
![Rocket Django Tasks Page - Styled with Tailwind-Flowbite AppSeed](https://github.com/app-generator/dummy/assets/57325382/d3b2ae6b-6971-4005-aec4-7aa95c7eac5a)
102+
103+
- You can start and cancel any task from the UI that exists as a script in the `tasks_scripts` folder.
104+
105+
### Adding a new script
106+
Django Celery allows you to create custom scripts that can be executed from the user interface (UI). These scripts can perform various tasks, such as backups, data processing, or sending emails.
107+
108+
- The first step is to locate the `tasks_scripts` directory within your project's base directory. This directory is where custom Celery scripts should be placed.
109+
110+
- Inside the `tasks_scripts` directory, create a new Python file. In this tutorial, we'll use the filename `backup_db.py`.
111+
```py
112+
# task_scripts/backup_db.py
113+
import os, shutil
114+
from datetime import datetime
115+
116+
def main():
117+
try:
118+
DB_LOCATION = "db.sqlite3"
119+
120+
BACKUP_NAME = f"db_backup_{datetime.now().strftime('%d_%m_%Y_%H_%M_%S')}.sqlite3"
121+
122+
print(BACKUP_NAME)
123+
124+
if not os.path.exists("db_backup"):
125+
os.mkdir("db_backup")
126+
127+
shutil.copyfile(DB_LOCATION, "db_backup/" + BACKUP_NAME)
128+
129+
exit(0)
130+
except Exception as e:
131+
print( 'Err: ' + str( e ) )
132+
exit(1)
133+
134+
if __name__ == "__main__":
135+
main()
136+
```
137+
138+
This script creates a backup of the current database in use by creating a copy of the database. Once you have added this script to the `tasks_scripts` folder, it can be executed from the application.
139+
140+
### Adding a new task
141+
Tasks to be executed by Celery can be added from the user interface of the application.
142+
143+
- Navigate to the `Tasks` section of the application. You can access it from the `Apps` menu on the sidebar.
144+
145+
- The scripts located in the `tasks_scripts` folder are already preloaded and ready to be executed.
146+
147+
- To execute tasks, you need to be logged in as an administrator. However, anyone can view the progress of tasks in execution.
148+
149+
![Rocket Django Tasks Page - Styled with Tailwind-Flowbite AppSeed](https://github.com/app-generator/dummy/assets/57325382/aad832d4-ff62-44a1-a973-23c42e13acd8)
150+
151+
On the tasks page, you can select and execute the desired tasks. The status of the last executed task and a history of previously executed tasks are displayed for your reference.
152+
153+
![Rocket Django Tasks Page - Styled with Tailwind-Flowbite AppSeed](https://github.com/app-generator/dummy/assets/57325382/cada9eb2-93ec-4f9c-85be-163798060471)
154+
155+
## Conclusion
156+
The Asynchronous task handler feature makes it easy to run time-consuming tasks without affecting the user experience. This can be helpful for tasks like sending emails, processing payments, or generating reports.
157+
158+
Rocket Django utilizing celery serves as a valuable tool for streamlining task processes and improving the overall user experience in your web development projects.
159+
160+
## ✅ Resources
161+
- 👉 [Celery](https://docs.celeryq.dev/en/stable/getting-started/introduction.html) official documentation
162+
- 👉 Join the [Community](https://discord.com/invite/fZC6hup) and chat with the team behind **Rocket Django**

0 commit comments

Comments
 (0)