|
| 1 | +## Celery docs |
| 2 | + |
| 3 | +### Celery introduction |
| 4 | +Celery is a tool that helps you manage tasks in the background, independent of the main application's workflow. Celery is a tool that helps you run time-consuming tasks without slowing down your application or making it unresponsive. Celery allows you to distribute tasks/jobs across several threads. |
| 5 | + |
| 6 | +**Rocket Django** offers the `Async task manager` feature, which makes use of Celery to run scripts in the background without affecting the main application's performance. This is done by placing the scripts in a task queue, which Celery then manages and executes. This allows users to run time-consuming tasks without having to wait for them to finish before continuing with other tasks. |
| 7 | + |
| 8 | +### Requirements |
| 9 | +Celery needs a way to send and receive messages, so you'll need to install a message broker like RabbitMQ or Redis. |
| 10 | + |
| 11 | +**Rocket Django** is pre-configured to utilize Redis as its message broker, but you have the flexibility to switch to your preferred broker if needed. |
| 12 | + |
| 13 | +### Installation and setup |
| 14 | +- Install the dependencies |
| 15 | +```bash |
| 16 | +$ pip install -r requirements.txt |
| 17 | +$ npm install |
| 18 | +``` |
| 19 | + |
| 20 | +- Copy the `env.sample` file and name it `.env`. You can do this using the following command in your command line |
| 21 | +```bash |
| 22 | +$ cp env.sample .env |
| 23 | +``` |
| 24 | + |
| 25 | +- Inside the `.env file`, create the variable `CELERY_BROKER`. If you are using Redis running on your computer, the value of `CELERY_BROKER` should be: |
| 26 | +```sh |
| 27 | +#.env |
| 28 | +... |
| 29 | +CELERY_BROKER="redis://localhost:6379" |
| 30 | +``` |
| 31 | + |
| 32 | +- To tell Celery where to find your Django settings, add `DJANGO_SETTINGS_MODULE` to your environmental variables. You can do this by opening a terminal window and running the following command: |
| 33 | +```bash |
| 34 | +$ export DJANGO_SETTINGS_MODULE=core.settings |
| 35 | +``` |
| 36 | + |
| 37 | +- The Celery configuration file, named `celery.py`, defines how Celery will operate within your Django project. It is located in your `home` directory. |
| 38 | +```py |
| 39 | +# home/celery.py |
| 40 | +... |
| 41 | +if os.environ.get('DJANGO_SETTINGS_MODULE'): |
| 42 | + |
| 43 | + app = Celery('core') |
| 44 | + |
| 45 | + # - namespace='CELERY' means all celery-related configuration keys should have a `CELERY_` prefix. |
| 46 | + app.config_from_object('django.conf:settings', namespace='CELERY') |
| 47 | + |
| 48 | + # Load task modules from all registered Django apps. |
| 49 | + app.autodiscover_tasks() |
| 50 | +``` |
| 51 | + |
| 52 | +- The configuration options for celery can be found in `core/settings.py`. More configuration options can be found in Celery's [documentation](https://docs.celeryq.dev/en/stable/userguide/configuration.html) |
| 53 | +```py |
| 54 | +CELERY_SCRIPTS_DIR = os.path.join(BASE_DIR, "tasks_scripts" ) |
| 55 | + |
| 56 | +CELERY_LOGS_URL = "/tasks_logs/" |
| 57 | +CELERY_LOGS_DIR = os.path.join(BASE_DIR, "tasks_logs" ) |
| 58 | + |
| 59 | +CELERY_BROKER_URL = os.environ.get("CELERY_BROKER", "redis://localhost:6379") |
| 60 | +CELERY_RESULT_BACKEND = os.environ.get("CELERY_BROKER", "redis://localhost:6379") |
| 61 | + |
| 62 | +CELERY_TASK_TRACK_STARTED = True |
| 63 | +CELERY_TASK_TIME_LIMIT = 30 * 60 |
| 64 | +CELERY_CACHE_BACKEND = "django-cache" |
| 65 | +CELERY_RESULT_BACKEND = "django-db" |
| 66 | +CELERY_RESULT_EXTENDED = True |
| 67 | +CELERY_RESULT_EXPIRES = 60*60*24*30 # Results expire after 1 month |
| 68 | +CELERY_ACCEPT_CONTENT = ["json"] |
| 69 | +CELERY_TASK_SERIALIZER = 'json' |
| 70 | +CELERY_RESULT_SERIALIZER = 'json' |
| 71 | +``` |
| 72 | + |
| 73 | +To create your scripts, head over to the `tasks_scripts` folder within the base directory. Script files saved in this location can be executed using the `Async task manager` feature. |
| 74 | + |
| 75 | +- In the root folder of the application, create a folder called `task_logs`. You can do that from the terminal using the command: |
| 76 | +```bash |
| 77 | +$ mkdir tasks_logs |
| 78 | +``` |
| 79 | + |
| 80 | +- Run the celery command from the terminal |
| 81 | +```bash |
| 82 | +$ celery -A home worker -l info -B |
| 83 | +``` |
| 84 | + |
| 85 | +- Run node server to allow the use of tailwind on another terminal |
| 86 | +```bash |
| 87 | +$ npm run dev |
| 88 | +``` |
| 89 | + |
| 90 | +- Run the Django server on a different terminal |
| 91 | +```bash |
| 92 | +$ python manage.py runserver |
| 93 | +``` |
| 94 | + |
| 95 | +Visit https://localhost:8000 to view the application. |
| 96 | + |
| 97 | + |
| 98 | + |
| 99 | +- Under the App menu in the sidebar, you will see a new route called `Tasks`. |
| 100 | + |
| 101 | + |
| 102 | + |
| 103 | +- You can start and cancel any task from the UI that exists as a script in the `tasks_scripts` folder. |
| 104 | + |
| 105 | +### Adding a new script |
| 106 | +Django Celery allows you to create custom scripts that can be executed from the user interface (UI). These scripts can perform various tasks, such as backups, data processing, or sending emails. |
| 107 | + |
| 108 | +- The first step is to locate the `tasks_scripts` directory within your project's base directory. This directory is where custom Celery scripts should be placed. |
| 109 | + |
| 110 | +- Inside the `tasks_scripts` directory, create a new Python file. In this tutorial, we'll use the filename `backup_db.py`. |
| 111 | +```py |
| 112 | +# task_scripts/backup_db.py |
| 113 | +import os, shutil |
| 114 | +from datetime import datetime |
| 115 | + |
| 116 | +def main(): |
| 117 | + try: |
| 118 | + DB_LOCATION = "db.sqlite3" |
| 119 | + |
| 120 | + BACKUP_NAME = f"db_backup_{datetime.now().strftime('%d_%m_%Y_%H_%M_%S')}.sqlite3" |
| 121 | + |
| 122 | + print(BACKUP_NAME) |
| 123 | + |
| 124 | + if not os.path.exists("db_backup"): |
| 125 | + os.mkdir("db_backup") |
| 126 | + |
| 127 | + shutil.copyfile(DB_LOCATION, "db_backup/" + BACKUP_NAME) |
| 128 | + |
| 129 | + exit(0) |
| 130 | + except Exception as e: |
| 131 | + print( 'Err: ' + str( e ) ) |
| 132 | + exit(1) |
| 133 | + |
| 134 | +if __name__ == "__main__": |
| 135 | + main() |
| 136 | +``` |
| 137 | + |
| 138 | +This script creates a backup of the current database in use by creating a copy of the database. Once you have added this script to the `tasks_scripts` folder, it can be executed from the application. |
| 139 | + |
| 140 | +### Adding a new task |
| 141 | +Tasks to be executed by Celery can be added from the user interface of the application. |
| 142 | + |
| 143 | +- Navigate to the `Tasks` section of the application. You can access it from the `Apps` menu on the sidebar. |
| 144 | + |
| 145 | +- The scripts located in the `tasks_scripts` folder are already preloaded and ready to be executed. |
| 146 | + |
| 147 | +- To execute tasks, you need to be logged in as an administrator. However, anyone can view the progress of tasks in execution. |
| 148 | + |
| 149 | + |
| 150 | + |
| 151 | +On the tasks page, you can select and execute the desired tasks. The status of the last executed task and a history of previously executed tasks are displayed for your reference. |
| 152 | + |
| 153 | + |
| 154 | + |
| 155 | +## Conclusion |
| 156 | +The Asynchronous task handler feature makes it easy to run time-consuming tasks without affecting the user experience. This can be helpful for tasks like sending emails, processing payments, or generating reports. |
| 157 | + |
| 158 | +Rocket Django utilizing celery serves as a valuable tool for streamlining task processes and improving the overall user experience in your web development projects. |
| 159 | + |
| 160 | +## ✅ Resources |
| 161 | +- 👉 [Celery](https://docs.celeryq.dev/en/stable/getting-started/introduction.html) official documentation |
| 162 | +- 👉 Join the [Community](https://discord.com/invite/fZC6hup) and chat with the team behind **Rocket Django** |
0 commit comments