Skip to content

Commit 51a2803

Browse files
committed
SIANXSVC-826: Added direct-reply result backend
1 parent c617b06 commit 51a2803

File tree

8 files changed

+359
-20
lines changed

8 files changed

+359
-20
lines changed

.github/workflows/test.yml

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,14 +13,14 @@ jobs:
1313
fail-fast: false
1414
matrix:
1515
python-version:
16-
- "3.6"
1716
- "3.7"
1817
- "3.8"
1918
- "3.9"
2019
- "3.10"
2120
celery-version:
2221
- "5.0"
2322
- "5.1"
23+
- "5.2"
2424

2525
steps:
2626
- uses: actions/checkout@v2
@@ -51,7 +51,8 @@ jobs:
5151
ln -s ./tests/test_project/manage.py manage.py
5252
5353
# run tests with coverage
54-
coverage run --source='./celery_amqp_backend' manage.py test
54+
coverage run --append --source='./celery_amqp_backend' manage.py test --settings=test_project.settings.backend
55+
coverage run --append --source='./celery_amqp_backend' manage.py test --settings=test_project.settings.direct_reply_backend
5556
coverage xml
5657
5758
- name: Upload coverage to Codecov

README.md

Lines changed: 34 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -5,16 +5,26 @@ celery-amqp-backend
55
[![Test Status](https://github.com/anexia/celery-amqp-backend/actions/workflows/test.yml/badge.svg?branch=main)](https://github.com/anexia/celery-amqp-backend/actions/workflows/test.yml)
66
[![Codecov](https://codecov.io/gh/anexia/celery-amqp-backend/branch/main/graph/badge.svg)](https://codecov.io/gh/anexia/celery-amqp-backend)
77

8-
`celery-amqp-backend` is a rewrite of the Celery's original `amqp://` result backend, which was removed from Celery
9-
with version 5.0. Celery encourages you to use the newer `rpc://` result backend, as it does not create a new
10-
result queue for each task and thus is faster in many circumstances. However, it's not always possible to switch
11-
to the new `rpc://` result backend, as it does have restrictions as follows:
8+
`celery-amqp-backend` contains two result backens for Celery.
9+
10+
# `AMQPBackend` result backend
11+
12+
The `AMQPBackend` result backend is a rewrite of the Celery's original `amqp://` result backend, which was removed from
13+
Celery with version 5.0. Celery encourages you to use the newer `rpc://` result backend, as it does not create a new
14+
result queue for each task and thus is faster in many circumstances. However, it's not always possible to switch to the
15+
new `rpc://` result backend, as it does have restrictions as follows:
1216
- `rpc://` does not support chords.
1317
- `rpc://` results may hold a wrong state.
1418
- `rpc://` may lose results when using `gevent` or `greenlet`.
1519

1620
The result backend `celery_amqp_backend.AMQPBackend://` does not suffer from the same issues.
1721

22+
# `DirectReplyAMQPBackend` result backend
23+
24+
The `DirectReplyAMQPBackend` result backend makes use of RabbitMQ's direct-reply feature. It is much faster than the
25+
traditional `AMQPBackend` result backend and should even beat Celery's built-in `rpc://` result backend. However,
26+
contrary to the `AMQPBackend` result backend it does not support chords.
27+
1828
# Installation
1929

2030
With a [correctly configured](https://pipenv.pypa.io/en/latest/basics/#basic-usage-of-pipenv) `pipenv` toolchain:
@@ -29,7 +39,7 @@ You may also use classic `pip` to install the package:
2939
pip install celery-amqp-backend
3040
```
3141

32-
# Getting started
42+
# Getting started with `AMQPBackend`
3343

3444
## Configuration options
3545

@@ -57,6 +67,19 @@ Default: `'direct'`
5767

5868
The type of the exchange created by the backend (e.g. `'direct'`, `'topic'` etc.).
5969

70+
# Getting started with `DirectReplyAMQPBackend`
71+
72+
## Important notes
73+
74+
* You must set the `reply_to` property of Celery tasks to `"amq.rabbitmq.reply-to"`.
75+
* The `DirectReplyAMQPBackend` does not support chords.
76+
77+
## Configuration options
78+
79+
### `result_backend: str`
80+
81+
Set to `'celery_amqp_backend.DirectReplyAMQPBackend://'` to use this result backend.
82+
6083
## Example configuration
6184

6285
```python
@@ -68,13 +91,12 @@ result_exchange_type = 'direct'
6891

6992
# Supported versions
7093

71-
| | Celery 5.0 | Celery 5.1 |
72-
|-------------|------------|------------|
73-
| Python 3.6 |||
74-
| Python 3.7 |||
75-
| Python 3.8 |||
76-
| Python 3.9 |||
77-
| Python 3.10 |||
94+
| | Celery 5.0 | Celery 5.1 | Celery 5.2 |
95+
|-------------|------------|------------|------------|
96+
| Python 3.7 ||||
97+
| Python 3.8 ||||
98+
| Python 3.9 ||||
99+
| Python 3.10 ||||
78100

79101
# List of developers
80102

celery_amqp_backend/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
from .exceptions import *
21
from .backend import *
2+
from .exceptions import *

0 commit comments

Comments
 (0)