Skip to content

RabbitMQ version to use

Jump to bottom
Giovanni Pizzi edited this page Jan 13, 2022 · 14 revisions

Summary

Unfortunately, RabbitMQ with versions >= 3.8 have changed and reduced some default timeouts, making it in practice incompatible with AiiDA and its design: AiiDA will work, but any workflow running for longer than 30 minutes (or whatever time is set by RabbitMQ) will Except.

For more details, you can check issue 5105 (and optionally the related #5278 and #5300).

As a consequence, while we find a longer-term solution (probably requiring to drop RabbitMQ and replace it with something else, that will require some development effort), we recommend to use a version of RabbitMQ <=3.7 (e.g. 3.7.28). Since this version is quite old, recent distributions might not even offer it, so we suggest here ways to use it.

Alternatively, one can use more recent versions of RabbitMQ adapting the timeouts; this however requires a bit of experience and requires to change some RabbitMQ configuration.

Using an old RabbitMQ version

In general, if you have a version of RabbitMQ <=3.7 (or you can install one on your computer in some way), then you are OK and you don't need to read further.

If however you are on a recent OS or distribution, the simplest way to use a compatible version of RabbitMQ is to run it via docker.

Instructions:

  1. Install docker e.g. via your package manager or following instructions here
  2. run the following command in the command line: 
    docker run --detach  --hostname aiida-rabbitmq --name aiida-rabbitmq-server --restart=unless-stopped --publish=127.0.0.1:5671:5671 --publish=127.0.0.1:5672:5672 --mount=type=volume,src=rabbitmq-volume,dst=/var/lib/rabbitmq rabbitmq:3.7.28

Some explanation on this long command:

  • It's important to specify a --hostname, see https://hub.docker.com/_/rabbitmq
  • We give a name to the container aiida-rabbitmq-server to easily identify it
  • We set the restart policy to unless-stopped (e.g. on failure or on reboot it restarts, if it was running before).
  • I publish the 2 relevant ports
  • I mount the data that RabbitMQ needs to store from a named docker volume (rabbitmq-volume), to persist messages between restarts
  • I start a 3.7 version (the latest available 3.7.28 at the moment of writing)

Note that the first time docker will have to pull the image; after that, subsequent restarts will be very fast.

Some additional notes on possible issues:

  • if, when starting, the command complains that the container named aiida-rabbitmq-server already exists, use docker kill aiida-rabbitmq-server (if still running and you really want to restart it), and then docker rm aiida-rabbitmq-server to remove the old one.

    • Another option is instead to not specify a --restart in the command, but put --rm to clean up the container automatically upon stopping. Then one does not need to run docker rm, but needs to restart the service at every failure or reboot.

  • If docker complains that the ports (5671, 5672) are already used, it means you already have some other rabbitmq server running in your machine (probably a system-wide installation). Either stop it (and make sure it does not restart at the next reboot); or change the ports in the command above to bind the ports to other ones. However, remember then to also configure AiiDA to connect to the correct Rabbitmq ports!

Clone this wiki locally