Skip to content

RabbitMQ version to use

Jump to bottom
Leopold Talirz edited this page Mar 4, 2022 · 14 revisions

Summary

Unfortunately, RabbitMQ with versions >= 3.8 have reduced some default timeouts, making it in practice incompatible with AiiDA and its design: AiiDA will work, but any workflow running for longer than 30 minutes (or whatever timeout is set by RabbitMQ) will except.

For more details, you can check issue #5105 (and optionally the related #5278 and #5300).

When using RabbitMQ >=3.8 you may edit a configuration file to change the timeout configured by default (see below). Otherwise, you can try installing RabbitMQ <=3.7 (e.g. 3.7.28). Since this version is somewhat dated, we offer some suggestions on how to install it.

In the medium term, we may need to find a different solution (probably requiring to drop RabbitMQ and replace it with something else, that will require some development effort).

Using a recent RabbitMQ version

These are instructions for Ubuntu

  • Become the root user (sudo su root)
  • Create /etc/rabbitmq/rabbitmq.conf with content (see docs)
# 100 hours in milliseconds (increase if you expect your workflows to run longer)
consumer_timeout = 3600000
  • Restart rabbitmq via service rabbitmq-server restart
  • Run rabbitmq-diagnostics status and check that your new configuration file is listed under the "Config files" section
  • TBD If you already have started AiiDA, it might have already created queues with the small timeouts and won't allow you to change those and fail (aiida-core docker container: remove the ~/.rabbitmq folder )

For more details, see also this issue comment.

Using an old RabbitMQ version

In general, if you have a version of RabbitMQ <=3.7 (or you can install one on your computer in some way), then you are OK and you don't need to read further.

If however you are on a recent OS or distribution, the simplest way to use a compatible version of RabbitMQ is to run it via docker.

Instructions

  1. Install docker e.g. via your package manager or following instructions here
  2. run the following command in the command line: 
    docker run --detach  --hostname aiida-rabbitmq --name aiida-rabbitmq-server --restart=unless-stopped --publish=127.0.0.1:5671:5671 --publish=127.0.0.1:5672:5672 --mount=type=volume,src=rabbitmq-volume,dst=/var/lib/rabbitmq rabbitmq:3.7.28

Some explanation on the command

  • It's important to specify a --hostname, see https://hub.docker.com/_/rabbitmq
  • We give a name to the container aiida-rabbitmq-server to easily identify it
  • We set the restart policy to unless-stopped (e.g. on failure or on reboot it restarts, if it was running before).
  • I publish the 2 relevant ports
  • I mount the data that RabbitMQ needs to store from a named docker volume (rabbitmq-volume), to persist messages between restarts
  • I start a 3.7 version (the latest available 3.7.28 at the moment of writing)

Note that the first time docker will have to pull the image; after that, subsequent restarts will be very fast.

Troubleshooting

  • if, when starting, the command complains that the container named aiida-rabbitmq-server already exists, use docker kill aiida-rabbitmq-server (if still running and you really want to restart it), and then docker rm aiida-rabbitmq-server to remove the old one.

    • Another option is instead to not specify a --restart in the command, but put --rm to clean up the container automatically upon stopping. Then one does not need to run docker rm, but needs to restart the service at every failure or reboot.

  • If docker complains that the ports (5671, 5672) are already used, it means you already have some other rabbitmq server running in your machine (probably a system-wide installation). Either stop it (and make sure it does not restart at the next reboot); or change the ports in the command above to bind the ports to other ones. However, remember then to also configure AiiDA to connect to the correct Rabbitmq ports!

Clone this wiki locally