diff --git a/guidelines/message-broker.md b/guidelines/message-broker.md new file mode 100644 index 0000000..4546200 --- /dev/null +++ b/guidelines/message-broker.md @@ -0,0 +1,139 @@ +# MONAI Deploy Message Broker Requirements/Design + +## Background + +The MONAI Deploy sub-systems, including the Workflow Manager and the Informatics Gateway, use shared virtual storage for sharing data. With the given design, a message broker service must notify all sub-systems when data is available or when an event has occurred. + +This document describes and decides on a **default** message broker service to be used in the MONAI Deploy project and defines message topics/events used for publishing and subscribing events. + +**Important**: users shall be able to replace the **default** message broker service with their choice, and the design shall consider both on-prem, cloud, and edge deployments. + +## Message Broker Services + +- [RabbitMQ](https://www.rabbitmq.com/) (default) +- [KubeMQ Server](https://kubemq.io/) +- [Google Cloud Pub/Sub](https://cloud.google.com/pubsub/) +- [Apache Kafka](https://kafka.apache.org/intro) + +## Topics + +This section defines topics each sub-system publishes in MONAI Deploy. + +### Informatics Gateway + +* WorkflowRequest (workflow.request) +* ExportTaskCompleted (export.completed) + +### Workflow Manager + +* ExportTaskRequest (export.request) +* TaskRequest (task.request) +* WorkflowStarted (workflow.started) +* WorkflowCompleted (workflow.completed) + +### Task Manager (Part of Workflow Manager) + +* TaskStarted (task.started) +* TaskCompleted (task.completed) + +## Topic Payload Specs + +This section defines the detailed payload of each topic. + +### WorkflowRequest +| Field | Type | Description | +|-|-|-| +| Transaction ID | string or UUID | a unique ID to locate the workflow. Used and generated by the event publishers. | +| Correlation ID | string or UUID | a unique ID that traces back to the original request, e.g., a DICOM association | +| Files | string[] | a list of file paths (URN) associated with the request | +| Metadata | string[] | JSON formatted strings associated with the request. E.g., DICOM headers | +| Workflows | string[] | *optional* - a list of workflows to execute | + +### ExportTaskCompleted + +| Field | Type | Description | +|-|-|-| +| Export Task ID | string or UUID | a unique ID to locate the export task. Generated by the Workflow Manager. | +| Correlation ID | string or UUID | a unique ID that traces back to the original request, e.g., a DICOM association. | +| Transaction ID | string or UUID | the unique ID of the actual workflow request. | +| Status | int (enum )| Status of the export task | +| Message | string | status details | +| Sink | string | Sink assigned for the export task | + +#### ExportTask + +| Field | Type | Description | +|-|-|-| +| Export Task ID | string or UUID | a unique ID to locate the export task. Generated by the Workflow Manager. | +| Correlation ID | string or UUID | a unique ID that traces back to the original request, e.g., a DICOM association. | +| Transaction ID | string or UUID | the unique ID of the actual workflow request. | +| Files | string[] | a list of file paths (URN) associated with the request | +| Sink | string | Sink assigned for the export task | + +#### TaskRequest + +| Field | Type | Description | +|-|-|-| +| Transaction ID | string or UUID | a unique ID to locate the workflow. Used and generated by the event publishers. | +| Correlation ID | string or UUID | a unique ID that traces back to the original request, e.g., a DICOM association | +| Payload Path | string | URN to the task payload | + +TBD + +#### WorkflowStarted + +| Field | Type | Description | +|-|-|-| +|-|-|-| +| Transaction ID | string or UUID | a unique ID to locate the workflow. Used and generated by the event publishers. | +| Correlation ID | string or UUID | a unique ID that traces back to the original request, e.g., a DICOM association | +| Status | int (enum) | current status of the workflow | +| State | int (enum) | current state of the workflow | +| DateStarted | datetime | Date time the workflow started | + +TBD + +#### WorkflowCompleted + +| Field | Type | Description | +|-|-|-| +|-|-|-| +| Transaction ID | string or UUID | a unique ID to locate the workflow. Used and generated by the event publishers. | +| Correlation ID | string or UUID | a unique ID that traces back to the original request, e.g., a DICOM association | +| Status | int (enum) | current status of the workflow | +| State | int (enum) | current state of the workflow | +| DateStarted | datetime | Date time the workflow started | +| DateCompleted | datetime | Date time the workflow completed | + + +TBD + +#### TaskStarted + +| Field | Type | Description | +|-|-|-| +| Transaction ID | string or UUID | a unique ID to locate the workflow. Used and generated by the event publishers. | +| Correlation ID | string or UUID | a unique ID that traces back to the original request, e.g., a DICOM association | +| Payload Path | string | URN to the task payload | +| Status | int (enum) | current status of the task | +| State | int (enum) | current state of the task | +| DateStarted | datetime | Date time the task started | + + +TBD + +#### TaskCompleted + +| Field | Type | Description | +|-|-|-| +| Transaction ID | string or UUID | a unique ID to locate the workflow. Used and generated by the event publishers. | +| Correlation ID | string or UUID | a unique ID that traces back to the original request, e.g., a DICOM association | +| Payload Path | string | URN to the task payload | +| Status | int (enum) | current status of the task | +| State | int (enum) | current state of the task | +| DateStarted | datetime | Date time the task started | +| DateCompleted | datetime | Date time the task completed | + + + +TBD