How do I structure a system with Zeebe and RabbitMQ?

Josh Wulf: Here is the complete round-trip in one place:

  1. Workflow instance is started via publishMessage or createWorkflowInstance
  2. BPMN flow enters task node.
  3. Zeebe worker fetches job for this task-type from the engine, and constructs request, appending the Zeebe Job Id for eventual correlation at completion.
  4. Zeebe worker publishes message to the appropriate RabbitMQ req/command channel.
  5. Remote service/subscriber picks up relevant message from RabbitMQ req/command channel and processes request.
  6. Remote service publishes response to RabbitMQ res channel.
  7. RabbitMQ worker picks up response (which has Zeebe Job Id for correlation) and completes job in Zeebe (“Zeebe fulfiller”). Optionally, publish a message with a correlation value to model the service return state in your BPMN using event-based gateways. Use a standardised pattern for this to keep the adapter layer generic.
  8. Zeebe engine moves on to next BPMN node.
  9. Repeat from step 2 until done.

See also:

Note: This post was generated by Slack Archivist from a conversation in the Zeebe Slack, a source of valuable discussions on Zeebe (get an invite). Someone in the Slack thought this was worth sharing!

If this post answered a question for you, hit the Like button - we use that to assess which posts to put into docs.

2 Likes

After step 4, what happens to the worker? You need to either call complete or failure. How does the job knows to wait for the completion from another remote service?

1 Like

The worker, in the Node example, is a callback function. It runs to completion and the execution of the worker handler exits.

The worker is not responsible for communicating the completion to the broker in this model. That responsibility is elsewhere in the system.

Having said that, I think my assumption when I wrote the Node client was that it would call one of the complete functions, so its capacity will reduce over time, because the number of activated jobs returned to the worker reduces the maxJobs it asks for in the activateJobs request:

capacity - activatedJobs + number of calls to a complete function => maxJobs in the ActivateJobsRequest

It may have a bug in this scenario. I’ll check it out.

0.23.0-alpha.1 of the Zeebe Node client (coming real soon) adds a method to the worker handler callback: complete.forwarded(). Its only effects are to declare the job as forwarded to an external system in the code (for humans) and increment worker available capacity by one (for the machine).

Zeebe worker publishes message to the appropriate RabbitMQ req/command channel - Do we need a connector (such as Kafka connect) or zeebe brings this out of the box to connect to any AMQP based messaging system ?

1 Like

Hi @omarrahul, Zeebe does not have any AMQP connector baked in. You have to put it in a worker.

Josh