Hi. We have RabbitMQ in place for message broker.
I found this link, but it seems to be more complicated when adding zeebe to rabbitmq.
In this case, which is the best approach for zeebe in microservices?
What is the pro-cons?
Thanks
Zeebe is a business process orchestrator. A message broker is for communication. They are separate concerns, although you can do peer-to-peer choreography over messages.
I can’t see a link in your post, so I’m not sure what you have read.
There is no simple answer to your question. It depends. There are a number of architectural patterns that you can use, and your operational requirements and your organisational structure are factors.
With more detail, I could give you a more detailed answer about your specific situation.
But if you want a dogmatic statement, then: “replace all message broker systems with Zeebe, because it is the new hotness” 
Well, we are migrating to microservices. All legacy systems were written on c#, with somemodules being rewritten on .net core.
Some newer systems were written on java spring boot. Data analytics teams works with python, although the data engineering team ingest data with java.
Most of our data flow involves customer entry system - voice system - core business systrm. The data comes back and forth among them.
Sometimes we also has satellite system that talks to core (e.g. getting existing customer transactions) or customer entry (e.g. from b2b partner through our open api).
For reliable publishing, we use transactional outbox
Outbox pattern we use for all publishers, with two message publisher engine works publishing data from multiple outbox table databases (around 20 right now). Basically they poll data from outbox tables every 2 second or so.
Most of our publishers/listeners works for 2-4 business process. The simplest one is sending data… E.g for every financial transaction then publish data to ledger system that will listens and create ledger entry, with small portion requires async response (thus we use coreography saga).
In case something wrong happened, we design our rabbitmq queues to retry n times (non blocking, error on data x will not delay processing data x+1) then if after certain retry and backoff, it still failed, we send data to dead letter queues. The dead letter processed by support team manually.
I’m looking for workflow like netflix conductor or zeebe, which seems interesting, but will it fit into our use case?
Also, can we reuse existing rabbitmq publishers / listeners?
Since official zeebe client is only on java and go, will our c# and python team can use this? Sometimes we need to talk from system P (java) to Q (c#).
How reliable is the c# or python community client?
Thanks for the detail @timpamungkas.
The C# and Python clients seem reliable, and they are not hard to patch / pull request if you find something that isn’t working for you.
The benefits of using Zeebe include reification of your business process, which enables you to inspect the current state of any instance in a GUI; reporting on overall system load / through-put metrics at the business level (by querying Elastic Search); and making business process changes at a high-level where BAs and management can look at the process diagram.
In a peer-to-peer choreography scenario, you need to deal with the eventuality where the recipient service is down.
In the Zeebe architecture, services pull work from the broker, so if a service is down, the current process state is in the broker until the service recovers.
Alerting when a service goes down can be done with service monitoring. If you want business alerting (this particular process instance is blocked and at risk of exceeding its SLA), then you need to model that with things like sub-processes a with boundary interrupt timer.
You do need to mitigate against the broker going away, and a message to the broker going into retry in a service, and then the service failing and losing its in-memory state.
In this case, the transactional outbox pattern could be used to deal with this. Rather than publishing messages directly to the broker, services could publish messages to the RabbitMQ service to be routed to the the broker.
Messages are used for decoupled asynchronous events. When you have a sequential event - some work pulled by a service, and then a response returned from the service before the process continues - you can model that with the job lifecycle: service activates a job, does some work, then completes the job, optionally with some update to the process state.
In terms of reusing RabbitMQ listeners/publishers: it depends.
A good way to approach this is to start to model your existing system in BPMN with the Zeebe Modeler, to get an idea of how a BPMN system is structured. You can get feedback on your model here, and rinse and repeat until you have a good grasp of what it could look like.
OK. I’ll try. Just found a problem about transaction compensation, but will open different topic for this
Thanks