In order to get responses from our external services (just one for the beginning: watson-assistant-worker) we use the event emitter approach, pub/sub design → service task publish-outcome in my workflow (as explained by Josh Wulf in this article https://zeebe.io/blog/2019/08/zeebe-rest-affinity/)
As you can see in the bpmn we created a message intermediate catch event called “Message received” to handle user inputs.
I have 2 questions:
Does my workflow make sense to you ?
And I’d like to know if the service task (pub/sub) still the best practice to send events or if zeebe implemented a Message Intermediate Throwing Event as available in camunda (Message Events | docs.camunda.org) ?
There is a CreateWorkflowWithResult API call now (the feature we were evaluating mentioned in that article) - but this is only good for a workflow that runs to completion.
In this case, it looks like a conversation loop, so using a service task worker to publish the intermediate outcomes is the way to model this.
There is no baked in “emit message” worker in Zeebe. The only “push” communication it does is via job activation, and you have to roll your own worker for that and manage the correlation, as you’ve done.
We have issues when we create multiple replicas of the service that contains the pub/sub design system (publish-outcome on the schema) as designed in this doc:
It seems that when we create multiple replicas, the instance that subscribe using event emitter can be different from the instance publishing the response (publish outcome), in this case we lost the response.
Is it a problem you already identified ?
Is there a way to keep a link between the subscribing instance and the publishing instance ? or another pattern to send events that allow us to scale this part of our workflow ?
It’s not up-to-date, but it gives you the idea, and if you roll with this implementation, I’m happy to take a Pull Request. I can even move it to the Zeebe org so you can get credits toward https://camunda.com/hacktoberfest and get a free t-shirt!
We started using Knative on our project. We thought we could solve the affinity problem using knative eventing system but it doesnt.
If you could finish the pull request, i’ll start to create a centralised pub/sub using redis on our project and go for the free t-shirt.