Workflow with Message Intermediate Throwing Event

Hi there,

Pretty new to Camunda products, looking for some direction.

We are building a chatbot with Zeebe using REST req/response from external services.

In order to get responses from our external services (just one for the beginning: watson-assistant-worker) we use the event emitter approach, pub/sub design → service task publish-outcome in my workflow (as explained by Josh Wulf in this article https://zeebe.io/blog/2019/08/zeebe-rest-affinity/)
As you can see in the bpmn we created a message intermediate catch event called “Message received” to handle user inputs.

I have 2 questions:

  • Does my workflow make sense to you ?
  • And I’d like to know if the service task (pub/sub) still the best practice to send events or if zeebe implemented a Message Intermediate Throwing Event as available in camunda (Message Events | docs.camunda.org) ?

Thanks for your help
Roland

2 Likes

Hi Roland,

Looks good!

There is a CreateWorkflowWithResult API call now (the feature we were evaluating mentioned in that article) - but this is only good for a workflow that runs to completion.

In this case, it looks like a conversation loop, so using a service task worker to publish the intermediate outcomes is the way to model this.

There is no baked in “emit message” worker in Zeebe. The only “push” communication it does is via job activation, and you have to roll your own worker for that and manage the correlation, as you’ve done.

Josh

1 Like

Hey Josh,

Thanks for your reply!

We have issues when we create multiple replicas of the service that contains the pub/sub design system (publish-outcome on the schema) as designed in this doc:

It seems that when we create multiple replicas, the instance that subscribe using event emitter can be different from the instance publishing the response (publish outcome), in this case we lost the response.

Is it a problem you already identified ?
Is there a way to keep a link between the subscribing instance and the publishing instance ? or another pattern to send events that allow us to scale this part of our workflow ?

thanks
Roland

1 Like

Yes, you need a centralised pub/sub using Redis or websockets. Here is a POC: https://github.com/jwulf/zeebe-affinity-service

It’s not up-to-date, but it gives you the idea, and if you roll with this implementation, I’m happy to take a Pull Request. I can even move it to the Zeebe org so you can get credits toward https://camunda.com/hacktoberfest and get a free t-shirt!

1 Like

Thanks Josh

I tested the implementation and created a pull request on zeebe-affinity-service https://github.com/jwulf/zeebe-affinity-service/pulls, to update zeebe-node to latest version.

We started using Knative on our project. We thought we could solve the affinity problem using knative eventing system but it doesnt.
If you could finish the pull request, i’ll start to create a centralised pub/sub using redis on our project and go for the free t-shirt.

1 Like

Thanks for that. I merged the PR and published a new version of the package.

It doesn’t have any tests, so let’s see if it still works!

Hello Josh,
I created a PR on zeebe-affinity-service for the redis pub/sub affinity system

1 Like

Epic! Thanks for this. I commented on it.

In case you want to build a chatbot then I think it would be a good idea to read some guides that explain how to do this properly.