This is a continuation of a Slack convo (available here) related to the removal of list and get workflows queries from the Zeebe API in 0.18.0. Because we consider this an important discussion about the product, we decided to move it to the forum so that it’s easier for new users to find and easier to reference in the future.
The last comment of that conversation was:
as a developer I find it very helpful to have symmetric APIs whenever feasible. if I can create a thing, I should be able to list those things too. If zeebe doesn’t support listing workflows, then every developer has to implement their own system for tracking what’s been deployed. That seems like an impediment to adoption. Further, recovery from system failures becomes more difficult. Is it truly a binary choice that ‘all workflows were lost’ or ‘no workflows were lost’? Perhaps this doesn’t matter in this case as users could simply redeploy every workflow they think they should have… ?
First of all thank you for your feedback. It is very valuable for us and helps us to shape the future of Zeebe. Regarding the current discussion of removing query APIs from Zeebe 0.18 I want to give you some background information which might help you to understand our decision. Zeebe is not designed to be a queryable state store, instead it focuses on the execution of stateful workflows. The only existing query APIs (list/get workflow) were initially implemented as a temporary solution before the exporter API was available. We now decided to remove them before the GA as they were inconsistent with the remaining API, i.e. no query APIs for workflow instances and jobs. This doesn’t mean that we will never implement a query API, but we’re not going to offer it right now.
At the moment the architecture of Zeebe enables users to query state by exporting the event stream to their preferred data stores which already have clients and query APIs in place. This removes a lot of complexity from the initial Zeebe scope. From my perspective, a usable implementation of a query API on top of Zeebe requires handling partitioned data sets and pagination/filtering support. I don’t want to say that this will never be implemented but at the moment is out of scope for the first GA.
maybe monitor should be split to two parts, one for event center and one for monitor ui
event center should be pure and independent. put it to broker is not a good choice in my opinion, it should be focus on storing/querying, it can be based on exporter.
event store should be released officially, but monitor ui maybe communal.
Our requirement is that we must know with certainty the exact bpm.xml that will be used when we instantiate a workflow. Currently, the only way to guarantee the bpm.xml file that will be used is to deploy the bpm.xml file every time we instantiate a workflow and to instantiate the workflow with the WorkflowKey returned.
Our feature request is for Zeebe, upon deploy of a workflow, to store and return the GIT hash for the bpm.xml file. Then, when creating a workflow instance, we could provide the file hash and the BPMNProcessId.
Why don’t you store the deployment version and git hash in your own caching layer (like a db)?
That can happen today. This feature request (if you open it in the GitHub and get community support around it) would probably take a few months to land.
Thanks for the suggestion. However, creating a caching layer would only guarantee we know which bpm.xml file will be used is if the only way to deploy and instantiate a workflow was through the caching layer. That would be an enormous effort to implement.
class MyZBClient extends ZBClient {
constructor(...args: any[]) {
super(...args)
}
deployworkflow(...args: any[) {
const res = await super.deployWorkflow(...args)
// cache res and bpmn to database
return res
}
}
And then publish a custom package and use that to deploy bpmn.
It is soluble with some low orbit engineering right now, in a number of ways.
Another option, that doesn’t rely on a modified client, (and this is the recommended architecture in @menski’s original comment) is to listen to the exporter and cache the deployment from there - then anyone can deploy with any client and you still get the deployment record to cache.
The other option is to open a feature request in the Zeebe GitHub issues, and get community input there. Feature requests with input / use-cases / popular support definitely influence the product roadmap.
The lack of query API is ridiculous.
How ironic for Zeebe to be advertised as a microservice orchestrator and yet it itself does not adhere to mciroservice principals of a self contained service with an API.
A query API is coming. With Camunda Cloud 1.3.0, we introduced the first iteration for query deployed processes. Have a look at the release blog post: Camunda Cloud 1.3.0 Released - Camunda.
