I’m exploring Zeebe and like product ambition. In my prototype I’ve tried to setup a Kibana dashboard to display mean time to resolve metric for a workflow. So far I’ve been unsuccessful and am looking for recommended approach.
Attempt 1: creating a elastic search query that uses timestamp to calculate MTTRx (Rx=resolve)
I was successful creating the query by painless scripting that summarises timestamp deltas over the elastic documents. In console view I can get MTTRx from scripted query. The code is not very readable and therefor prone to errors. However none of this matters since Kibana doesn’t support displaying cross-document queries so there is no way for me to create a MTTRx dashboard this way.
Attempt 2: Calcuate MTTRx myself and create dashboard based on workflow instance payload
I built a micro services that adds a timestamp as a first and last process step and store the timestamps in the payload. I also calculate a TTRx variable which is the diff between the end and start timestamps. So far so good. I Kibana I run into two problems: 1) the metadata.valueType == VARIABLE is created for the TTRx payload variable however it has no info on which workflow it belongs to. So I would only be able to calculate MTTRx overall all my workflows as one bucket which holds little to no value. 2) the value.value field is of type “text” and not “keyword” which implies that is not “aggregatable” which means I can’t create a scripted field to convert to integer or time or even create a dashboard based on it.
thanks for exploring Zeebe. We also noticed that it is not easily possible to get such metrics from the current elastic indexes.
Instead of using the existing indices, you could either create a new exporter which creates an index specific to your use case. Or you could have a small application which aggregates the current index and creates a new metric index.
We can also discuss if Zeebe should provide something out of the box here, therefore your input is greatly appreciated.
Regarding your second attempt, I have some questions so we might incorporate your feedback in one of the next releases
the metadata.valueType == VARIABLE is created for the TTRx payload variable however it has no info on which workflow it belongs to
If I understand you correctly you are missing a workflowKey reference?
the value.value field is of type “text” and not “keyword”
We will discuss this limitation, and might change the type. As the variables concept is quite new this feedback is really helpful. Thanks.
Ideally I would like to have the bpmnProcessId available from the VARIABLE document. Then (given that the “text” type of value.value is changed) it would be very easy to create a dashboard of MTTRx for each process.
workflowKey would be better than nothing but then I would need to maintain a key -> bpmn-name mapping outside of zeebe and create a scripted field in Kibana to use that mapping to add BPMN name to all variables in order for the data to be complete to create a visualisation.
Wanted to let you know that we opened an issue where we’re starting to collect ideas on a different indexing approach for an Elasticsearch exporter that would enable more reporting in Kibana: https://github.com/zeebe-io/zeebe/issues/2196