Hey. In Zeebe 0.22, it became possible to wait for the results of a workflow with a specific timeout. In the case of a timeout, we do not get the context of the workflow and even the identifier of the instance of the workflow, as I understand it https://zeebe.io/blog/2019/10/0.22-awaitable-outcomes/. So we can’t get an intermediate result.
Is there a possibility in Zeebe to get an intermediate result of the context of the workflow in cases: if the waiting time is over, an error occurs, some worker went into retray, the processing time of the workflow can dynamically increase depending on the parameters or even freeze up?
Hi @dmkost, unfortunately right now it is one or the other. You can either wait for it, or get a handle for it.
As a potential workaround, you can increase the timeout for the create workflow request. You could make it ridiculously long - that way it will not timeout and lose the handle.
And then you would wrap the operation is a timer that can deal with a process that is taking longer than expected / acceptable, without losing the eventual handle.
Thanks @jwulf . This can solve some of the problems But for some cases it is interesting for us to get an intermediate answer. Something like “The task is accepted, but will be completed later” when it is not completed during the timeout or force to influence this from the process.
We want to do this through publication / subscription:
create an instance of the workflow asynchronously
subscribe to the id of the instance of the workflow with a timeout
in case of completion, the event is intercepted and published with the identifier of the workflow instance
the intermediate result can be realized through the publication of the event by the worker in the process itself, if necessary
in the case of a timer or event, we receive the final or intermediate data from the storage, depending on the context
Are you planning to do something like this in Zeebe?
This requires a fundamental change in the architecture. The gateway sends a request to the broker, then gets a response. Apart from that it is stateless. The broker cannot communicate proactively to the gateway, and there is no “query API” from the gateway, or other statefulness in the system.
Once the CreateWorkflowInstance command is sent to a specific broker, that broker responds - either on the creation or the completion - the gateway responds to the client, then all state is torn down.
Introducing statefulness into the gateway - broker system is a fairly significant change, and one that will have impact on performance.
Prior to CreateWorkflowInstanceWithResult, we used to do all these kinds of things using a notification service task and either a singleton wfi creator / job worker combo, or a distributed state frontend using sockets or Redis.
In terms of the realistic future, that’s probably the best way to do it.
Another path (not mutually exclusive), is to open a feature request in GitHub and put your use case in it.
But for some cases it is interesting for us to get an intermediate answer. Something like “The task is accepted, but will be completed later” when it is not completed during the timeout or force to influence this from the process.