io.zeebe.client.api.command.ClientStatusException: deadline exceeded after 19999976300ns

Hello Team,

My micro-service going into infinite loop with below error. (Error message down below)

I have set like this:
return zeebe.newWorker()
.jobType(“my.job”)
.handler(ProvisionJob())
.timeout(Duration.ofMinutes(30))
.open();

Seems like 30 minutes setting is getting over written to 19 min from the system configuraiton.

Could you please tell us where we can set a custom value? I want to increase the timeout from 19 to 30 min.

Thanks for the help.

Error:
2019-09-16 15:31:19.839 WARN 1 — [pool-2-thread-1] io.zeebe.client.job.worker : Worker default failed to handle job with key 4503599628584815 of type sp2-provision-saptop, sending fail command to broker

io.zeebe.client.api.command.ClientStatusException: deadline exceeded after 19999976300ns
at io.zeebe.client.impl.ZeebeClientFutureImpl.transformExecutionException(ZeebeClientFutureImpl.java:93) ~[zeebe-client-java-0.20.0.jar!/:0.20.0]
at io.zeebe.client.impl.ZeebeClientFutureImpl.join(ZeebeClientFutureImpl.java:50) ~[zeebe-client-java-0.20.0.jar!/:0.20.0]
at com.swissre.g69.ietv.provisioninfrastructure.ProvisionFPSL.handle(ProvisionFPSL.java:41) ~[classes!/:na]
at io.zeebe.client.impl.worker.JobRunnableFactory.executeJob(JobRunnableFactory.java:44) [zeebe-client-java-0.20.0.jar!/:0.20.0]
at io.zeebe.client.impl.worker.JobRunnableFactory.lambda$create$0(JobRunnableFactory.java:39) [zeebe-client-java-0.20.0.jar!/:0.20.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_212]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_212]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_212]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[na:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_212]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_212]
Caused by: java.util.concurrent.ExecutionException: io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 19999976300ns
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) ~[na:1.8.0_212]
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895) ~[na:1.8.0_212]
at io.zeebe.client.impl.ZeebeClientFutureImpl.join(ZeebeClientFutureImpl.java:48) ~[zeebe-client-java-0.20.0.jar!/:0.20.0]
… 10 common frames omitted
Caused by: io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 19999976300ns
at io.grpc.Status.asRuntimeException(Status.java:533) ~[grpc-api-1.22.1.jar!/:1.22.1]
at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:442) ~[grpc-stub-1.22.1.jar!/:1.22.1]
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39) ~[grpc-api-1.22.1.jar!/:1.22.1]
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23) ~[grpc-api-1.22.1.jar!/:1.22.1]
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40) ~[grpc-api-1.22.1.jar!/:1.22.1]
at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:700) ~[grpc-core-1.22.1.jar!/:1.22.1]
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39) ~[grpc-api-1.22.1.jar!/:1.22.1]
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23) ~[grpc-api-1.22.1.jar!/:1.22.1]
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40) ~[grpc-api-1.22.1.jar!/:1.22.1]
at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:399) ~[grpc-core-1.22.1.jar!/:1.22.1]
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:507) ~[grpc-core-1.22.1.jar!/:1.22.1]
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:66) ~[grpc-core-1.22.1.jar!/:1.22.1]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:627) ~[grpc-core-1.22.1.jar!/:1.22.1]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$700(ClientCallImpl.java:515) ~[grpc-core-1.22.1.jar!/:1.22.1]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:686) ~[grpc-core-1.22.1.jar!/:1.22.1]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:675) ~[grpc-core-1.22.1.jar!/:1.22.1]
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) ~[grpc-core-1.22.1.jar!/:1.22.1]
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) ~[grpc-core-1.22.1.jar!/:1.22.1]
… 3 common frames omitted

Hey @prasadkoribilli

first of all thanks for using Zeebe.

The problem is that you configured the job timeout, which means it is the time how long the job is assigned to that specific worker. But there is also the default long polling configured of 10 seconds and the request timeout of 20 seconds. So it you want to increase that just use .requestTimeout(Duration.ofMinutes(30)).

I hope that helps.

Greets
Chris

Hello @Zelldon

Thanks for your reply. I will try that setting out.
But if I read the exception below,

"io.zeebe.client.job.worker : Worker default failed to handle job with key 4503599628584815 of type sp2-provision-saptop, sending fail command to broker
io.zeebe.client.api.command.ClientStatusException: deadline exceeded after 19999976300ns "

worker is not able to send the command to broker. Which is communication from Worker to Broker.
But the parameter that you suggested is more for polling, meaning getting workers from Broker.

Are they not two different things? One is for sending command other is polling a job.

Please enlighten me a bit here.

Thanks.

Hey @prasadkoribilli sorry might that I misunderstood your problem. I thought your request times out and with requestTimeout you can configure that (+long polling).

Could you please share your job handler code and the complete log?

Greets
Chris