Algorithm/Framework behind Zeebe backpressure protocol

Camunda 8 Topics Discussion & Questions
1

I was wondering on the concept of how Zeebe backpressure protocol was implemented?

https://docs.zeebe.io/basics/job-workers.html

What is the Algorithm/Framework behind Zeebe backpressure protocol?

Can someone walkthrough me to Zeebe backpressure protocol implementation.

Thanks,
Aravindh RS

2

Hi @aravindhrs

Good question–that page in the documentation is a bit out of date, and I recently wrote an updated version that will explain this concept more clearly.

But those updates aren’t merged yet, so here’s a quick summary:
• Starting with Zeebe 0.12, Zeebe job workers request jobs from the broker; in the client, you can configure how many jobs are requested and at what interval.
• The broker will continue to create and queue jobs regardless of how quickly workers are requesting / completing them.
• And so the workers ultimately have control over the rate at which they take on new jobs, and this won’t impact job creation by the broker.
• It should also be mentioned that workers can be scaled up so that multiple workers are working on the same type of job.

In the future, there might be support for more sophisticated broker - client communication, e.g. via long-polling. We’ve heard feedback that something this would be helpful but we haven’t explicitly put it on the roadmap yet. Always interested to hear feedback there.

Best,
Mike

1 Like
3