Zeebe embedded gateway is not sharing load across 3 node cluster

Hi Team,

I am new to Zeebe and trying a POC using 3 physical machines using flowing-retail-kafka model.
Configured brokers (zeebe-broker-0.20.1) and able to see all 3 brokers are up and running , replicating data across partitions in all 3 machines.

All nodes have embedded gateway.

Using flowing-retail-kafka-order-zeebe sample with zeebe.brokerContactPoint=50.1xx.20.2xx:26500, able to successfully create and complete instances.
when i tried with 100 requests , All my requests are going to 50.1xx.20.2xx node alone. other nodes are replicating data in partition and not creating instances or processing in parallel.

Attached config file for reference, Could you please help me figure out what i am missing to achieve parallel processing of requests.

removed comments to make the file short
[gateway]
enable = true
[gateway.network]
host = “50.1xx.20.2xx”
port = 26500
[gateway.cluster]
contactPoint = “50.1xx.20.2xx:26501”
transportBuffer = “128M”
requestTimeout = “35s”
[gateway.threads]

managementThreads = 1

[gateway.monitoring]

enabled = false

[network]
host = “50.1xx.20.2xx”

portOffset = 0

[network.commandApi]
host = “50.1xx.20.2xx”

Sets the port used for gateway-to-broker communication

port = 26501

Sets the size of the buffer used for buffering outgoing messages

sendBufferSize = “16M”
[network.internalApi]

Overrides the host used for internal broker-to-broker communication

host = “50.1xx.20.2xx”

Sets the port used for internal broker-to-broker communication

port = 26502
[network.monitoringApi]

Overrides the host used for exposing monitoring information

host = “50.1xx.20.2xx”

Sets the port used for exposing monitoring information

port = 9600
[data]
directories = [ “data” ]
logSegmentSize = “512M”
snapshotPeriod = “15m”
maxSnapshots = “3”
snapshotReplicationPeriod = “5m”
[cluster]
nodeId = 0
partitionsCount = 2
replicationFactor = 3
clusterSize = 3
initialContactPoints = [“50.1xx.20.2xx:26502”,“1x2.10.20.2xx:26502”,“1x2.11.20.1xx:26502”]
clusterName = “zeebe-cluster”
[threads]
cpuThreadCount = 2
ioThreadCount = 2

Hi @Vinkark, see here:https://docs.google.com/document/d/15coCXw9rA-bZyY_OIttrjWfJ6D8H2DIDOGOGXwQcdzk/edit#heading=h.xyv183z0e7ma

1 Like

Hi jwulf,
Many Thanks for your pointer to use stand alone gateway.
I started 2 gateways as below configuration, still unable to see load shared across nodes.

(removed other attributes to keep short )
Config 1:
{ “network”: {
“host”: “1x2.10.20.2xx”,
“port”: 26600 },
“cluster”: {
“contactPoint”: “1x2.10.20.2xx:26502”,
“host”: “1x2.10.20.2xx”,
“port”: 26602 }}
Config 2:
{ “network”: {
“host”: “1x2.11.20.1xx”,
“port”: 26600 },
“cluster”: {
“contactPoint”: “1x2.11.20.1xx:26502”,
“host”: “1x2.11.20.1xx”,
“port”: 26602 }}

And used

zeebe:brokerontactPoint=1x2.11.20.1xx:26600 in Spring client

below are logs of both gateway instances

INFO io.zeebe.transport.endpoint - Registering endpoint for node ‘1’ with address ‘50.1xx.20.2xx:26501’ on transport ‘gateway-broker-client’
INFO io.zeebe.transport.endpoint - Registering endpoint for node ‘2’ with address ‘1x2.10.20.2xx:26501’ on transport ‘gateway-broker-client’
INFO io.zeebe.transport.endpoint - Registering endpoint for node ‘0’ with address ‘1x2.11.20.1xx:26501’ on transport ‘gateway-broker-client’