Kafka - Zeebe Connector

Hi

I am trying to get the demo Kafka - Zeebe Connector running. The Ping Pong application.

I have the following processes running:
C:\Camunda\elasticsearch-6.8.0>bin\elasticsearch.bat
C:\Camunda\zeebe-broker-0.20.0>bin\broker.bat
C:\Camunda\camunda-operate-1.0.0>bin\operate.bat

C:\Kafka\kafka_2.12-2.4.0>bin\windows\zookeeper-server-start.bat config\zookeeper.properties
C:\Kafka\kafka_2.12-2.4.0>bin\windows\kafka-server-start.bat config/server.properties

I initiate the workflow using:

bin\zbctl.exe create instance --variables "{\"name\": \"pong\", \"payload\": { \"foo\": \"bar\"}, \"key\": 2}" ping-pong

I can see the running instance in the Operate GUI. It is waiting at the “To Kafka” Service Task.
I cannot see any messages published to the pong topic using this command:

bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic pong --from-beginning

I copied the uber zeebe connector jar into C:\Kafka\kafka_2.12-2.4.0\libs
I copied the files
quickstart-zeebe-source.properties
quickstart-zeebe-sink.properties
into C:\Kafka\kafka_2.12-2.4.0\config

and started the connect-standalone process using:

C:\Kafka\kafka_2.12-2.4.0>bin\windows\connect-standalone.bat config\connect-standalone.properties config\quickstart-zeebe-source.properties config\quickstart-zeebe-sink.properties

There are many errors during start up but the example foo/bar file Connector example works.
https://kafka.apache.org/quickstart
Step 7.

After initiating a new Ping Pong workflow execution I see no message activity in the connector-standalone process or the kafka broker process.

Can anyone point me in the right direction please.

Thanks
Alan

1 Like

@Alan have you checked this file:

Are you running with that docker compose?

Hi @salaboy, thanks for the response. No I am not using docker-compose. At the moment I need to run on my Windows desktop. The Confluent Kafka docker does not run on Windows. So I am running all of the processes manually.

elasticsearch
zeebe-broker
camunda-operate
zookeeper
kafka-server-start.bat

"The Confluent Kafka docker does not run on Windows." really? do you know why?
You can run docker compose on windows right?

Sorry my Windows desktop is a virtualized windows instance. I thought it was not possible to run another VM for docker on top of this. I will double check if that is correct. But I still should be able to get the zeebe integration running manually. Thanks

@Alan I am sure that you can run docker compose even if it is in a virtual machine.
If you want to get it working by manually running all the services, I am sure that you can copy each component configuration from the docker-compose.yaml file… You will need to replicate what those docker containers are providing to make it work. You probably want to use the same versions as well.