Kafka connector issue

Camunda 8 Topics Discussion & Questions
1

Hi, what I have configured is working on MacOS but failing on Windows.

The Sink is working fine, but not my source.

Source properties:

name=zeebe-source
connector.class=io.zeebe.kafka.connect.ZeebeSourceConnector
tasks.max=1

key.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=false
value.converter=org.apache.kafka.connect.json.JsonConverter
value.converter.schemas.enable=false

zeebe.client.broker.contactPoint=localhost:26500
zeebe.client.security.plaintext=true
zeebe.client.worker.maxJobsActive=100
zeebe.client.job.worker=kafka-connector
zeebe.client.job.timeout=5000
zeebe.client.requestTimeout=10000
job.types=activate-isk-account-manually
job.header.topics=kafka-topic

Note: I have removed the lic info above to make less text.

BPMN task header is configured with the right Kafka topic.

I’ve followed the instructions for Running without Docker using Kafka 2.13. The connector is not picking up the job. How can I troubleshoot this issue?

EDITED:

This is the audit log:

Zeebe1
Zeebe11887×855 74 KB

2

Does ELEMENT ACTIVATED means that the worker (Kafka Connector) has acknowledged the job?

3

No, it means that the broker has activated the element, and there is now a job waiting for a worker.

4

Thanks @jwulf, any idea what can be the issue, or where to start troubleshooting?

5

The Broker side is working fine. What about logs from Kafka? Is the connector plugin loaded? Does it throw an errors? Does it emit any logs?

6

Hi @jwulf,

Broker is working fine and as expected, yes. The Kafka logs show nothing in particular. The Kafka Connect standalone is loading the plugin:

[2020-08-05 15:51:24,243] INFO Loading plugin from: /etc/kafka-connect/jars/kafka-connect-zeebe-1.0.0-SNAPSHOT-uber.jar (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:220)
[2020-08-05 15:51:25,501] INFO Registered loader: PluginClassLoader{pluginLocation=file:/etc/kafka-connect/jars/kafka-connect-zeebe-1.0.0-SNAPSHOT-uber.jar} (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:243)
[2020-08-05 15:51:25,501] INFO Added plugin 'io.zeebe.kafka.connect.ZeebeSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:172)
[2020-08-05 15:51:25,501] INFO Added plugin 'io.zeebe.kafka.connect.ZeebeSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:172)

The creation of workers seems fine as well:

[2020-08-05 15:51:28,647] INFO Creating connector zeebe-source of type io.zeebe.kafka.connect.ZeebeSourceConnector (org.apache.kafka.connect.runtime.Worker:235)
[2020-08-05 15:51:28,649] INFO Instantiated connector zeebe-source with version 1.0.0-SNAPSHOT of type class io.zeebe.kafka.connect.ZeebeSourceConnector (org.apache.kafka.connect.runtime.Worker:238)
[2020-08-05 15:51:28,650] INFO ZeebeSourceConnectorConfig values: 
	job.header.topics = kafka-topic
	job.types = [activate-isk-account-manually]
	job.variables = [kafka-variables]
	zeebe.client.broker.contactPoint = localhost:26500
	zeebe.client.job.timeout = 5000
	zeebe.client.requestTimeout = 10000
	zeebe.client.security.plaintext = true
	zeebe.client.worker.maxJobsActive = 100
	zeebe.client.worker.name = kafka-connector
 (io.zeebe.kafka.connect.source.ZeebeSourceConnectorConfig:279)
[2020-08-05 15:51:28,651] INFO Finished creating connector zeebe-source (org.apache.kafka.connect.runtime.Worker:257)

Anywhere else where the connector is supposed to write log?

7

I had messed up my configuration. Everything works now.

2 Likes
8

@jonas @berndruecker
I am facing the same issue. Can you please post the change you made in the configuration ?

9