Kafka connect fails in docker-compose

I’m using docker-compose from kafka-connect-zeebe following the the install guide. mvn clean install -DskipTests worked fine. However, when I try to run docker-compose up the connect service doesn’t start properly and I’m getting an empty response from the server when running any command like this: curl localhost:8083/

Here are some logs which may help:

kafka_1 | [2020-01-22 11:36:36,591] INFO Opening socket connection to server zookeeper/172.24.0.2:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
kafka_1 | [2020-01-22 11:36:37,837] WARN Attempting to send response via channel for which there is no open connection, connection id 172.24.0.6:19092-172.24.0.7:51372-21 (kafka.network.Processor)
kafka_1 | [2020-01-22 11:36:36,629] WARN Attempting to send response via channel for which there is no open connection, connection id 172.24.0.6:19092-172.24.0.9:46752-21 (kafka.network.Processor)
kafka_1 | [2020-01-22 11:36:44,019] WARN Attempting to send response via channel for which there is no open connection, connection id 172.24.0.6:19092-172.24.0.9:46802-22 (kafka.network.Processor)
kafka_1 | [2020-01-22 11:36:44,153] INFO Socket connection established to zookeeper/172.24.0.2:2181, initiating session (org.apache.zookeeper.ClientCnxn)
kafka_1 | [2020-01-22 11:36:48,600] WARN Client session timed out, have not heard from server in 6127ms for sessionid 0x16fcd000c600001 (org.apache.zookeeper.ClientCnxn)
kafka_1 | [2020-01-22 11:36:51,094] INFO Client session timed out, have not heard from server in 6127ms for sessionid 0x16fcd000c600001, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
zookeeper_1 | 2020-01-22 11:36:43,543 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /172.24.0.6:34712
zookeeper_1 | 2020-01-22 11:36:57,792 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@921] - Client attempting to renew session 0x16fcd000c600001 at /172.24.0.6:34712
kafka_1 | [2020-01-22 11:36:58,993] INFO [Controller id=1] Processing automatic preferred replica leader election (kafka.controller.KafkaController)
zookeeper_1 | 2020-01-22 11:37:00,263 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@666] - Invalid session 0x16fcd000c600001 for client /172.24.0.6:34712, probably expired
schema-registry_1 | [2020-01-22 11:37:05,846] INFO [Schema registry clientId=sr-1, groupId=schema-registry] Discovered group coordinator kafka:19092 (id: 2147483646 rack: null) (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)
zookeeper_1 | 2020-01-22 11:37:01,439 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1008] - Closed socket connection for client /172.24.0.6:34712 which had sessionid 0x16fcd000c600001
schema-registry_1 | [2020-01-22 11:37:18,580] INFO [Schema registry clientId=sr-1, groupId=schema-registry] Group coordinator kafka:19092 (id: 2147483646 rack: null) is unavailable or invalid, will attempt rediscovery (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)
kafka_1 | [2020-01-22 11:37:19,794] WARN Attempting to send response via channel for which there is no open connection, connection id 172.24.0.6:19092-172.24.0.9:46808-22 (kafka.network.Processor)

Hey nasushkov!

Wired - I just double checked and could start the latest head on docker compose without problems - even if you can get these warnings temporarily when Kafka is not yet up. Can you see any exceptions (ERROR not WARN) in the logs? What’s your environment? What I had regularly in the past is that I got problems with Kafka in Docker if I don’t reserve enough memory for docker. At the moment I give it 6.5 GB (of my 16GB of RAM) and it works Ok on my Windows 10 machine.

Best
Bernd

1 Like

Hi Bernd,
I tried this setup on my Mac Book Pro and iMac with 16GB of RAM both having the same issue. When I run docker-compose ps I see the following:

docker_connect_1 /etc/confluent/docker/run Exit 137
docker_control-center_1 /etc/confluent/docker/run Exit 1
docker_elasticsearch_1 /usr/local/bin/docker-entr … Up 0.0.0.0:9200->9200/tcp, 9300/tcp
docker_kafka_1 /etc/confluent/docker/run Up 0.0.0.0:9092->9092/tcp
docker_operate_1 /bin/tini – /usr/local/op … Up 0.0.0.0:8080->8080/tcp
docker_schema-registry_1 /etc/confluent/docker/run Up 0.0.0.0:8081->8081/tcp
docker_zeebe_1 tini – /usr/local/bin/sta … Up 0.0.0.0:26500->26500/tcp, 26501/tcp, 26502/tcp, 0.0.0.0:9600->9600/tcp
docker_zookeeper_1 /docker-entrypoint.sh zkSe … Up 0.0.0.0:2181->2181/tcp, 2888/tcp, 3888/tcp

For the Docker I have 3 and 4 GB of reserved memory accordingly, so this can be the issue. I’ll try to increase it to the suggested 6.5 and see what happens. Thanks for the tip!

I’ve checked both machines with a new Docker configuration of 6,5GB of RAM and it works like a charm.I think, it can be a good point to add this environment requirement to the README

Great to hear that helped! Yeah - you are right, I will make a note…

Hi Bernd, Great work!! But can you explain here/make a video to deploy workflow in docker version by not executing through distribution?

Hey Ambika809! Could you probably rephrase your question? Unfortunately, I am not yet getting it. Thanks, Bernd.