Hello. I’m using helm to install zeebe/zeebe-full and it’s failing to start elasticsearch. Wonder why? I followed the instructions on Zeebe | Camunda Cloud Docs. The only thing I did not do was the pair of commands concerning initializing the tiller pod. My cluster already has tiller. Please help.
$ helm install --name test1 zeebe/zeebe-full
I’ve included output from kubectl get pods and kubectl logs.
NAME READY STATUS RESTARTS AGE
pod/elasticsearch-master-0 0/1 Pending 0 57m
pod/elasticsearch-master-1 0/1 Pending 0 57m
pod/elasticsearch-master-2 0/1 Pending 0 57m
pod/test1-nginx-ingress-controller-5cc79c99fc-wzhv2 1/1 Running 0 57m
pod/test1-nginx-ingress-default-backend-7557646fcc-7bc8z 1/1 Running 0 57m
pod/test1-operate-7dfb57fd9d-fr5ts 0/1 CrashLoopBackOff 13 57m
pod/test1-zeebe-0 0/1 Pending 0 57m
pod/test1-zeebe-1 0/1 Pending 0 57m
pod/test1-zeebe-2 0/1 Pending 0 57m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/elasticsearch-master ClusterIP 10.100.200.164 <none> 9200/TCP,9300/TCP 57m
service/elasticsearch-master-headless ClusterIP None <none> 9200/TCP,9300/TCP 57m
service/kubernetes ClusterIP 10.100.200.1 <none> 443/TCP 6d5h
service/test1-nginx-ingress-controller LoadBalancer 10.100.200.67 10.216.0.80 80:30328/TCP,443:32520/TCP 57m
service/test1-nginx-ingress-default-backend ClusterIP 10.100.200.220 <none> 80/TCP 57m
service/test1-operate ClusterIP 10.100.200.211 <none> 80/TCP 57m
service/test1-zeebe ClusterIP None <none> 9600/TCP,26500/TCP 57m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/test1-nginx-ingress-controller 1/1 1 1 57m
deployment.apps/test1-nginx-ingress-default-backend 1/1 1 1 57m
deployment.apps/test1-operate 0/1 1 0 57m
NAME DESIRED CURRENT READY AGE
replicaset.apps/test1-nginx-ingress-controller-5cc79c99fc 1 1 1 57m
replicaset.apps/test1-nginx-ingress-default-backend-7557646fcc 1 1 1 57m
Here’s the log from the operate pod that restarts continuously.
$ kubectl logs test1-operate-7dfb57fd9d-fr5ts
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.2.1.RELEASE)
2020-03-24 02:52:55.576 INFO 6 --- [ main] o.c.o.Application : Starting Application v1.2.0 on test1-operate-7dfb57fd9d-fr5ts with PID 6 (/usr/local/operate/lib/camunda-operate-webapp-1.2.0-exec.jar started by root in /usr/local/operate)
2020-03-24 02:52:55.586 DEBUG 6 --- [ main] o.c.o.Application : Running with Spring Boot v2.2.1.RELEASE, Spring v5.2.1.RELEASE
2020-03-24 02:52:55.586 INFO 6 --- [ main] o.c.o.Application : The following profiles are active: auth
2020-03-24 02:52:59.521 INFO 6 --- [ main] o.s.b.w.e.t.TomcatWebServer : Tomcat initialized with port(s): 8080 (http)
2020-03-24 02:52:59.588 INFO 6 --- [ main] o.a.c.h.Http11NioProtocol : Initializing ProtocolHandler ["http-nio-8080"]
2020-03-24 02:52:59.590 INFO 6 --- [ main] o.a.c.c.StandardService : Starting service [Tomcat]
2020-03-24 02:52:59.591 INFO 6 --- [ main] o.a.c.c.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.27]
2020-03-24 02:52:59.774 INFO 6 --- [ main] o.a.c.c.C.[.[.[/] : Initializing Spring embedded WebApplicationContext
2020-03-24 02:52:59.774 INFO 6 --- [ main] o.s.w.c.ContextLoader : Root WebApplicationContext: initialization completed in 4085 ms
2020-03-24 02:53:01.378 INFO 6 --- [ main] o.c.o.WebappModuleConfiguration : Starting module: webapp
2020-03-24 02:53:01.485 DEBUG 6 --- [ main] o.c.o.e.ElasticsearchConnector : Creating Elasticsearch connection...
2020-03-24 02:53:02.798 ERROR 6 --- [ main] o.c.o.e.ElasticsearchConnector : Error occurred while connecting to Elasticsearch: clustername [elasticsearch], elasticsearch-master:9200. Will be retried...
java.net.ConnectException: Timeout connecting to [elasticsearch-master/10.100.200.164:9200]
at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:959) ~[elasticsearch-rest-client-6.8.4.jar!/:6.8.5]
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:233) ~[elasticsearch-rest-client-6.8.4.jar!/:6.8.5]
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1764) ~[elasticsearch-rest-high-level-client-6.8.5.jar!/:6.8.5]
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1734) ~[elasticsearch-rest-high-level-client-6.8.5.jar!/:6.8.5]
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1696) ~[elasticsearch-rest-high-level-client-6.8.5.jar!/:6.8.5]
at org.elasticsearch.client.ClusterClient.health(ClusterClient.java:146) ~[elasticsearch-rest-high-level-client-6.8.5.jar!/:6.8.5]
at org.camunda.operate.es.ElasticsearchConnector.checkHealth(ElasticsearchConnector.java:89) ~[camunda-operate-common-1.2.0.jar!/:1.2.0]
at org.camunda.operate.es.ElasticsearchConnector.createEsClient(ElasticsearchConnector.java:75) ~[camunda-operate-common-1.2.0.jar!/:1.2.0]
at org.camunda.operate.es.ElasticsearchConnector.esClient(ElasticsearchConnector.java:51) ~[camunda-operate-common-1.2.0.jar!/:1.2.0]
at org.camunda.operate.es.ElasticsearchConnector$$EnhancerBySpringCGLIB$$ecf76157.CGLIB$esClient$0(<generated>) ~[camunda-operate-common-1.2.0.jar!/:1.2.0]
at org.camunda.operate.es.ElasticsearchConnector$$EnhancerBySpringCGLIB$$ecf76157$$FastClassBySpringCGLIB$$ba34aef2.invoke(<generated>) ~[camunda-operate-common-1.2.0.jar!/:1.2.0]
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) ~[spring-core-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:363) ~[spring-context-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
at org.camunda.operate.es.ElasticsearchConnector$$EnhancerBySpringCGLIB$$ecf76157.esClient(<generated>) ~[camunda-operate-common-1.2.0.jar!/:1.2.0]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:640) ~[spring-beans-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:475) ~[spring-beans-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1338) ~[spring-beans-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]