Help with helm install

Hello. I’m using helm to install zeebe/zeebe-full and it’s failing to start elasticsearch. Wonder why? I followed the instructions on https://docs.zeebe.io/kubernetes/installing-helm.html. The only thing I did not do was the pair of commands concerning initializing the tiller pod. My cluster already has tiller. Please help.

$ helm install --name test1 zeebe/zeebe-full

I’ve included output from kubectl get pods and kubectl logs.

NAME                                                       READY   STATUS             RESTARTS   AGE
pod/elasticsearch-master-0                                 0/1     Pending            0    	 57m
pod/elasticsearch-master-1                                 0/1     Pending            0    	 57m
pod/elasticsearch-master-2                                 0/1     Pending            0    	 57m
pod/test1-nginx-ingress-controller-5cc79c99fc-wzhv2        1/1     Running            0    	 57m
pod/test1-nginx-ingress-default-backend-7557646fcc-7bc8z   1/1     Running            0    	 57m
pod/test1-operate-7dfb57fd9d-fr5ts                         0/1     CrashLoopBackOff   13   	 57m
pod/test1-zeebe-0                                          0/1     Pending            0    	 57m
pod/test1-zeebe-1                                          0/1     Pending            0    	 57m
pod/test1-zeebe-2                                          0/1     Pending            0    	 57m


NAME                                          TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
service/elasticsearch-master                  ClusterIP      10.100.200.164   <none>        9200/TCP,9300/TCP            57m
service/elasticsearch-master-headless         ClusterIP      None             <none>        9200/TCP,9300/TCP            57m
service/kubernetes                            ClusterIP      10.100.200.1     <none>        443/TCP                      6d5h
service/test1-nginx-ingress-controller        LoadBalancer   10.100.200.67    10.216.0.80   80:30328/TCP,443:32520/TCP   57m
service/test1-nginx-ingress-default-backend   ClusterIP      10.100.200.220   <none>        80/TCP                       57m
service/test1-operate                         ClusterIP      10.100.200.211   <none>        80/TCP                       57m
service/test1-zeebe                           ClusterIP      None             <none>        9600/TCP,26500/TCP           57m


NAME                                                  READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/test1-nginx-ingress-controller        1/1     1            1           57m
deployment.apps/test1-nginx-ingress-default-backend   1/1     1            1           57m
deployment.apps/test1-operate                         0/1     1            0           57m

NAME                                                             DESIRED   CURRENT   READY   AGE
replicaset.apps/test1-nginx-ingress-controller-5cc79c99fc        1         1         1       57m
replicaset.apps/test1-nginx-ingress-default-backend-7557646fcc   1         1         1       57m

Here’s the log from the operate pod that restarts continuously.

$ kubectl logs test1-operate-7dfb57fd9d-fr5ts

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.2.1.RELEASE)

2020-03-24 02:52:55.576  INFO 6 --- [           main] o.c.o.Application                        : Starting Application v1.2.0 on test1-operate-7dfb57fd9d-fr5ts with PID 6 (/usr/local/operate/lib/camunda-operate-webapp-1.2.0-exec.jar started by root in /usr/local/operate)
2020-03-24 02:52:55.586 DEBUG 6 --- [           main] o.c.o.Application                        : Running with Spring Boot v2.2.1.RELEASE, Spring v5.2.1.RELEASE
2020-03-24 02:52:55.586  INFO 6 --- [           main] o.c.o.Application                        : The following profiles are active: auth
2020-03-24 02:52:59.521  INFO 6 --- [           main] o.s.b.w.e.t.TomcatWebServer              : Tomcat initialized with port(s): 8080 (http)
2020-03-24 02:52:59.588  INFO 6 --- [           main] o.a.c.h.Http11NioProtocol                : Initializing ProtocolHandler ["http-nio-8080"]
2020-03-24 02:52:59.590  INFO 6 --- [           main] o.a.c.c.StandardService                  : Starting service [Tomcat]
2020-03-24 02:52:59.591  INFO 6 --- [           main] o.a.c.c.StandardEngine                   : Starting Servlet engine: [Apache Tomcat/9.0.27]
2020-03-24 02:52:59.774  INFO 6 --- [           main] o.a.c.c.C.[.[.[/]                        : Initializing Spring embedded WebApplicationContext
2020-03-24 02:52:59.774  INFO 6 --- [           main] o.s.w.c.ContextLoader                    : Root WebApplicationContext: initialization completed in 4085 ms
2020-03-24 02:53:01.378  INFO 6 --- [           main] o.c.o.WebappModuleConfiguration          : Starting module: webapp
2020-03-24 02:53:01.485 DEBUG 6 --- [           main] o.c.o.e.ElasticsearchConnector           : Creating Elasticsearch connection...
2020-03-24 02:53:02.798 ERROR 6 --- [           main] o.c.o.e.ElasticsearchConnector           : Error occurred while connecting to Elasticsearch: clustername [elasticsearch], elasticsearch-master:9200. Will be retried...

java.net.ConnectException: Timeout connecting to [elasticsearch-master/10.100.200.164:9200]
	at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:959) ~[elasticsearch-rest-client-6.8.4.jar!/:6.8.5]
	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:233) ~[elasticsearch-rest-client-6.8.4.jar!/:6.8.5]
	at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1764) ~[elasticsearch-rest-high-level-client-6.8.5.jar!/:6.8.5]
	at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1734) ~[elasticsearch-rest-high-level-client-6.8.5.jar!/:6.8.5]
	at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1696) ~[elasticsearch-rest-high-level-client-6.8.5.jar!/:6.8.5]
	at org.elasticsearch.client.ClusterClient.health(ClusterClient.java:146) ~[elasticsearch-rest-high-level-client-6.8.5.jar!/:6.8.5]
	at org.camunda.operate.es.ElasticsearchConnector.checkHealth(ElasticsearchConnector.java:89) ~[camunda-operate-common-1.2.0.jar!/:1.2.0]
	at org.camunda.operate.es.ElasticsearchConnector.createEsClient(ElasticsearchConnector.java:75) ~[camunda-operate-common-1.2.0.jar!/:1.2.0]
	at org.camunda.operate.es.ElasticsearchConnector.esClient(ElasticsearchConnector.java:51) ~[camunda-operate-common-1.2.0.jar!/:1.2.0]
	at org.camunda.operate.es.ElasticsearchConnector$$EnhancerBySpringCGLIB$$ecf76157.CGLIB$esClient$0(<generated>) ~[camunda-operate-common-1.2.0.jar!/:1.2.0]
	at org.camunda.operate.es.ElasticsearchConnector$$EnhancerBySpringCGLIB$$ecf76157$$FastClassBySpringCGLIB$$ba34aef2.invoke(<generated>) ~[camunda-operate-common-1.2.0.jar!/:1.2.0]
	at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) ~[spring-core-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
	at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:363) ~[spring-context-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
	at org.camunda.operate.es.ElasticsearchConnector$$EnhancerBySpringCGLIB$$ecf76157.esClient(<generated>) ~[camunda-operate-common-1.2.0.jar!/:1.2.0]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
	at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
	at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:640) ~[spring-beans-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
	at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:475) ~[spring-beans-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1338) ~[spring-beans-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]

Hey @kwalker17,

your elastic search are seem to be pending, but eventually they should become running.
Is that the case? K8s needs sometimes a bit more time to schedule the pods, especially if it can’t find a sufficient node. You can run a describe command to see the events of the pod scheduling.

Greets
Chris

1 Like

Thank you Chris for your reply. I needed to add a storageclass to my cluster. Sweet!