Expose zeebe gateway out of k8s cluster

Hi I have installed zeebe helm chart in a brandnew k8s cluster inside AWS EKS. And now I need to connect to the zeebe cluster (gateway port 26500) from outside of that k8s cluster (eg. Another k8s cluster and standalone ec2 machines). I can’t seem to figure out how to expose this. Only way I could connect to the gateway via is through port forwarding as indicated in https://docs.zeebe.io/kubernetes/installing-helm.html
Please note that my k8s cluster is in private subnet only and it managed to create the internal elb for nginx controller as well (I’m accessing the elb from one of the ec2 machines).
I tried adding - name: gateway protocol: TCP port: 26500 targetPort: gateway nodePort: 31500
in to the exsisting nginx controller but it doesn’t seem to forward my request to zeebe gateway.

Can you please help me to identify what I’m doing wrong

@amilaezycollect Hi there… so the Zeebe Gateway is using GRPC protocol to communicate with the Zeebe Clients (zbctl or any other client) For that reason you need an ingress controller configured to route GRPC traffic, which is not trivial. You need to look for documentation on your loadbalancers on how to route GRPC traffic to be able to expose the gateway outside your cluster.
Hope this helps

Well, I tried creating an ingress and it did create an NLB in aws side. But when I try to access it give me this error
First received frame was not SETTINGS. Hex dump for first 5 bytes
And when I search about this issue, it points to
This is usually because the server is using SSL, but the client is not
But I don’t believe that I use SSL in gateway as it’s default false.

@amilaezycollect as far as I remember that is the whole problem with GRCP and NGINX controller, if you want the ingress controller to route both HTTP 1 and HTTP 2 traffic you need to find a way for the router to understand when it is http 1 and when it is http 2. Camunda Cloud as far as I know uses the TLS handshake to make this difference (look at the comment from aledbf -> https://github.com/kubernetes/ingress-nginx/issues/3938). There is nothing specific for Zeebe, you just need to be able to route HTTP 1 and HTTP 2 traffic via the same ingress controller. If you really want to make it work you will need to use TLS, which involves certificates and the like, and was the main reason why we keep it out from the community charts.