I am new to Zeebe and started with local project setup. I am using standalone broker with embedded gateway to try out Zeebe related functionalities. I have also enabled the Prometheus monitoring API configurations and noticed that actual metrics URL is working fine but the host URL is not working. As a result I couldn’t use it in Grafana to show dashboards related to Zeebe.
I have configured the standalone broker with embedded gateway to have the monitoring API enabled. Added the configurations as follows to the appropriate yaml file
I can see the metrics related to Zeebe in the below link
http:<MY_HOST>:9600/metrics (This works)
but the following link doesn’t work
http:<MY_HOST>:9600 (This doesn’t work)
Since the landing server URL doesn’t work when I am configuring the datasource in Grafana it is not able to add the datasource as intended. I am getting the HTTP Error Not Found.
Is there any configuration available to make the landing server URL work so that I can configure the datasource in Grafana correctly and create dashboards for Zeebe?
Setup details:
a) Operating System
Distributor ID: RedHatEnterpriseServer
Description: Red Hat Enterprise Linux Server release 6.10 (Santiago)
Release: 6.10
b) Zeebe Version: 0.23.0
c) Prometheus: prometheus-2.19.1.linux-amd64
d) Grafana: grafana-7.0.6
Please let me know if I have to enable or add any configurations to make this work.
Hi @Zelldon - Sorry it was due to wrong configuration and environment it wasn’t working. Now I could able to get it working fine. Thanks for your reply.
@Zelldon - Quick question, I was testing in Zeebe 0.23.1 version and after correcting all environment and configurations it used to work perfectly fine. Now I upgraded Zeebe to latest 0.24.1 version and the configurations are retained same. Just wondering whether there is any change in behavior from Zeebe perspective where metrics URL returns HTTP status code 404.
http://<MY_HOST_IP>:9600/metrics
returns server returned HTTP status 404 in Prometheus targets.
@pihme - Please let me know whether we need to turn on any configurations. I have kept zeebe running for longer time and still no luck with the metrics URL in 0.24.1 version. Tried reverting back to older version to perform sanity check and it works fine. Only in newer version I am seeing this behavior. Please let me know your thoughts.
Can you tell me what gets returned when you access
[host]:9600/actuator/
This returns a JSON and should contain the string Prometheus.
If it doesn’t, and if you specify the config file via SPRING_CONFIG_LOCATION=[path to file] then change it to export SPRING_CONFIG_LOCATION='classpath:/,file:./[path to config file]'.
This was one of the reasons why we couldn’t access the metrics. PR is pending.
It didn’t return prometheus URL if I give [host]:9600/actuator, however after changing the spring config location to what you have pointed out to add classpath showed me the prometheus URL. Now with the same URL [host]:9600/metrics I could able to see it working.
Just wondering whether is it by design this has changed or something is modified? I couldn’t find or missed in Zeebe upgrade documentation.
Anyhow thanks @Zelldon and @pihme for assisting me and sorting this issue out.
Hi everyone, I was following this thread because I am having issues configuring the metrics.
I have updated my zeebe_configmap with:
zeebe:
broker:
network:
port: 9600
But I cannot still open my :9600/metrics.
I have created a rule to allow egress to 9600 but seems it is not working either. Here a screenshot of my configuration: