We have the prometheus endpoint being scraped on our datastore servers. I am attempting to create a metric for the amount of time that it takes to perform ldap searches/binds/adds/deletes etc…
When I hit the metrics endpoint and grep for specific actions, it appears as though the metrics for those requests hardly ever change. For example, In Grafana, if I look at the ds_connection_handlers_ldap_request_seconds for ‘search’ on a specific host and the .5 quantile for that monitor has been the exact same for 7 days straight. There are thousands of search requests that hit hit that server every minute. Yet, the metric never changes off of the exact same value which is a little more than 2 milliseconds.
Here is our config on the prometheus endpoint it seems like we’re neither excluding metrics nor including them. Any thoughts on why those values are available, yet never changing?
1) authorization-mechanism HTTP Basic
2) base-path /metrics/prometheus
3) enabled true
4) excluded-metric-pattern -
5) included-metric-pattern -
You can extract those statistics directly from DS. You need a service account that has access to
cn=monitor data, see:
You can pull all stats for LDAPS:
$ bin/ldapsearch --nopropertiesfile --hostname localhost --port 1636 --bindDN uid=monitor --bindpassword password --searchScope base --baseDN "cn=LDAPS,cn=connection handlers,cn=monitor" --useSsl --trustAll "(&)" \*
Or target a specific operation (e.g. search with scope sub):
$ bin/ldapsearch --nopropertiesfile --hostname localhost --port 1636 --bindDN uid=monitor --bindpassword password --searchScope base --baseDN "cn=LDAPS,cn=connection handlers,cn=monitor" --useSsl --trustAll "(&)" ds-mon-requests-search-sub
You can also run a support extract job and get all stats in a file called monitor.ldif:
Having said that, I checked the Prometheus endpoint and my observations are similar to yours. I also connected Grafana, and the values being plotted do change, so I am not entirely sure why at this point.
Looking at the stats out of DS (which is the actual source) percentiles are re-calculated at frequent intervals but not exactly real time. So, live changes may not be visible right away. Also a lot of stats start from the moment DS is started, therefore if DS has been running for a very long time, it may be that some stats do not change much any more.
You may evaluate live performance with the provided rating tools: