Prometheus monitoring endpoint access number : is this possible?-CodePudding

I have an HTTP API server with endpoints like https://example.com/api/login and https://example.com/api/logout . etc

I have been looking for a way to monitor each API's access number every minutes and record them as a graph.

The easy way I was doing was basically adding some server code to record each access to a log file during access and then scraping through the log file.

However I think maybe prometheus can do this as well and this would be a much more proper way and more maintainable in the future.

I am completely new to prometheus and I did some research and found out maybe the blackbox exporter can do what I want. However the documentation seems to indicate that it will only "probe" the endpoint, which based on my understanding is to test the stability/availability parameters of the endpoint, rather than recording the number of access to each endpoint.

Will "recording access number for specified endpoints" be possible with prometheus? Or other open-source monitoring tools?

CodePudding user response：

If you are self-monitoring your Prometheus instance(s), there is a metric called prometheus_http_requests_total which exposes the total HTTP requests for each endpoint.

If you don't know what I mean by self-monitoring your Prometheus, just add a scrape job like this:

  - job_name: 'prometheus'
    static_configs:
    - targets:
      - localhost:9090 # your Prometheus port

CodePudding user response：

Prometheus gathers data from HTTP endpoints, meaning that it makes an HTTP GET request and the response should contain data in OpenMetrics format. Prometheus cannot read logs to create metrics.

However, there are lots of exporters and some of them can create metrics from logs. You can find an incomplete list here: https://prometheus.io/docs/instrumenting/exporters/#logging . The setup will be as following:

you install an exporter next to your logs and configure it to read them and make metrics;
you install Prometheus somewhere and configure it to scrape the exporters;
then you can use Prometheus UI or Grafana to view graps etc.

An alternative way will be to integrate Prometheus library into that API of yours. Then you can count requests internally and create an API endpoint for Prometheus to visit. In my opinion this is better than metrics via logs - less software, less things that can break.

At last, you can consider using Loki (https://grafana.com/oss/loki/) or the Elastic Stack (https://www.elastic.co/elastic-stack/). These tools are to gather logs in one place but you can also use them to create dashboards, build graphs, etc.