Software
Metrics and timeseries
We use prometheus
to scrape endpoints for data. These endpoints are simple text, made available on a given port by an exporter
Prometheus
has a query language to query the timeseries.
Logs
We use promtail
as the agent to send log data to a loki
instance.
loki
has a query language to search the logs and return log entries.
An alternative would be elasticsearch
as the log database and kibana
to make queries into the logs.
Presentation
We use grafana
as the dashboard "engine" and to query for different datasources. Prometheus
and loki
are two datasources. There are others as well.
Testing
- Download a test vm from here and install it in vmware workstation
- Connect it to vmnet111 (or the equivante 192.168.111.0/24 subnet)
- There is an information page on http://192.168.111.20
Unprivileged user is sysuser:sysuser123
and admins user is root:root123
Working with grafana
- Make a block diagram showing the relationship between
node_exporter
,prometheus
andgrafana
. - Add a
grafana
dashboard that uses thenode_exporter
values fromprometheus
- Add a
grafana
dashboard that uses the logs fromloki
-
Set up an extra VM with node_exporter and update
prometheus
to scrape from itThis is either a new Linux VM or the openbsd router.
If you have the obsolete OpenBSD 6.6 router, please upgrade to version 7.0. See here