Monitoring

Hyvor Relay integrates with the Prometheus/Grafana stack for monitoring.

• Prometheus
• Grafana
• Alertmanager
• Logs

Prometheus

Hyvor Relay exposes metrics in the Prometheus format on port 9667 on each server. It only serves metrics to connections from private and CGNAT IP addresses, assuming your monitoring system is on a private network.

To get started, you can add the following to your prometheus.yaml file:

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'hyvor-relay'
    static_configs:
      - targets: 
        - 'your-relay-server-1-private-ip:9667'
        - 'your-relay-server-2-private-ip:9667'

The following metrics are available from each Relay server. Metrics marked as "global" are only exposed from the Leader server.

Grafana

First, make sure you have added the Prometheus data source in Grafana. Then, copy the dashboard JSON from grafana.json and import it into your Grafana instance.

Alertmanager

You can set up Alertmanager to receive alerts when certain conditions are met in your Relay instance. As a starting point, you can use our default alertmanager.yaml file. You can customize it to suit your needs.

Logs

By default, Hyvor Relay container writes logs to stdout/stderr. In Docker, you can view the logs using the following command:

docker logs -f relay

You can forward these logs to your log aggregation system using a Docker plugin or another log collector.

Loki

Here is an example Docker Compose configuration for forwarding logs to Loki. First, install the Loki Docker plugin:

arch=$(uname -m)
docker plugin install grafana/loki-docker-driver:3.3.2-${arch} --alias loki --grant-all-permissions

Then, update your compose.yaml to include the logging configuration:

app:
  logging:
	driver: loki
	options:
	  loki-url: "https://mylokiurl.com/loki/api/v1/push"
	  loki-retries: "2"
	  loki-max-backoff: "800ms"
	  loki-timeout: "1s"
	  keep-file: "true"
	  mode: non-blocking

Make sure to replace the loki-url and restart your Docker containers.