add alert configuration steps
BIN
simplexalerts/alert.png
Normal file
After Width: | Height: | Size: 546 KiB |
BIN
simplexalerts/alert_rules.png
Normal file
After Width: | Height: | Size: 128 KiB |
34
simplexalerts/contact_points.drawio
Normal file
Before Width: | Height: | Size: 149 KiB After Width: | Height: | Size: 108 KiB |
|
@ -92,7 +92,7 @@ Download the simplex-chat ubuntu release as shown:
|
||||||
Run your client in server mode:
|
Run your client in server mode:
|
||||||
|
|
||||||
|
|
||||||
[user@devnode:~]$ simplex-chat -d clientDB -p 1337
|
[user@devnode:~]$ simplex-chat -d clientDB -p 1337 -x
|
||||||
No user profiles found, it will be created now.
|
No user profiles found, it will be created now.
|
||||||
Please choose your display name.
|
Please choose your display name.
|
||||||
It will be sent to your contacts when you connect.
|
It will be sent to your contacts when you connect.
|
||||||
|
@ -225,14 +225,159 @@ so it looks like this:
|
||||||
|
|
||||||
##### Start the alerter
|
##### Start the alerter
|
||||||
|
|
||||||
Here using docker becomes a hassle, since we are going to have to make the simplex-chat port accessible. For security easons it runs on the loopback interface.
|
Run the container
|
||||||
|
|
||||||
|
sudo docker run -v $(pwd):/config --network="host" --rm simplex-alerter -c /config/config.yml -e 127.0.0.1:1337
|
||||||
|
|
||||||
|
It will connect to the simplex-chat client we started earlier. You can check the metrics to make sure it's running by checking the metrics page
|
||||||
|
|
||||||
|
curl http://localhost:7898/metrics | less
|
||||||
|
# HELP python_gc_objects_collected_total Objects collected during gc
|
||||||
|
# TYPE python_gc_objects_collected_total counter
|
||||||
|
python_gc_objects_collected_total{generation="0"} 600.0
|
||||||
|
python_gc_objects_collected_total{generation="1"} 15.0
|
||||||
|
python_gc_objects_collected_total{generation="2"} 0.0
|
||||||
|
# HELP python_gc_objects_uncollectable_total Uncollectable objects found during GC
|
||||||
|
# TYPE python_gc_objects_uncollectable_total counter
|
||||||
|
python_gc_objects_uncollectable_total{generation="0"} 0.0
|
||||||
|
python_gc_objects_uncollectable_total{generation="1"} 0.0
|
||||||
|
python_gc_objects_uncollectable_total{generation="2"} 0.0
|
||||||
|
# HELP python_gc_collections_total Number of times this generation was collected
|
||||||
|
...
|
||||||
|
...
|
||||||
|
...
|
||||||
|
|
||||||
|
|
||||||
let's create a socat tunnel with:
|
### Configuring a grafana endpoint
|
||||||
|
|
||||||
apt install socat
|
Browse to the contact points page in grafana and click on "add a new contact point"
|
||||||
sudo socat TCP-LISTEN:31337,bind=172.17.0.1,reuseaddr,fork TCP:127.0.0.1:1337&
|
|
||||||
|
|
||||||
Now run our container:
|
|
||||||
|
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
|
||||||
|
Fill out the details, don't forget the path after the URL, as it's how the alerter knows where to deliver the messages
|
||||||
|
|
||||||
|
![new_contact_point.png]
|
||||||
|
|
||||||
|
And click on Test!
|
||||||
|
|
||||||
|
|
||||||
|
Result:
|
||||||
|
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
|
||||||
|
### Configuring an actual alert
|
||||||
|
|
||||||
|
Now that our webhook is ready we can configure an actual alert!
|
||||||
|
|
||||||
|
Go to alert rules and click on "create a new alert rule"
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
Now you need to configure it:
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
- set a name
|
||||||
|
- make sure it uses the Prometheus data source
|
||||||
|
- add an alerting condition (must be True to fire)
|
||||||
|
- Use the preview button to check that the alert would indeed be firing upon creation
|
||||||
|
|
||||||
|
|
||||||
|
#### Keeping things Tidy
|
||||||
|
|
||||||
|
Your alert must live in a folder:
|
||||||
|
|
||||||
|
|
||||||
|
Click on "New folder"
|
||||||
|

|
||||||
|
|
||||||
|
and add a folder name
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
#### Alert Evaluation
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
Alerts are regularly *evaluated* by grafana. Which means grafana will run the query at specific intervals and fire the alert if the conditions specified are filled.
|
||||||
|
|
||||||
|
|
||||||
|
Let's imagine that we want to keep a close eye on this alert, as if 0 ever not equals 0 then we will have big problems.
|
||||||
|
|
||||||
|
|
||||||
|
First configure an evaluation group and set its name. You can leave the one minute evaluation timing as it's the shortest.
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
|
||||||
|
You can leave all other options to their default.
|
||||||
|
|
||||||
|
|
||||||
|
#### Alert Contact Point
|
||||||
|
|
||||||
|
Now we are going to use the alert contact point we created earlier:
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
Choose from the drop-down menu the web hook we configured.
|
||||||
|
|
||||||
|
|
||||||
|
#### Alert Message
|
||||||
|
|
||||||
|
When the conditions are fulfilled you want an information to be conveyed: that's where you configure it
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
|
||||||
|
#### And now let's blow up some phones
|
||||||
|
|
||||||
|
Save the rule and exit, in 1 minute it will be evaluated and you will receive a a notification
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
|
||||||
|
And here's the alert:
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
|
||||||
|
# Configuring those systems as systemd services
|
||||||
|
|
||||||
|
To turn the simplex-chat and the alerter into systemd services, you only need to create two files:
|
||||||
|
|
||||||
|
## /etc/systemd/system/simplex-chat.service
|
||||||
|
|
||||||
|
[Unit]
|
||||||
|
Requires=tor.service
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
ExecStart=simplex-chat -d /etc/alerter_clientDB -p 1337 -x
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
|
||||||
|
## /etc/systemd/system/alerter.service
|
||||||
|
|
||||||
|
|
||||||
|
[Unit]
|
||||||
|
Requires=simplex-chat.service
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
ExecStart=docker run --rm simplex-alerter -c /etc/alerter-config.yaml -e 127.0.0.1:1337
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=simplex-chat.service
|
||||||
|
|
||||||
|
## Enable the services
|
||||||
|
Now enable the services
|
||||||
|
|
||||||
|
systemctl enable simplex-chat.service
|
||||||
|
systemctl enable alerter.service
|
||||||
|
|
||||||
|
|
||||||
|
# Conclusion
|
||||||
|
|
||||||
|
We now have an easy way to set multiple alerts to different groups based on our monitoring system, furthermore those alerts will be sent over tor through a privacy-preserving messaging system.
|
||||||
|
|
32
simplexalerts/new_alert.drawio
Normal file
BIN
simplexalerts/new_alert.png
Normal file
After Width: | Height: | Size: 171 KiB |
38
simplexalerts/new_contact_point.drawio
Normal file
BIN
simplexalerts/new_contact_point.png
Normal file
After Width: | Height: | Size: 66 KiB |
47
simplexalerts/test_alert.drawio
Normal file
BIN
simplexalerts/test_alert.png
Normal file
After Width: | Height: | Size: 96 KiB |
BIN
simplexalerts/test_alert_2.png
Normal file
After Width: | Height: | Size: 30 KiB |
BIN
simplexalerts/test_alert_2_1.png
Normal file
After Width: | Height: | Size: 15 KiB |
20
simplexalerts/test_alert_3.drawio
Normal file
BIN
simplexalerts/test_alert_3.png
Normal file
After Width: | Height: | Size: 109 KiB |
BIN
simplexalerts/test_alert_3_1.png
Normal file
After Width: | Height: | Size: 33 KiB |
26
simplexalerts/test_alert_4.drawio
Normal file
BIN
simplexalerts/test_alert_4.png
Normal file
After Width: | Height: | Size: 57 KiB |
BIN
simplexalerts/test_alert_5.png
Normal file
After Width: | Height: | Size: 47 KiB |
20
simplexalerts/test_alert_6.drawio
Normal file
BIN
simplexalerts/test_alert_6.png
Normal file
After Width: | Height: | Size: 32 KiB |
BIN
simplexalerts/test_alert_7.png
Normal file
After Width: | Height: | Size: 77 KiB |