How do you monitor and manage your fleet?

[chris400]

As the title says I am curious how you manage your Proxmox environments with it's containers.
While there is a helper script to update the container's OS via a cronjob, you would also need to update the application with either running the script again or a simple 'update' in the container. To do so becomes more and more tedious with the number of containers you are running.

In addition to that I am thinking about actually monitoring the homelab (online status of containers, disk space etc.) and to notify me on my phone.

If you would combine both things I would also like to be notified whenever there is an application update available. I would prefer not do the update automatically, but first take a look at the changelog.

[Mellowlynx]

There are multiple ways, this is what I use currently.

Uptime-kuma
Monitors most of the PVE nodes, LXC containers, and services that run in the containers. (so two checks for a service, it's LXC and service)

Grafana
I use Grafana to monitor the nodes and LXC resources.
It will alert me if something is wrong or has an unusual high load.

Notifications:
For notifications, both in Uptime-Kuma and Grafana I use Pushover.

Update maintenance:
For updates, I made a view Ansible Playbooks.
The OS updates are done twice a week, right after the backup is made. (No backup, no update and message to my phone)
For the applications, I update tabout half by hand and the rest by running a playbook.
When the playbook is finished, I check every service if everything went fine.

Stats:
I run a total of 15 LXC's and 4 VMs in production on 5 nodes.
Maintaining this by hand is still possible, and I do it about once a week.

[di5cord20]

Hi @Mellowlynx, thanks for sharing your approach. I have yet to explore Ansible. If you're open and willing, a guide to document your approach, with an example to get us started, would be greatly appreciated.

Personally, I haven't used Pushover. Like you, I have both Uptime Kuma and Grafana. I leverage Discord updates for Uptime Kuma but haven't setup any notifications for Grafana (yet). Will need to look into that as well.

[rui1610]

Would be interested in such an example as well as I'm trying to fully automate my Proxmox setup with Ansible and Terraform.

[chris400]

Would be interested in such an example as well as I'm trying to fully automate my Proxmox setup with Ansible and Terraform.

How far did you get yet? Anything worth sharing?

[rui1610]

I was able to setup an Ubuntu VM completely automated (no manual installation and configuration needed). So far within a private GitHub repo, but happy to share my code in a public repo in the next days. Will put a link here in the chat once I'm done.

[Mellowlynx]

I will see what I can share soon.
Currently, I'm sick and not match online.

I will see what I can share as most is from the place I work at.