feature(grouping): Add grouping feature to rebalance VMs together if defined.

Fixes: #3

.changelogs/1.0.0/3_feature_add_vm_grouping.yaml

@@ -0,0 +1,2 @@
+added:
+  - Add VM grouping feature to migrate vm workload groups together. [#3]

.changelogs/1.0.0/release_meta.yml

@@ -0,0 +1 @@
+date: TBD

README.md

@@ -17,6 +17,7 @@
   * Manuel
   * Proxmox GUI Integration
   * Quick Start
+  * VM Grouping
 * Motivation
 * References
 * Packages
@@ -114,7 +115,7 @@ A manual installation is possible and also supports BSD based systems. Proxmox R
 The executable must be able to read the config file, if no dedicated config file is given by the `-c` argument, PLB tries to read it from `/etc/proxlb/proxlb.conf`.
 
 ### Proxmox GUI Integration
-<img align="left" src="https://cdn.gyptazy.ch/images/proxlb-GUI-integration.jpg"/> PLB can also be directly be used from the Proxmox Web UI by installing the optional package `pve-proxmoxlb-service-ui` package which has a dependency on the `proxlb` package. For the Web UI integration, it requires to be installed (in addition) on the nodes on the cluster. Afterwards, a new menu item is present in the HA chapter called `Rebalancing`. This chapter provides two possibilities:
+<img align="left" src="https://cdn.gyptazy.ch/images/proxlb-GUI-integration.jpg"/> PLB can also be directly be used from the Proxmox Web UI by installing the optional package `pve-proxlb-ui` package which has a dependency on the `proxlb` package. For the Web UI integration, it requires to be installed (in addition) on the nodes on the cluster. Afterwards, a new menu item is present in the HA chapter called `Rebalancing`. This chapter provides two possibilities:
 * Rebalancing VM workloads
 * Migrate VM workloads away from a defined node (e.g. maintenance preparation)
 
@@ -130,6 +131,21 @@ systemctl restart proxlb
 systemctl status proxlb
 ```
 
+### VM Grouping
+<img align="left" src="https://cdn.gyptazy.ch/images/proxlb-vm-grouping-for-rebalancing.jpg"/> In the Proxmox WEB UI, you can group VMs using the notes field. While Proxmox doesn't natively support tagging or flagging VMs, you can utilize the VM's notes/description field for this purpose. You can still include any other notes and comments in the description field, but to enable grouping, you must add a new line starting with `proxlb-grouping:` followed by the group name.
+
+Example:
+```
+This is a great VM
+proxlb-grouping: db-gyptazy01-workload-group01
+
+foo bar With some more text.
+Important is only the proxlb-grouping line with a name and
+we can still use this field.
+```
+
+The notes field is evaluated for each VM. All VMs with the same group name (e.g., `db-gyptazy01-workload-group01`) will be rebalanced together on the same host.
+
 ### Logging
 ProxLB uses the `SystemdHandler` for logging. You can find all your logs in your systemd unit log or in the journalctl.
 

docs/03_FAQ.md

@@ -21,3 +21,18 @@ Jul 06 10:25:16 build01 proxlb[7285]:  proxlb: Error: [python-imports]: Could no
 
 Debian/Ubuntu: apt-get install python3-proxmoxer
 If the package is not provided by your systems repository, you can also install it by running `pip3 install proxmoxer`.
+
+### VM Grouping
+<img align="left" src="https://cdn.gyptazy.ch/images/proxlb-vm-grouping-for-rebalancing.jpg"/> In the Proxmox WEB UI, you can group VMs using the notes field. While Proxmox doesn't natively support tagging or flagging VMs, you can utilize the VM's notes/description field for this purpose. You can still include any other notes and comments in the description field, but to enable grouping, you must add a new line starting with `proxlb-grouping:` followed by the group name.
+
+Example:
+```
+This is a great VM
+proxlb-grouping: db-gyptazy01-workload-group01
+
+foo bar With some more text.
+Important is only the proxlb-grouping line with a name and
+we can still use this field.
+```
+
+The notes field is evaluated for each VM. All VMs with the same group name (e.g., `db-gyptazy01-workload-group01`) will be rebalanced together on the same host.

proxlb

@@ -29,6 +29,7 @@ try:
     _imports = True
 except ImportError as error:
     _imports = False
+import re
 import requests
 import sys
 import time
@@ -250,8 +251,13 @@ def get_vm_statistics(api_object, ignore_vms):
     for node in api_object.nodes.get():
         for vm in api_object.nodes(node['node']).qemu.get():
 
+            # Get the comment field for the VM for VM grouping
+            vm_comment       = __get_vm_comment(api_object, node, vm['vmid'])
+            vm_grouping      = __get_vm_grouping(vm_comment)
+
             if vm['status'] == 'running' and vm['name'] not in ignore_vms_list:
                 vm_statistics[vm['name']] = {}
+                vm_statistics[vm['name']]['grouping']       = vm_grouping
                 vm_statistics[vm['name']]['cpu_total']      = vm['cpus']
                 vm_statistics[vm['name']]['cpu_used']       = vm['cpu']
                 vm_statistics[vm['name']]['memory_total']   = vm['maxmem']
@@ -270,6 +276,30 @@ def get_vm_statistics(api_object, ignore_vms):
     return vm_statistics
 
 
+def __get_vm_comment(api_object, node, vmid):
+    """ Get a comment for a VM from a given VMID. """
+    info_prefix  = 'Info: [api-get-vm-comment]:'
+
+    vm_config = api_object.nodes(node['node']).qemu(vmid).config.get()
+    logging.info(f'{info_prefix} Got VM comment from API.')
+    return vm_config.get('description', None)
+
+
+def __get_vm_grouping(vm_comment):
+    """ Get the grouping name from the comment field for each VM if present. """
+    info_prefix  = 'Info: [api-get-vm-grouping-comment]:'
+
+    if vm_comment is None:
+        logging.info(f'{info_prefix} No grouping comment for VM found.')
+        return None
+
+    _comment = re.split("\n", vm_comment)
+    for line in _comment:
+        if line.startswith('proxlb-grouping:'):
+            logging.info(f'{info_prefix} Got grouping comment {line.strip("proxlb-grouping: ")} for VM found.')
+            return line.strip('proxlb-grouping: ')
+
+
 def balancing_calculations(balancing_method, node_statistics, vm_statistics):
     """ Calculate re-balancing of VMs on present nodes across the cluster. """
     error_prefix = 'Error: [rebalancing-calculator]:'
@@ -278,7 +308,6 @@ def balancing_calculations(balancing_method, node_statistics, vm_statistics):
     if balancing_method not in ['memory', 'disk', 'cpu']:
         logging.error(f'{error_prefix} Invalid balancing method: {balancing_method}')
         sys.exit(2)
-        return node_statistics, vm_statistics
 
     sorted_vms = sorted(vm_statistics.items(), key=lambda item: item[1][f'{balancing_method}_used'], reverse=True)
     logging.info(f'{info_prefix} Balancing will be done for {balancing_method} efficiency.')