feature: Add feature to make API timeout configureable

Fixes: #91

.changelogs/1.0.2/40_add_option_to_run_only_on_cluster_master_node.yml

@@ -0,0 +1,2 @@
+added:
+  - Add option to run ProxLB only on the Proxmox's master node in the cluster (reg. HA feature). [#40]

.changelogs/1.0.2/41_add_option_run_migration_parallel_or_sequential.yml

@@ -0,0 +1,2 @@
+added:
+  - Add option to run migrations in parallel or sequentially. [#41]

.changelogs/1.0.2/45_fix_daemon_timer.yml

@@ -0,0 +1,2 @@
+changed:
+  - Fix daemon timer to use hours instead of minutes. [#45]

.changelogs/1.0.2/49_fix_cmake_debian_packaging.yml

@@ -0,0 +1,2 @@
+fixed:
+  - Fix CMake packaging for Debian package to avoid overwriting the config file. [#49]

.changelogs/1.0.2/release_meta.yml

@@ -0,0 +1 @@
+date: 2024-08-13

.changelogs/1.0.3/51_add_storage_balancing.yml

@@ -0,0 +1,2 @@
+added:
+  - Add storage balancing function. [#51]

.changelogs/1.0.3/53_code_improvements.yml

@@ -0,0 +1,6 @@
+added:
+  - Add a convert function to cast all bool alike options from configparser to bools. [#53]
+  - Add a config parser options for future features. [#53]
+  - Add a config versio schema that must be supported by ProxLB. [#53]
+changed:
+  - Improve the underlying code base for future implementations. [#53]

.changelogs/1.0.3/60_allow_api_hosts_to_be_given_as_an_optional_list.yml

@@ -0,0 +1,2 @@
+added:
+  - Add feature to allow the API hosts being provided as a comma separated list. [#60]

.changelogs/1.0.3/64_improve_error_handling_and_validations.yml

@@ -0,0 +1,2 @@
+fixed:
+  - Improved the overall validation and error handling. [#64]

.changelogs/1.0.3/67_fix_anti_affinity_rules.yml

@@ -0,0 +1,2 @@
+fixed:
+  - Fix anti-affinity rules not evaluating a new and different node. [#67]

.changelogs/1.0.3/68_adjust_logging_ha_services.yml

@@ -0,0 +1,2 @@
+changed:
+  - Provide a more reasonable output when HA services are not active in a Proxmox cluster. [#68]

.changelogs/1.0.3/71_fix_ignore_vm_node_handling_if_unset.yml

@@ -0,0 +1,2 @@
+fixed:
+  - Fix handling of unset `ignore_nodes` and `ignore_vms` resulted in an attribute error. [#71]

.changelogs/1.0.3/74_fix_documentation_master_only.yml

@@ -0,0 +1,2 @@
+fixed:
+  - Fix documentation for the master_only parameter placed in the wrong config section. [#74]

.changelogs/1.0.3/8_add_best_next_node_for_placement.yml

@@ -0,0 +1,2 @@
+added:
+  - Add cli arg `-b` to return the next best node for next VM/CT placement. [#8]

.changelogs/1.0.3/bug_fix_cluster_master_only.yml

@@ -0,0 +1,2 @@
+fixed:
+  - Fixed `master_only` function by inverting the condition.

.changelogs/1.0.3/docs_fix_default_config_add_dedicated_user_howto.yml

@@ -0,0 +1,4 @@
+fixed:
+  - Fix bug in the `proxlb.conf` in the vm_balancing section.
+added:
+  - Add doc how to add dedicated user for authentication. (by @Dulux-Oz) 

.changelogs/1.0.3/release_meta.yml

@@ -0,0 +1 @@
+date: 2024-09-12

.changelogs/1.0.4/58_add_maintenance_mode.yml

@@ -0,0 +1,2 @@
+added:
+  - Add maintenance mode to evacuate a node and move workloads for other nodes in the cluster. [#58]

.changelogs/1.0.4/79_storage_balancing_whitelist.yml

@@ -0,0 +1,3 @@
+changed:
+  - Run storage balancing only on supported shared storages. [#79]
+  - Run storage balancing only when needed to save time. [#79]

.changelogs/1.0.4/81_documentation_infrastrucutre.yml

@@ -0,0 +1,2 @@
+fixed:
+  - Fix documentation for the underlying infrastructure. [#81]

.changelogs/1.0.4/89_add_version_output.yml

@@ -0,0 +1,2 @@
+added:
+  - Add version output cli arg. [#89]

.changelogs/1.0.4/91_make_api_timeout_configureable.yml

@@ -0,0 +1,2 @@
+added:
+  - Add feature to make API timeout configureable. [#91]

.changelogs/1.0.4/release_meta.yml

@@ -0,0 +1 @@
+date: TBD

CHANGELOG.md

@@ -6,6 +6,47 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 
+## [1.0.3] - 2024-09-12
+
+### Added
+
+- Add a convert function to cast all bool alike options from configparser to bools. [#53]
+- Add a config parser options for future features. [#53]
+- Add a config versio schema that must be supported by ProxLB. [#53]
+- Add feature to allow the API hosts being provided as a comma separated list. [#60]
+- Add storage balancing function. [#51]
+- Add doc how to add dedicated user for authentication. (by @Dulux-Oz)
+- Add cli arg `-b` to return the next best node for next VM/CT placement. [#8]
+
+### Changed
+
+- Provide a more reasonable output when HA services are not active in a Proxmox cluster. [#68]
+- Improve the underlying code base for future implementations. [#53]
+
+### Fixed
+
+- Fix documentation for the master_only parameter placed in the wrong config section. [#74]
+- Fixed `master_only` function by inverting the condition.
+- Improved the overall validation and error handling. [#64]
+- Fix bug in the `proxlb.conf` in the vm_balancing section.
+- Fix handling of unset `ignore_nodes` and `ignore_vms` resulted in an attribute error. [#71]
+- Fix anti-affinity rules not evaluating a new and different node. [#67]
+
+
+## [1.0.2] - 2024-08-13
+
+### Added
+
+- Add option to run migration in parallel or sequentially. [#41]
+- Add option to run ProxLB only on the Proxmox's master node in the cluster (reg. HA feature). [#40]
+
+### Changed
+
+- Fix daemon timer to use hours instead of minutes. [#45]
+- Fix CMake packaging for Debian package to avoid overwriting the config file. [#49]
+- Fix wonkey code style.
+
+
 ## [1.0.0] - 2024-08-01
 
 ### Added
@@ -37,4 +78,4 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
-- Development release of ProxLB.
+- Development release of ProxLB.

README.md

@@ -1,5 +1,5 @@
 # ProxLB - (Re)Balance VM Workloads in Proxmox Clusters
-<img align="left" src="https://cdn.gyptazy.ch/images/Prox-LB-logo.jpg"/>
+<img align="left" src="https://cdn.gyptazy.com/images/Prox-LB-logo.jpg"/>
 <br>
 
 <p float="center"><img src="https://img.shields.io/github/license/gyptazy/ProxLB"/><img src="https://img.shields.io/github/contributors/gyptazy/ProxLB"/><img src="https://img.shields.io/github/last-commit/gyptazy/ProxLB/main"/><img src="https://img.shields.io/github/issues-raw/gyptazy/ProxLB"/><img src="https://img.shields.io/github/issues-pr/gyptazy/ProxLB"/></p>
@@ -15,14 +15,16 @@
   - [Usage](#usage)
     - [Dependencies](#dependencies)
     - [Options](#options)
+      - [Notes](#notes)
     - [Parameters](#parameters)
     - [Balancing](#balancing)
       - [General](#general)
       - [By Used Memory of VMs/CTs](#by-used-memory-of-vmscts)
       - [By Assigned Memory of VMs/CTs](#by-assigned-memory-of-vmscts)
-    - [Grouping](#grouping)
-      - [Include (Stay Together)](#include-stay-together)
-      - [Exclude (Stay Separate)](#exclude-stay-separate)
+      - [Storage Balancing](#storage-balancing)
+    - [Affinity Rules / Grouping Relationships](#affinity-rules--grouping-relationships)
+      - [Affinity (Stay Together)](#affinity-stay-together)
+      - [Anti-Affinity (Keep Apart)](#anti-affinity-keep-apart)
       - [Ignore VMs (Tag Style)](#ignore-vms-tag-style)
     - [Systemd](#systemd)
     - [Manual](#manual)
@@ -35,10 +37,13 @@
   - [Downloads](#downloads)
     - [Packages](#packages)
     - [Repository](#repository)
+      - [Stable Releases](#stable-releases)
+      - [Beta/Testing Releases](#betatesting-releases)
     - [Container Images (Docker/Podman)](#container-images-dockerpodman)
   - [Misc](#misc)
     - [Bugs](#bugs)
     - [Contributing](#contributing)
+    - [Documentation](#documentation)
     - [Support](#support)
     - [Author(s)](#authors)
 
@@ -52,13 +57,17 @@ Intelligent rebalancing is a key feature of ProxLB where it re-balances VMs base
 Automated rebalancing reduces the need for manual actions, allowing operators to focus on other critical tasks, thereby increasing operational efficiency.
 
 ### Video of Migration
-<img src="https://cdn.gyptazy.ch/images/proxlb-rebalancing-demo.gif"/>
+<img src="https://cdn.gyptazy.com/images/proxlb-rebalancing-demo.gif"/>
 
 ## Features
-* Rebalance the cluster by:
+* Rebalance VMs/CTs in the cluster by:
   * Memory
   * Disk (only local storage)
   * CPU
+* Rebalance Storage in the cluster
+  * Rebalance VMs/CTs disks to other storage pools
+  * Rebalance by used storage
+* Get best Node for new VM/CT placement in cluster
 * Performing
   * Periodically
   * One-shot solution
@@ -66,6 +75,7 @@ Automated rebalancing reduces the need for manual actions, allowing operators to
   * Rebalance only VMs
   * Rebalance only CTs
   * Rebalance all (VMs and CTs)
+  * Rebalance VM/CT disks (Storage)
 * Filter
   * Exclude nodes
   * Exclude virtual machines
@@ -98,31 +108,44 @@ Running PLB is easy and it runs almost everywhere since it just depends on `Pyth
 ### Options
 The following options can be set in the `proxlb.conf` file:
 
-| Option | Example | Description |
-|------|:------:|:------:|
-| api_host | hypervisor01.gyptazy.ch | Host or IP address of the remote Proxmox API. |
-| api_user | root@pam | Username for the API. |
-| api_pass | FooBar | Password for the API. |
-| verify_ssl | 1 | Validate SSL certificates (1) or ignore (0). (default: 1) |
-| method | memory | Defines the balancing method (default: memory) where you can use `memory`, `disk` or `cpu`. |
-| mode | used | Rebalance by `used` resources (efficiency) or `assigned` (avoid overprovisioning) resources. (default: used)|
-| mode_option | byte | Rebalance by node's resources in `bytes` or `percent`. (default: bytes) |
-| type | vm | Rebalance only `vm` (virtual machines), `ct` (containers) or `all` (virtual machines & containers). (default: vm)|
-| balanciness | 10 | Value of the percentage of lowest and highest resource consumption on nodes may differ before rebalancing. (default: 10) |
-| ignore_nodes | dummynode01,dummynode02,test* | Defines a comma separated list of nodes to exclude. |
-| ignore_vms | testvm01,testvm02 | Defines a comma separated list of VMs to exclude. (`*` as suffix wildcard or tags are also supported) |
-| daemon | 1 | Run as a daemon (1) or one-shot (0). (default: 1) |
-| schedule | 24 | Hours to rebalance in hours. (default: 24) |
-| log_verbosity | INFO | Defines the log level (default: CRITICAL) where you can use `INFO`, `WARN` or `CRITICAL` |
+| Section | Option | Example | Description |
+|------|:------:|:------:|:------:|
+| `proxmox` | api_host | hypervisor01.gyptazy.com | Host or IP address (or comma separated list) of the remote Proxmox API. |
+| | api_user | root@pam | Username for the API. |
+| | api_pass | FooBar | Password for the API. |
+| | verify_ssl | 1 | Validate SSL certificates (1) or ignore (0). (default: 1) |
+| | timeout | 10 | Timeout for the Proxmox API in sec. (default: 10) |
+| `vm_balancing` | enable | 1 | Enables VM/CT balancing. |
+| | method | memory | Defines the balancing method (default: memory) where you can use `memory`, `disk` or `cpu`. |
+| | mode | used | Rebalance by `used` resources (efficiency) or `assigned` (avoid overprovisioning) resources. (default: used)|
+| | mode_option | byte | Rebalance by node's resources in `bytes` or `percent`. (default: bytes) |
+| | type | vm | Rebalance only `vm` (virtual machines), `ct` (containers) or `all` (virtual machines & containers). (default: vm)|
+| | balanciness | 10 | Value of the percentage of lowest and highest resource consumption on nodes may differ before rebalancing. (default: 10) |
+| | parallel_migrations | 1 | Defines if migrations should be done parallely or sequentially. (default: 1) |
+| | maintenance_nodes | dummynode03,dummynode04 | Defines a comma separated list of nodes to set them into maintenance mode and move VMs/CTs to other nodes. |
+| | ignore_nodes | dummynode01,dummynode02,test* | Defines a comma separated list of nodes to exclude. |
+| | ignore_vms | testvm01,testvm02 | Defines a comma separated list of VMs to exclude. (`*` as suffix wildcard or tags are also supported) |
+| `storage_balancing` | enable | 0 | Enables storage balancing. |
+| | balanciness | 10 | Value of the percentage of lowest and highest storage consumption may differ before rebalancing. (default: 10) |
+| | parallel_migrations | 1 | Defines if migrations should be done parallely or sequentially. (default: 1) |
+| `update_service` | enable | 0 | Enables the automated update service (rolling updates). |
+| `api` | enable | 0 | Enables the ProxLB API. |
+| `service`| daemon | 1 | Run as a daemon (1) or one-shot (0). (default: 1) |
+| | schedule | 24 | Hours to rebalance in hours. (default: 24) |
+| | master_only | 0 | Defines is this should only be performed (1) on the cluster master node or not (0). (default: 0) |
+| | log_verbosity | INFO | Defines the log level (default: CRITICAL) where you can use `INFO`, `WARN` or `CRITICAL` |
+| | config_version | 3 | Defines the current config version schema for ProxLB |
 
 An example of the configuration file looks like:
 ```
 [proxmox]
-api_host: hypervisor01.gyptazy.ch
+api_host: hypervisor01.gyptazy.com
 api_user: root@pam
 api_pass: FooBar
 verify_ssl: 1
-[balancing]
+timeout: 10
+[vm_balancing]
+enable: 1
 method: memory
 mode: used
 type: vm
@@ -133,10 +156,38 @@ type: vm
 # Rebalancing:     node01: 41% memory consumption :: node02: 52% consumption
 # No rebalancing:  node01: 43% memory consumption :: node02: 50% consumption
 balanciness: 10
+# Enable parallel migrations. If set to 0 it will wait for completed migrations
+# before starting next migration.
+parallel_migrations: 1
+maintenance_nodes: dummynode03,dummynode04
 ignore_nodes: dummynode01,dummynode02
 ignore_vms: testvm01,testvm02
+[storage_balancing]
+enable: 0
+[update_service]
+enable: 0
+[api]
+enable: 0
 [service]
+# The master_only option might be useful if running ProxLB on all nodes in a cluster
+# but only a single one should do the balancing. The master node is obtained from the Proxmox
+# HA status.
+master_only: 0
 daemon: 1
+config_version: 3
+```
+
+#### Notes
+* If running ProxLB on more than one Proxmox node you can set `api_host` to a comma-separated list of each node's IP address or hostname. (Example: `api_host: node01.gyptazy.com,node02.gyptazy.com,node03.gyptazy.com`)
+* The `verify_ssl` parameter can switch between the mode to verify trusted remote certificates. Keep in mind, that even local ones are **not** trusted by default and need to be imported to the truststore.
+* Even when using only the `vm_balancing` mode, ensure to have the other sections listed in your config:
+```
+[storage_balancing]
+enable: 0
+[update_service]
+enable: 0
+[api]
+enable: 0
 ```
 
 ### Parameters
@@ -145,12 +196,15 @@ The following options and parameters are currently supported:
 | Option | Long Option | Description | Default |
 |------|:------:|------:|------:|
 | -c | --config | Path to a config file. | /etc/proxlb/proxlb.conf (default) |
-| -d | --dry-run | Perform a dry-run without doing any actions. | Unset |
-| -j | --json | Return a JSON of the VM movement. | Unset |
+| -d | --dry-run | Performs a dry-run without doing any actions. | Unset |
+| -j | --json | Returns a JSON of the VM movement. | Unset |
+| -b | --best-node | Returns the best next node for a VM/CT placement (useful for further usage with Terraform/Ansible). | Unset |
+| -m | --maintenance | Sets node(s) to maintenance mode & moves workloads away. | Unset |
+| -v | --version | Returns the ProxLB version on stdout. | Unset |
 
 ### Balancing
 #### General
-In general, virtual machines and containers can be rebalanced and moved around nodes in the cluster. Often, this also works without downtime without any further downtimes. However, this does **not** work with containers. LXC based containers will be shutdown, copied and started on the new node. Also to note, live migrations can work fluently without any issues but there are still several things to be considered. This is out of scope for ProxLB and applies in general to Proxmox and your cluster setup. You can find more details about this here: https://pve.proxmox.com/wiki/Migrate_to_Proxmox_VE.
+In general, virtual machines (VMs), containers (CTs) can be rebalanced and moved around nodes or shared storage (storage balancing) in the cluster. Often, this also works without downtime without any further downtimes. However, this does **not** work with containers. LXC based containers will be shutdown, copied and started on the new node. Also to note, live migrations can work fluently without any issues but there are still several things to be considered. This is out of scope for ProxLB and applies in general to Proxmox and your cluster setup. You can find more details about this here: https://pve.proxmox.com/wiki/Migrate_to_Proxmox_VE.
 
 #### By Used Memory of VMs/CTs
 By continuously monitoring the current resource usage of VMs, ProxLB intelligently reallocates workloads to prevent any single node from becoming overloaded. This approach ensures that resources are balanced efficiently, providing consistent and optimal performance across the entire cluster at all times. To activate this balancing mode, simply activate the following option in your ProxLB configuration:
@@ -168,15 +222,30 @@ mode: assigned
 
 Afterwards, restart the service (if running in daemon mode) to activate this rebalancing mode.
 
-### Grouping
-#### Include (Stay Together)
-<img align="left" src="https://cdn.gyptazy.ch/images/plb-rebalancing-include-balance-group.jpg"/> Access the Proxmox Web UI by opening your web browser and navigating to your Proxmox VE web interface, then log in with your credentials. Navigate to the VM you want to tag by selecting it from the left-hand navigation panel. Click on the "Options" tab to view the VM's options, then select "Edit" or "Add" (depending on whether you are editing an existing tag or adding a new one). In the tag field, enter plb_include_ followed by your unique identifier, for example, plb_include_group1. Save the changes to apply the tag to the VM. Repeat these steps for each VM that should be included in the group.
+#### Storage Balancing
+Starting with ProxLB 1.0.3, ProxLB also supports the balancing of underlying shared storage. In this case, all attached disks (`rootfs` in a context of a CT) of a VM or CT are being fetched and evaluated. If a VM has multiple disks attached, the disks can also be distributed over different storages. As a result, only shared storage is supported. Non shared storage would require to move the whole VM including all attached disks to the parent's node local storage.
 
-#### Exclude (Stay Separate)
-<img align="left" src="https://cdn.gyptazy.ch/images/plb-rebalancing-exclude-balance-group.jpg"/> Access the Proxmox Web UI by opening your web browser and navigating to your Proxmox VE web interface, then log in with your credentials. Navigate to the VM you want to tag by selecting it from the left-hand navigation panel. Click on the "Options" tab to view the VM's options, then select "Edit" or "Add" (depending on whether you are editing an existing tag or adding a new one). In the tag field, enter plb_exclude_ followed by your unique identifier, for example, plb_exclude_critical. Save the changes to apply the tag to the VM. Repeat these steps for each VM that should be excluded from being on the same node.
+Limitations:
+* Only shared storage
+* Only supported for the following VM disk types:
+    * ide (only disks, not CD)
+    * nvme
+    * scsi
+    * virtio
+    * sata
+    * rootfs (Container)
+
+*Note: Storage balancing is currently in beta and should be used carefully.*
+
+### Affinity Rules / Grouping Relationships
+#### Affinity (Stay Together)
+<img align="left" src="https://cdn.gyptazy.com/images/plb-rebalancing-include-balance-group.jpg"/> Access the Proxmox Web UI by opening your web browser and navigating to your Proxmox VE web interface, then log in with your credentials. Navigate to the VM you want to tag by selecting it from the left-hand navigation panel. Click on the "Options" tab to view the VM's options, then select "Edit" or "Add" (depending on whether you are editing an existing tag or adding a new one). In the tag field, enter plb_include_ followed by your unique identifier, for example, plb_include_group1. Save the changes to apply the tag to the VM. Repeat these steps for each VM that should be included in the group.
+
+#### Anti-Affinity (Keep Apart)
+<img align="left" src="https://cdn.gyptazy.com/images/plb-rebalancing-exclude-balance-group.jpg"/> Access the Proxmox Web UI by opening your web browser and navigating to your Proxmox VE web interface, then log in with your credentials. Navigate to the VM you want to tag by selecting it from the left-hand navigation panel. Click on the "Options" tab to view the VM's options, then select "Edit" or "Add" (depending on whether you are editing an existing tag or adding a new one). In the tag field, enter plb_exclude_ followed by your unique identifier, for example, plb_exclude_critical. Save the changes to apply the tag to the VM. Repeat these steps for each VM that should be excluded from being on the same node.
 
 #### Ignore VMs (Tag Style)
-<img align="left" src="https://cdn.gyptazy.ch/images/plb-rebalancing-ignore-vm.jpg"/>  In Proxmox, you can ensure that certain VMs are ignored during the rebalancing process by setting a specific tag within the Proxmox Web UI, rather than solely relying on configurations in the ProxLB config file. This can be achieved by adding the tag 'plb_ignore_vm' to the VM. Once this tag is applied, the VM will be excluded from any further rebalancing operations, simplifying the management process.
+<img align="left" src="https://cdn.gyptazy.com/images/plb-rebalancing-ignore-vm.jpg"/>  In Proxmox, you can ensure that certain VMs are ignored during the rebalancing process by setting a specific tag within the Proxmox Web UI, rather than solely relying on configurations in the ProxLB config file. This can be achieved by adding the tag 'plb_ignore_vm' to the VM. Once this tag is applied, the VM will be excluded from any further rebalancing operations, simplifying the management process.
 
 ### Systemd
 When installing a Linux distribution (such as .deb or .rpm) file, this will be shipped with a systemd unit file. The default configuration file will be sourced from `/etc/proxlb/proxlb.conf`.
@@ -193,7 +262,7 @@ A manual installation is possible and also supports BSD based systems. Proxmox R
 The executable must be able to read the config file, if no dedicated config file is given by the `-c` argument, PLB tries to read it from `/etc/proxlb/proxlb.conf`.
 
 ### Proxmox GUI Integration
-<img align="left" src="https://cdn.gyptazy.ch/images/proxlb-GUI-integration.jpg"/> PLB can also be directly be used from the Proxmox Web UI by installing the optional package `pve-proxmoxlb-service-ui` package which has a dependency on the `proxlb` package. For the Web UI integration, it requires to be installed (in addition) on the nodes on the cluster. Afterwards, a new menu item is present in the HA chapter called `Rebalancing`. This chapter provides two possibilities:
+<img align="left" src="https://cdn.gyptazy.com/images/proxlb-GUI-integration.jpg"/> PLB can also be directly be used from the Proxmox Web UI by installing the optional package `pve-proxmoxlb-service-ui` package which has a dependency on the `proxlb` package. For the Web UI integration, it requires to be installed (in addition) on the nodes on the cluster. Afterwards, a new menu item is present in the HA chapter called `Rebalancing`. This chapter provides two possibilities:
 * Rebalancing VM workloads
 * Migrate VM workloads away from a defined node (e.g. maintenance preparation)
 
@@ -201,8 +270,8 @@ The executable must be able to read the config file, if no dedicated config file
 The easiest way to get started is by using the ready-to-use packages that I provide on my CDN and to run it on a Linux Debian based system. This can also be one of the Proxmox nodes itself.
 
 ```
-wget https://cdn.gyptazy.ch/files/amd64/debian/proxlb/proxlb_1.0.0_amd64.deb
-dpkg -i proxlb_1.0.0_amd64.deb
+wget https://cdn.gyptazy.com/files/os/debian/proxlb/proxlb_1.0.3_amd64.deb
+dpkg -i proxlb_1.0.3_amd64.deb
 # Adjust your config
 vi /etc/proxlb/proxlb.conf
 systemctl restart proxlb
@@ -255,47 +324,54 @@ Here you can find some overviews of references for and about the ProxLB (PLB):
 
 | Description | Link |
 |------|:------:|
-| General introduction into ProxLB | https://gyptazy.ch/blog/proxlb-rebalancing-vm-workloads-across-nodes-in-proxmox-clusters/ |
-| Howto install and use ProxLB on Debian to rebalance vm workloads in a Proxmox cluster | https://gyptazy.ch/howtos/howto-install-and-use-proxlb-to-rebalance-vm-workloads-across-nodes-in-proxmox-clusters/ |
+| General introduction into ProxLB | https://gyptazy.com/blog/proxlb-rebalancing-vm-workloads-across-nodes-in-proxmox-clusters/ |
+| Howto install and use ProxLB on Debian to rebalance vm workloads in a Proxmox cluster | https://gyptazy.com/howtos/howto-install-and-use-proxlb-to-rebalance-vm-workloads-across-nodes-in-proxmox-clusters/ |
 
 ## Downloads
 ProxLB can be obtained in man different ways, depending on which use case you prefer. You can use simply copy the code from GitHub, use created packages for Debian or RedHat based systems, use a Repository to keep ProxLB always up to date or simply use a Container image for Docker/Podman.
 
 ### Packages
 Ready to use packages can be found at:
-* https://cdn.gyptazy.ch/files/amd64/debian/proxlb/
-* https://cdn.gyptazy.ch/files/amd64/ubuntu/proxlb/
-* https://cdn.gyptazy.ch/files/amd64/redhat/proxlb/
-* https://cdn.gyptazy.ch/files/amd64/freebsd/proxlb/
-
+* https://cdn.gyptazy.com/files/os/debian/proxlb/
+* https://cdn.gyptazy.com/files/os/ubuntu/proxlb/
+* https://cdn.gyptazy.com/files/os/redhat/proxlb/
 
 ### Repository
 Debian based systems can also use the repository by adding the following line to their apt sources:
 
+#### Stable Releases
 ```
-deb https://repo.gyptazy.ch/ /
+deb https://repo.gyptazy.com/stable /
 ```
 
-The Repository's GPG key can be found at: `https://repo.gyptazy.ch/repo/KEY.gpg`
+#### Beta/Testing Releases
+```
+deb https://repo.gyptazy.com/testing /
+```
+
+The Repository's GPG key can be found at: `https://repo.gyptazy.com/repository.gpg`
 
 You can also simply import it by running:
 
 ```
-# KeyID:  DEB76ADF7A0BAADB51792782FD6A7A70C11226AA
-# SHA256: 5e44fffa09c747886ee37cc6e9e7eaf37c6734443cc648eaf0a9241a89084383  KEY.gpg
+# KeyID:  17169F23F9F71A14AD49EDADDB51D3EB01824F4C
+# UID:    gyptazy Solutions Repository <contact@gyptazy.com>
+# SHA256: 52c267e6f4ec799d40cdbdb29fa518533ac7942dab557fa4c217a76f90d6b0f3  repository.gpg
 
-wget -O /etc/apt/trusted.gpg.d/proxlb.asc https://repo.gyptazy.ch/repo/KEY.gpg
+wget -O /etc/apt/trusted.gpg.d/proxlb.asc https://repo.gyptazy.com/repository.gpg
 ```
 
-*Note: The defined repositories `repo.gyptazy.ch` and `repo.proxlb.de` are the same!*
+*Note: The defined repositories `repo.gyptazy.com` and `repo.proxlb.de` are the same!*
 
 ### Container Images (Docker/Podman)
 Container Images for Podman, Docker etc., can be found at:
 | Version | Image |
 |------|:------:|
-| latest | cr.gyptazy.ch/proxlb/proxlb:latest |
-| v1.0.0 | cr.gyptazy.ch/proxlb/proxlb:v1.0.0 |
-| v0.9.9 | cr.gyptazy.ch/proxlb/proxlb:v0.9.9 |
+| latest | cr.gyptazy.com/proxlb/proxlb:latest |
+| v1.0.3 | cr.gyptazy.com/proxlb/proxlb:v1.0.3 |
+| v1.0.2 | cr.gyptazy.com/proxlb/proxlb:v1.0.2 |
+| v1.0.0 | cr.gyptazy.com/proxlb/proxlb:v1.0.0 |
+| v0.9.9 | cr.gyptazy.com/proxlb/proxlb:v0.9.9 |
 
 ## Misc
 ### Bugs
@@ -304,14 +380,17 @@ Bugs can be reported via the GitHub issue tracker [here](https://github.com/gypt
 ### Contributing
 Feel free to add further documentation, to adjust already existing one or to contribute with code. Please take care about the style guide and naming conventions. You can find more in our [CONTRIBUTING.md](https://github.com/gyptazy/ProxLB/blob/main/CONTRIBUTING.md) file.
 
+### Documentation
+You can also find additional and more detailed documentation within the [docs/](https://github.com/gyptazy/ProxLB/tree/main/docs) directory.
+
 ### Support
-If you need assistance or have any questions, we offer support through our dedicated [chat room](https://matrix.to/#/#proxlb:gyptazy.ch) in Matrix and on Reddit. Join our community for real-time help, advice, and discussions. Connect with us in our dedicated chat room for immediate support and live interaction with other users and developers. You can also visit our [Reddit community](https://www.reddit.com/r/Proxmox/comments/1e78ap3/introducing_proxlb_rebalance_your_vm_workloads/) to post your queries, share your experiences, and get support from fellow community members and moderators. You may also just open directly an issue [here](https://github.com/gyptazy/ProxLB/issues) on GitHub. We are here to help and ensure you have the best experience possible.
+If you need assistance or have any questions, we offer support through our dedicated [chat room](https://matrix.to/#/#proxlb:gyptazy.com) in Matrix and on Reddit. Join our community for real-time help, advice, and discussions. Connect with us in our dedicated chat room for immediate support and live interaction with other users and developers. You can also visit our [GitHub Community](https://github.com/gyptazy/ProxLB/discussions/) to post your queries, share your experiences, and get support from fellow community members and moderators. You may also just open directly an issue [here](https://github.com/gyptazy/ProxLB/issues) on GitHub. We are here to help and ensure you have the best experience possible.
 
 | Support Channel | Link |
 |------|:------:|
-| Matrix | [#proxlb:gyptazy.ch](https://matrix.to/#/#proxlb:gyptazy.ch) |
-| Reddit | [Reddit community](https://www.reddit.com/r/Proxmox/comments/1e78ap3/introducing_proxlb_rebalance_your_vm_workloads/)  |
+| Matrix | [#proxlb:gyptazy.com](https://matrix.to/#/#proxlb:gyptazy.com) |
+| GitHub Community | [GitHub Community](https://github.com/gyptazy/ProxLB/discussions/)
 | GitHub | [ProxLB GitHub](https://github.com/gyptazy/ProxLB/issues) |
 
 ### Author(s)
- * Florian Paul Azim Hoberg @gyptazy (https://gyptazy.ch)
+ * Florian Paul Azim Hoberg @gyptazy (https://gyptazy.com)

docs/02_Configuration.md

@@ -23,4 +23,26 @@ Afterwards, restart the service (if running in daemon mode) to activate this reb
 <img align="left" src="https://cdn.gyptazy.ch/images/plb-rebalancing-exclude-balance-group.jpg"/> Access the Proxmox Web UI by opening your web browser and navigating to your Proxmox VE web interface, then log in with your credentials. Navigate to the VM you want to tag by selecting it from the left-hand navigation panel. Click on the "Options" tab to view the VM's options, then select "Edit" or "Add" (depending on whether you are editing an existing tag or adding a new one). In the tag field, enter plb_exclude_ followed by your unique identifier, for example, plb_exclude_critical. Save the changes to apply the tag to the VM. Repeat these steps for each VM that should be excluded from being on the same node.
 
 ### Ignore VMs (tag style)
-<img align="left" src="https://cdn.gyptazy.ch/images/plb-rebalancing-ignore-vm.jpg"/>  In Proxmox, you can ensure that certain VMs are ignored during the rebalancing process by setting a specific tag within the Proxmox Web UI, rather than solely relying on configurations in the ProxLB config file. This can be achieved by adding the tag 'plb_ignore_vm' to the VM. Once this tag is applied, the VM will be excluded from any further rebalancing operations, simplifying the management process.
+<img align="left" src="https://cdn.gyptazy.ch/images/plb-rebalancing-ignore-vm.jpg"/>  In Proxmox, you can ensure that certain VMs are ignored during the rebalancing process by setting a specific tag within the Proxmox Web UI, rather than solely relying on configurations in the ProxLB config file. This can be achieved by adding the tag 'plb_ignore_vm' to the VM. Once this tag is applied, the VM will be excluded from any further rebalancing operations, simplifying the management process.
+
+## Authentication / User Account / User / Permissions
+### Authentication
+ProxLB also supports different accounts in ProxLB. Therefore, you can simply create a new user and group and add the required roles permissions.
+
+### Creating Dedicated User for Balanciung
+It is recommended to not use the `root@pam` user for balancing. Therefore, creating a new user might be suitable and is very easy to create.
+A new user can be created by the gui, api and cli. The required roles are stated in the next chapter, but you can also use the following lines
+to create a user on the cli with the required roles to balance VMs & CTs.
+
+```
+pveum role add ProxLBAdmin --privs Datastore.Audit,Sys.Audit,VM.Audit,VM.Migrate
+pveum user add proxlb_admin@pve --password <password>
+pveum acl modify / --roles ProxLBAdmin --users proxlb_admin@pve
+```
+
+### Required Roles
+When using ProxLB with a dedicated account, you might also keep the assigned roles low. Therefore, you need to ensure that the newly created user is at least assigned to the following roles:
+* Datastore.Audit (Required for storage evaluation)
+* Sys.Audit (Required to get resource metrics of the nodes)
+* VM.Audit  (Requited to get resource metrics of VMs/CTs)
+* VM.Migrate (Required for migration of VMs/CTs)

docs/03_FAQ.md

@@ -27,6 +27,16 @@ ProxLB is a load-balancing system designed to optimize the distribution of virtu
 
 Before starting any migrations, ProxLB validates that rebalancing actions are necessary and beneficial. Depending on the selected balancing mode — such as CPU, memory, or disk — it creates a balancing matrix. This matrix sorts the VMs by their maximum used or assigned resources, identifying the VM with the highest usage. ProxLB then places this VM on the node with the most free resources in the selected balancing type. This process runs recursively until the operator-defined Balanciness is achieved. Balancing can be defined for the used or max. assigned resources of VMs/CTs.
 
+### ProxLB config version is too low
+ProxLB may run into an error when the used config schema version is too low. This might happen after major changes that require new config options. Please make sure, to use a supported config version in addition to your running ProxLB config.
+
+Example Error:
+```
+Error: [config-version-validator]: ProxLB config version 2 is too low. Required: 3.
+```
+
+The easiest way to solve this, is by taking the minimum required config schema version from a git tag, representing the ProxLB version.
+
 ### Logging
 ProxLB uses the `SystemdHandler` for logging. You can find all your logs in your systemd unit log or in the `journalctl`. In default, ProxLB only logs critical events. However, for further understanding of the balancing it might be useful to change this to `INFO` or `DEBUG` which can simply be done in the [proxlb.conf](https://github.com/gyptazy/ProxLB/blob/main/proxlb.conf#L14) file by changing the `log_verbosity` parameter.
 
@@ -74,4 +84,4 @@ If you need assistance or have any questions, we offer support through our dedic
 |------|:------:|
 | Matrix | [#proxlb:gyptazy.ch](https://matrix.to/#/#proxlb:gyptazy.ch) |
 | Reddit | [Reddit community](https://www.reddit.com/r/Proxmox/comments/1e78ap3/introducing_proxlb_rebalance_your_vm_workloads/)  |
-| GitHub | [ProxLB GitHub](https://github.com/gyptazy/ProxLB/issues) |
+| GitHub | [ProxLB GitHub](https://github.com/gyptazy/ProxLB/issues) |

packaging/CMakeLists.txt

@@ -1,5 +1,5 @@
 cmake_minimum_required(VERSION 3.16)
-project(proxmox-rebalancing-service VERSION 1.0.0)
+project(proxmox-rebalancing-service VERSION 1.0.3)
 
 install(PROGRAMS ../proxlb DESTINATION /bin)
 install(FILES ../proxlb.conf DESTINATION /etc/proxlb)
@@ -30,12 +30,11 @@ set(CPACK_DEBIAN_PACKAGE_ARCHITECTURE "amd64")
 set(CPACK_DEBIAN_PACKAGE_SUMMARY "ProxLB - Rebalance VM workloads across nodes in Proxmox clusters.")
 set(CPACK_DEBIAN_PACKAGE_DESCRIPTION "ProxLB - Rebalance VM workloads across nodes in Proxmox clusters.")
 set(CPACK_DEBIAN_PACKAGE_CONTROL_EXTRA "${CMAKE_CURRENT_SOURCE_DIR}/changelog_debian")
-set(CPACK_DEBIAN_PACKAGE_DEPENDS "python3")
+set(CPACK_DEBIAN_PACKAGE_DEPENDS "python3, python3-proxmoxer")
 set(CPACK_DEBIAN_PACKAGE_LICENSE "GPL 3.0")
 
-
 # Install
 set(CPACK_PACKAGING_INSTALL_PREFIX ${CMAKE_INSTALL_PREFIX})
-set(CPACK_DEBIAN_PACKAGE_CONTROL_EXTRA "${CMAKE_CURRENT_SOURCE_DIR}/postinst")
+set(CPACK_DEBIAN_PACKAGE_CONTROL_EXTRA "${CMAKE_CURRENT_SOURCE_DIR}/postinst;${CMAKE_CURRENT_SOURCE_DIR}/conffiles")
 set(CPACK_RPM_POST_INSTALL_SCRIPT_FILE "${CMAKE_CURRENT_SOURCE_DIR}/postinst")
 include(CPack)

packaging/changelog_debian

@@ -1,3 +1,33 @@
+proxlb (1.0.3) unstable; urgency=low
+
+  * Add a convert function to cast all bool alike options from configparser to bools.
+  * Add a config parser options for future features.
+  * Add a config versio schema that must be supported by ProxLB.
+  * Add feature to allow the API hosts being provided as a comma separated list.
+  * Add storage balancing function.
+  * Add doc how to add dedicated user for authentication. (by @Dulux-Oz)
+  * Add cli arg `-b` to return the next best node for next VM/CT placement.Fix some wonkey code styles.
+  * Provide a more reasonable output when HA services are not active in a Proxmox cluster.
+  * Improve the underlying code base for future implementations.
+  * Fix documentation for the master_only parameter placed in the wrong config section.
+  * Fixed `master_only` function by inverting the condition.
+  * Improved the overall validation and error handling.
+  * Fix bug in the `proxlb.conf` in the vm_balancing section.
+  * Fix handling of unset `ignore_nodes` and `ignore_vms` resulted in an attribute error.
+  * Fix anti-affinity rules not evaluating a new and different node.
+
+ -- Florian Paul Azim Hoberg <gyptazy@gyptazy.ch>  Wed, 11 Sep 2024 17:31:03 +0200
+
+proxlb (1.0.2) unstable; urgency=low
+
+  * Add option to run migration in parallel or sequentially.
+  * Add option to run ProxLB only on a Proxmox cluster master (req. HA feature).
+  * Fix daemon timer to use hours instead of minutes.
+  * Fix CMake packaging for Debian package to avoid overwriting the config file.
+  * Fix some wonkey code styles.
+
+ -- Florian Paul Azim Hoberg <gyptazy@gyptazy.ch>  Tue, 13 Aug 2024 17:28:14 +0200
+
 proxlb (1.0.0) unstable; urgency=low
 
   * Initial release of ProxLB.

packaging/changelog_redhat

@@ -1,3 +1,26 @@
+* Wed Sep 12 2024 Florian Paul Azim Hoberg <gyptazy@gyptazy.ch>
+- Add a convert function to cast all bool alike options from configparser to bools.
+- Add a config parser options for future features.
+- Add a config versio schema that must be supported by ProxLB.
+- Add feature to allow the API hosts being provided as a comma separated list.
+- Add storage balancing function.
+- Add doc how to add dedicated user for authentication. (by @Dulux-Oz)
+- Add cli arg `-b` to return the next best node for next VM/CT placement.Fix some wonkey code styles.
+- Provide a more reasonable output when HA services are not active in a Proxmox cluster.
+- Improve the underlying code base for future implementations.
+- Fix documentation for the master_only parameter placed in the wrong config section.
+- Fixed `master_only` function by inverting the condition.
+- Improved the overall validation and error handling.
+- Fix bug in the `proxlb.conf` in the vm_balancing section.
+- Fix handling of unset `ignore_nodes` and `ignore_vms` resulted in an attribute error.
+- Fix anti-affinity rules not evaluating a new and different node.
+
+* Tue Aug 13 2024 Florian Paul Azim Hoberg <gyptazy@gyptazy.ch>
+- Add option to run migration in parallel or sequentially.
+- Add option to run ProxLB only on a Proxmox cluster master (req. HA feature).
+- Fixed daemon timer to use hours instead of minutes.
+- Fixed some wonkey code styles.
+
 * Thu Aug 01 2024 Florian Paul Azim Hoberg <gyptazy@gyptazy.ch>
 - Initial release of ProxLB.
 

packaging/conffiles

@@ -0,0 +1 @@
+/etc/proxlb/proxlb.conf

proxlb.conf

@@ -3,12 +3,21 @@ api_host: hypervisor01.gyptazy.ch
 api_user: root@pam
 api_pass: FooBar
 verify_ssl: 1
-[balancing]
+[vm_balancing]
+enable: 1
 method: memory
 mode: used
+maintenance_nodes: dummynode03,dummynode04
 ignore_nodes: dummynode01,dummynode02
 ignore_vms: testvm01,testvm02
+[storage_balancing]
+enable: 0
+[update_service]
+enable: 0
+[api]
+enable: 0
 [service]
 daemon: 1
 schedule: 24
 log_verbosity: CRITICAL
+config_version: 3