Merge pull request #267 from gyptazy/release/1.1.5

release: Create release 1.1.5

.changelogs/1.1.0/114_refactor_code_base.yml

@@ -1,6 +1,5 @@
 fixed:
   - Refactored code base for ProxLB [#114]
-  - Renamed package from `proxlb` to `python3-proxlb` to align with Debian packaging guidelines [#114]
   - Switched to `pycodestyle` for linting [#114]
   - Package building will be done within GitHub actions pipeline [#114]
   - ProxLB now only returns a warning when no guests for further balancing are not present (instead of quitting) [132#]
@@ -9,4 +8,4 @@ fixed:
   - Stop balancing when movement would get worste (new force param to enfoce for affinity rules) [#128]
   - Added requested documentation regarding Proxmox HA groups [#127]
   - Rewrite of the whole affinity/anti-affinity rules evaluation and placement [#123]
-  - Fixed the `ignore` parameter for nodes where the node and guests on the node will be untouched [#102]
+  - Fixed the `ignore` parameter for nodes where the node and guests on the node will be untouched [#102]

.changelogs/1.1.0/137_fix_systemd_unit_file.yml

@@ -0,0 +1,2 @@
+fixed:
+  - Fix the systemd unit file to start ProxLB after pveproxy (by @robertdahlem). [#137]

.changelogs/1.1.0/release_meta.yml

@@ -1 +1 @@
-date: TBD
+date: 2025-04-01

.changelogs/1.1.1/163_fix_ignore_vm_tag.yml

@@ -0,0 +1,2 @@
+fixed:
+  - Fix tag evluation for VMs for being ignored for further balancing [#163]

.changelogs/1.1.1/165_improve_logging_servity.yml

@@ -0,0 +1,2 @@
+fixed:
+  - Improve logging verbosity of messages that had a wrong servity [#165]

.changelogs/1.1.1/168_add_more_flexible_schedules.yml

@@ -0,0 +1,2 @@
+feature:
+  - Add a more flexible way to define schedules in minutes or hours (by @gyptazy) [#168]

.changelogs/1.1.1/171_set_correct_python_path_docker_image.yml

@@ -0,0 +1,2 @@
+fixed:
+  - Fix Python path for Docker entrypoint (by @crandler) [#170]

.changelogs/1.1.1/174_honor_balancing_activation_value.yml

@@ -0,0 +1,2 @@
+fixed:
+  - Honor the value when balancing should not be performed and stop balancing [#174]

.changelogs/1.1.1/176_change_turn_daemon_mode_on_default.yml

@@ -0,0 +1,2 @@
+changed:
+  - Change the default behaviour of the daemon mode to active [#176]

.changelogs/1.1.1/180_change_default_balancing_to_used_instead_assigned.yml

@@ -0,0 +1,2 @@
+changed:
+  - Change the default banalcing mode to used instead of assigned [#180]

.changelogs/1.1.1/184_validate_for_sufficient_user_permissions.yml

@@ -0,0 +1,2 @@
+feature:
+  - Add validation for the minimum required permissions of a user in Proxmox [#184]

.changelogs/1.1.1/185-logging-handler-for-no-systemd-integration.yml

@@ -0,0 +1,2 @@
+fix:
+  - add handler to log messages with severity less than info to the screen when there is no systemd integration, for instance, inside a docker container (by @glitchvern) [#185]

.changelogs/1.1.1/187_allow_use_of_minutes_instead_of_hours.yml

@@ -0,0 +1,2 @@
+fixed:
+  - allow the use of minutes instead of hours and only accept hours or minutes in the format (by @glitchvern) [#187]

.changelogs/1.1.1/195_set_cpu_used_to_cpu_usage_times_core_count.yml

@@ -0,0 +1,2 @@
+fixed:
+  - Set cpu_used to the cpu usage, which is a percent, times the total number of cores to get a number where guest cpu_used can be added to nodes cpu_used and be meaningful (by @glitchvern) [#195]

.changelogs/1.1.1/197_remove_hard_coded_memory_usage_from_lowest_usage_node.yml

@@ -0,0 +1,2 @@
+fixed:
+  - Remove hard coded memory usage from lowest usage node and use method and mode specified in configuration instead (by @glitchvern) [#197]

.changelogs/1.1.1/200_requery_zero_guest_cpu_used.yml

@@ -0,0 +1,2 @@
+fixed:
+  - Requery a guest if that running guest reports 0 cpu usage (by @glitchvern) [#200]

.changelogs/1.1.1/204_fix_migration_log_relationship_of_guest_type.yml

@@ -0,0 +1,2 @@
+fixed:
+  - Fix the guest type relationship in the logs when a migration job failed (by @gyptazy) [#204]

.changelogs/1.1.1/205_add_api_upstream_error_message_on_migration_failure.yml

@@ -0,0 +1,2 @@
+added:
+  - Providing the API upstream error message when migration fails in debug mode (by @gyptazy) [#205]

.changelogs/1.1.1/release_meta.yml

@@ -0,0 +1 @@
+date: 2025-04-20

.changelogs/1.1.2/137_fix_systemd_unit_file.yml

@@ -0,0 +1,2 @@
+fixed:
+  - Fix systemd unit file to run after network on non PVE nodes (by @robertdahlem) [#137]

.changelogs/1.1.2/157_add_proxmox_api_retry_mechanism.yml

@@ -0,0 +1,2 @@
+added:
+  - Add a configurable retry mechanism when connecting to the Proxmox API (by @gyptazy) [#157]

.changelogs/1.1.2/218_add_1_to_one_relationship_guest_node_pinning.yml

@@ -0,0 +1,2 @@
+added:
+  - Add 1-to-1 relationships between guest and hypervisor node to ping a guest on a node (by @gyptazy) [#218]

.changelogs/1.1.2/222_fix_force_type_cast_cpu_metrics_to_int.yml

@@ -0,0 +1,2 @@
+fixed:
+  - Force type cast cpu count of guests to int for some corner cases where a str got returned (by @gyptazy). [#222]

.changelogs/1.1.2/release_meta.yml

@@ -0,0 +1 @@
+date: 2025-05-13

.changelogs/1.1.3/189_add_reload_function.yml

@@ -0,0 +1,2 @@
+added:
+  - Add relaod (SIGHUP) function to ProxLB to reload the configuration (by @gyptazy). [#189]

.changelogs/1.1.3/232_align_maintenance_mode_proxmox_ha.yml

@@ -0,0 +1,2 @@
+fixed:
+  - Align maintenance mode with Proxmox HA maintenance mode (by @gyptazy). [#232]

.changelogs/1.1.3/239_add_optional_delay_time_until_service_starts.yml

@@ -0,0 +1,2 @@
+added:
+  - Add optional wait time parameter to delay execution until the service takes action (by @gyptazy). #239

.changelogs/1.1.3/241_make_amount_of_parallel_migrations_configurable.yml

@@ -0,0 +1,2 @@
+added:
+  - Make the amount of parallel migrations configurable (by @gyptazy). [#241]

.changelogs/1.1.3/94_cpu_balancing_use_avg_instead_current_consumption.yml

@@ -0,0 +1,2 @@
+changed:
+  - Use the average CPU consumption of a guest within the last 60 minutes instead of the current CPU usage (by @philslab-ninja & @gyptazy). [#94]

.changelogs/1.1.3/release_meta.yml

@@ -0,0 +1 @@
+date: 2025-06-19

.changelogs/1.1.4/245_add_guest_pinning_to_groups_of_nodes.yml

@@ -0,0 +1,2 @@
+added:
+  - Allow pinning of guests to a group of nodes (@gyptazy). [#245]

.changelogs/1.1.4/248_fix_dry_run_combination_balancing_disabled.yml

@@ -0,0 +1,2 @@
+fixed:
+  - Fixed an issue where balancing was performed in combination of deactivated balancing and dry-run mode (@gyptazy). [#248]

.changelogs/1.1.4/255_fix_loglevels.yml

@@ -0,0 +1,2 @@
+fixed:
+  - Modified log levels to make output lighter at INFO level (@pmarasse) [#255]

.changelogs/1.1.4/release_meta.yml

@@ -0,0 +1 @@
+date: 2025-06-27

.changelogs/1.1.5/260_allow_custom_api_ports.yml

@@ -0,0 +1,2 @@
+added:
+  - Allow custom API ports instead of fixed tcp/8006 (@gyptazy). [#260]

.changelogs/1.1.5/release_meta.yml

@@ -0,0 +1 @@
+date: 2025-07-14

.github/workflows/20-pipeline-build-deb-package.yml

@@ -28,7 +28,7 @@ jobs:
       - name: Check out repository
         uses: actions/checkout@v3
         with:
-          ref: 'development'
+          ref: ${{ github.ref }}
 
       - name: Set up Docker with Debian image
         run: |
@@ -39,9 +39,13 @@ jobs:
           docker run --rm -v $(pwd):/workspace -w /workspace debian:latest bash -c "
             # Install dependencies
             apt-get update && \
-            apt-get install -y python3 python3-setuptools debhelper dh-python python3-pip python3-stdeb python3-proxmoxer python3-requests python3-urllib3 && \
-            # Build package
-            python3 setup.py --command-packages=stdeb.command bdist_deb && \
+            apt-get install -y python3 python3-setuptools debhelper dh-python python3-pip python3-stdeb python3-proxmoxer python3-requests python3-urllib3 devscripts python3-all && \
+            # Build package using stdeb / setuptools
+            # python3 setup.py --command-packages=stdeb.command bdist_deb && \
+            # Build native package
+            dpkg-buildpackage -us -uc && \
+            mkdir package && \
+            mv ../*.deb package/ && \
             echo 'OK: Debian package successfully created.'
           "
 
@@ -49,7 +53,7 @@ jobs:
         uses: actions/upload-artifact@v4
         with:
           name: debian-package
-          path: deb_dist/*.deb
+          path: package/*.deb
 
   integration-test-debian:
     needs: build-package-debian
@@ -59,15 +63,16 @@ jobs:
         uses: actions/download-artifact@v4
         with:
           name: debian-package
-          path: deb_dist/
+          path: package/
 
       - name: Set up Docker with Debian image
         run: docker pull debian:latest
 
       - name: Install and test Debian package in Docker container
         run: |
-          docker run --rm -v $(pwd)/deb_dist:/deb_dist -w /deb_dist debian:latest bash -c "
+          docker run --rm -v $(pwd)/package:/package -w /package debian:latest bash -c "
             apt-get update && \
-            apt-get install -y ./python3-proxlb*.deb && \
+            apt-get install -y systemd && \
+            apt-get install -y ./proxlb*.deb && \
             python3 -c 'import proxlb; print(\"OK: Debian package successfully installed.\")'
           "

.github/workflows/30-pipeline-build-container-amd64.yml

@@ -0,0 +1,26 @@
+name: "Build Container Image: AMD64"
+on: [push]
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v4
+    - name: Set up QEMU
+      uses: docker/setup-qemu-action@v3
+    - name: Set up Docker Buildx
+      uses: docker/setup-buildx-action@v3
+    - name: Build amd64 image and save as tar
+      run: |
+        docker buildx build \
+          --platform linux/amd64 \
+          --load \
+          -t proxlb-image:amd64 \
+          .
+
+        docker save proxlb-image:amd64 -o proxlb_image_amd64.tar
+    - name: Upload Docker image artifact
+      uses: actions/upload-artifact@v4
+      with:
+        name: proxlb-image-amd64
+        path: proxlb_image_amd64.tar

.github/workflows/30-pipeline-build-container-arm64.yml

@@ -0,0 +1,26 @@
+name: "Build Container Image: ARM64"
+on: [push]
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v4
+    - name: Set up QEMU
+      uses: docker/setup-qemu-action@v3
+    - name: Set up Docker Buildx
+      uses: docker/setup-buildx-action@v3
+    - name: Build arm64 image and save as tar
+      run: |
+        docker buildx build \
+          --platform linux/arm64 \
+          --load \
+          -t proxlb-image:arm64 \
+          .
+
+        docker save proxlb-image:arm64 -o proxlb_image_arm64.tar
+    - name: Upload Docker image artifact
+      uses: actions/upload-artifact@v4
+      with:
+        name: proxlb-image-arm64
+        path: proxlb_image_arm64.tar

.github/workflows/30-pipeline-build-container-multi-arch.yml

@@ -0,0 +1,23 @@
+name: "Build Container Image: Multiarch"
+on: [push]
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v4
+    - name: Set up QEMU
+      uses: docker/setup-qemu-action@v3
+    - name: Set up Docker Buildx
+      uses: docker/setup-buildx-action@v3
+    - name: Build multi-arch image and save as tar
+      run: |
+        docker buildx build \
+          --platform linux/amd64,linux/arm64 \
+          --output type=tar,dest=proxlb_image_multiarch.tar \
+          .
+    - name: Upload Docker image artifact
+      uses: actions/upload-artifact@v4
+      with:
+        name: proxlb-image-multiarch
+        path: proxlb_image_multiarch.tar

CHANGELOG.md

@@ -0,0 +1,213 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+
+## [1.1.5] - 2025-07-14
+
+### Added
+
+- Allow custom API ports instead of fixed tcp/8006 (@gyptazy). [#260]
+
+
+## [1.1.4] - 2025-06-27
+
+### Added
+
+- Allow pinning of guests to a group of nodes (@gyptazy). [#245]
+
+### Fixed
+
+- Modified log levels to make output lighter at INFO level (@pmarasse) [#255]
+- Fixed an issue where balancing was performed in combination of deactivated balancing and dry-run mode (@gyptazy). [#248]
+
+
+## [1.1.3] - 2025-06-19
+
+### Added
+
+- Add relaod (SIGHUP) function to ProxLB to reload the configuration (by @gyptazy). [#189]
+- Add optional wait time parameter to delay execution until the service takes action (by @gyptazy). [#239]
+- Make the amount of parallel migrations configurable (by @gyptazy). [#241]
+
+### Changed
+
+- Use the average CPU consumption of a guest within the last 60 minutes instead of the current CPU usage (by @philslab-ninja & @gyptazy). [#94]
+
+### Fixed
+
+- Align maintenance mode with Proxmox HA maintenance mode (by @gyptazy). [#232]
+
+
+## [1.1.2] - 2025-05-13
+
+### Added
+
+- Add a configurable retry mechanism when connecting to the Proxmox API (by @gyptazy) [#157]
+- Add 1-to-1 relationships between guest and hypervisor node to ping a guest on a node (by @gyptazy) [#218]
+
+### Fixed
+
+- Force type cast cpu count of guests to int for some corner cases where a str got returned (by @gyptazy). [#222]
+- Fix systemd unit file to run after network on non PVE nodes (by @robertdahlem) [#137]
+
+
+## [1.1.1] - 2025-04-20
+
+### Added
+
+- Providing the API upstream error message when migration fails in debug mode (by @gyptazy) [#205]
+
+### Changed
+
+- Change the default behaviour of the daemon mode to active [#176]
+- Change the default banalcing mode to used instead of assigned [#180]
+
+### Fixed
+
+- Set cpu_used to the cpu usage, which is a percent, times the total number of cores to get a number where guest cpu_used can be added to nodes cpu_used and be meaningful (by @glitchvern) [#195]
+- Fix tag evluation for VMs for being ignored for further balancing [#163]
+- Honor the value when balancing should not be performed and stop balancing [#174]
+- allow the use of minutes instead of hours and only accept hours or minutes in the format (by @glitchvern) [#187]
+- Remove hard coded memory usage from lowest usage node and use method and mode specified in configuration instead (by @glitchvern) [#197]
+- Fix the guest type relationship in the logs when a migration job failed (by @gyptazy) [#204]
+- Requery a guest if that running guest reports 0 cpu usage (by @glitchvern) [#200]
+- Fix Python path for Docker entrypoint (by @crandler) [#170]
+- Improve logging verbosity of messages that had a wrong servity [#165]
+
+
+## [1.1.0] - 2025-04-01
+
+### Fixed
+
+- Refactored code base for ProxLB [#114]
+- Switched to `pycodestyle` for linting [#114]
+- Package building will be done within GitHub actions pipeline [#114]
+- ProxLB now only returns a warning when no guests for further balancing are not present (instead of quitting) [132#]
+- All nodes (according to the free resources) will be used now [#130]
+- Fixed logging outputs where highest/lowest were mixed-up [#129]
+- Stop balancing when movement would get worste (new force param to enfoce for affinity rules) [#128]
+- Added requested documentation regarding Proxmox HA groups [#127]
+- Rewrite of the whole affinity/anti-affinity rules evaluation and placement [#123]
+- Fixed the `ignore` parameter for nodes where the node and guests on the node will be untouched [#102]
+
+
+## [1.0.6] - 2024-12-24
+
+### Fixed
+
+- Fix maintenance mode when using cli arg and config mode by using the merged list (by @CartCaved). [#119]
+- Fix that a scheduler time definition of 1 (int) gets wrongly interpreted as a bool (by @gyptazy). [#115]
+
+
+## [1.0.5] - 2024-10-30
+
+### Changed
+
+- Change docs to make bool usage in configs more clear (by @gyptazy). [#104]
+
+### Fixed
+
+- Fix node (and its objects) evaluation when not reachable, e.g., maintenance (by @gyptazy). [#107]
+- Fix migration from local disks (by @greenlogles). [#113]
+- Fix evaluation of maintenance mode where comparing list & string resulted in a crash (by @glitchvern). [#106]
+- Fix allowed values (add DEBUG, WARNING) for log verbosity (by @gyptazy). [#98]
+
+
+## [1.0.4] - 2024-10-11
+
+### Added
+
+- Add maintenance mode to evacuate a node and move workloads for other nodes in the cluster. [#58]
+- Add feature to make API timeout configureable. [#91]
+- Add version output cli arg. [#89]
+
+### Changed
+
+- Run storage balancing only on supported shared storages. [#79]
+- Run storage balancing only when needed to save time. [#79]
+
+### Fixed
+
+- Fix CPU balancing where calculations are done in float instead of int. (by @glitchvern) [#75]
+- Fix documentation for the underlying infrastructure. [#81]
+
+
+## [1.0.3] - 2024-09-12
+
+### Added
+
+- Add cli arg `-b` to return the next best node for next VM/CT placement. [#8]
+- Add a convert function to cast all bool alike options from configparser to bools. [#53]
+- Add a config parser options for future features. [#53]
+- Add a config versio schema that must be supported by ProxLB. [#53]
+- Add feature to allow the API hosts being provided as a comma separated list. [#60]
+- Add doc how to add dedicated user for authentication. (by @Dulux-Oz)
+- Add storage balancing function. [#51]
+
+### Changed
+
+- Provide a more reasonable output when HA services are not active in a Proxmox cluster. [#68]
+- Improve the underlying code base for future implementations. [#53]
+
+### Fixed
+
+- Fix anti-affinity rules not evaluating a new and different node. [#67]
+- Fixed `master_only` function by inverting the condition.
+- Fix documentation for the master_only parameter placed in the wrong config section. [#74]
+- Fix bug in the `proxlb.conf` in the vm_balancing section.
+- Fix handling of unset `ignore_nodes` and `ignore_vms` resulted in an attribute error. [#71]
+- Improved the overall validation and error handling. [#64]
+
+
+## [1.0.2] - 2024-08-13
+
+### Added
+
+- Add option to run ProxLB only on the Proxmox's master node in the cluster (reg. HA feature). [#40]
+- Add option to run migrations in parallel or sequentially. [#41]
+
+### Changed
+
+- Fix daemon timer to use hours instead of minutes. [#45]
+
+### Fixed
+
+- Fix CMake packaging for Debian package to avoid overwriting the config file. [#49]
+
+
+## [1.0.0] - 2024-08-01
+
+### Added
+
+- Add feature to prevent VMs from being relocated by defining the 'plb_ignore_vm' tag. [#7]
+- Add feature to prevent VMs from being relocated by defining a wildcard pattern. [#7]
+- Add Docker/Podman support. [#10 by @daanbosch]
+- Add option to rebalance by assigned VM resources to avoid overprovisioning. [#16]
+- Add feature to make log verbosity configurable [#17].
+- Add dry-run support to see what kind of rebalancing would be done. [#6]
+- Add LXC/Container integration. [#27]
+- Add exclude grouping feature to rebalance VMs from being located together to new nodes. [#4]
+- Add include grouping feature to rebalance VMs bundled to new nodes. [#3]
+- Add option_mode to rebalance by node's free resources in percent (instead of bytes). [#29]
+
+### Changed
+
+- Adjusted general logging and log more details.
+
+
+## [0.9.9] - 2024-07-06
+
+### Added
+
+- Initial public development release of ProxLB.
+
+
+## [0.9.0] - 2024-02-01
+
+### Added
+
+- Development release of ProxLB.

Dockerfile

@@ -0,0 +1,28 @@
+# Use the latest Alpine image
+FROM alpine:latest
+
+# Labels
+LABEL maintainer="gyptazy@gyptazy.com"
+LABEL org.label-schema.name="ProxLB"
+LABEL org.label-schema.description="ProxLB - An advanced load balancer for Proxmox clusters."
+LABEL org.label-schema.vendor="gyptazy"
+LABEL org.label-schema.url="https://proxlb.de"
+LABEL org.label-schema.vcs-url="https://github.com/gyptazy/ProxLB"
+
+# Install Python3
+RUN apk add --no-cache python3 py3-pip
+
+# Create a directory for the app
+WORKDIR /app
+
+# Copy the python program from the current directory to /app
+COPY proxlb /app/proxlb
+
+# Copy requirements to the container
+COPY requirements.txt /app/requirements.txt
+
+# Install dependencies in the virtual environment
+RUN pip install --break-system-packages -r /app/requirements.txt
+
+# Set the entry point to use the virtual environment's python
+ENTRYPOINT ["/usr/bin/python3", "/app/proxlb/main.py"]

README.md

@@ -4,10 +4,6 @@
 
 <p float="center"><img src="https://img.shields.io/github/license/gyptazy/ProxLB"/><img src="https://img.shields.io/github/contributors/gyptazy/ProxLB"/><img src="https://img.shields.io/github/last-commit/gyptazy/ProxLB/main"/><img src="https://img.shields.io/github/issues-raw/gyptazy/ProxLB"/><img src="https://img.shields.io/github/issues-pr/gyptazy/ProxLB"/></p>
 
-
-# :warning: Important: ProxLB 1.1.x is coming
-This repository is currently under heavy work and changes. During that time it might come to issues, non working pipelines or wrong documentation. Please select a stable release tag for a suitable version during this time!
-
 ## Table of Contents
 1. [Introduction](#introduction)
 2. [Features](#features)
@@ -15,26 +11,24 @@ This repository is currently under heavy work and changes. During that time it m
 4. [Installation](#installation)
    1. [Requirements / Dependencies](#requirements--dependencies)
    2. [Debian Package](#debian-package)
-   3. [RedHat Package](#redhat-package)
    4. [Container / Docker](#container--docker)
    5. [Source](#source)
-5. [Upgrading](#upgrading)
-   1. [Upgrading from < 1.1.0](#upgrading-from--110)
-   2. [Upgrading from >= 1.1.0](#upgrading-from--110)
-6. [Usage / Configuration](#usage--configuration)
+5. [Usage / Configuration](#usage--configuration)
    1. [GUI Integration](#gui-integration)
    2. [Proxmox HA Integration](#proxmox-ha-integration)
    3. [Options](#options)
-7. [Affinity & Anti-Affinity Rules](#affinity--anti-affinity-rules)
+6. [Affinity & Anti-Affinity Rules](#affinity--anti-affinity-rules)
    1. [Affinity Rules](#affinity-rules)
    2. [Anti-Affinity Rules](#anti-affinity-rules)
-8. [Maintenance](#maintenance)
-9. [Misc](#misc)
+   3. [Ignore VMs](#ignore-vms)
+   4. [Pin VMs to Hypervisor Nodes](#pin-vms-to-hypervisor-nodes)
+7. [Maintenance](#maintenance)
+8. [Misc](#misc)
    1. [Bugs](#bugs)
    2. [Contributing](#contributing)
    3. [Documentation](#documentation)
    4. [Support](#support)
-10. [Author(s)](#authors)
+9. [Author(s)](#authors)
 
 
 ## Introduction
@@ -94,23 +88,131 @@ The dependencies can simply be installed with `pip` by running the following com
 pip install -r requirements.txt
 ```
 
-Distribution packages, such like the provided `.deb` package will automatically resolve and install all required dependencies by using already packaged version from the distribution's repository.
+*Note: Distribution packages, such like the provided `.deb` package will automatically resolve and install all required dependencies by using already packaged version from the distribution's repository. By using the Docker (container) image or Debian packages, you do not need to take any care of the requirements listed here.*
 
 ### Debian Package
+ProxLB is a powerful and flexible load balancer designed to work across various architectures, including `amd64`, `arm64`, `rv64` and many other ones that support Python. It runs independently of the underlying hardware, making it a versatile choice for different environments. This chapter covers the step-by-step process to install ProxLB on Debian-based systems, including Debian clones like Ubuntu.
 
-### RedHat Package
+#### Quick-Start
+You can simply use this snippet to install the repository and to install ProxLB on your system.
 
-### Container / Docker
+```bash
+echo "deb https://repo.gyptazy.com/stable /" > /etc/apt/sources.list.d/proxlb.list
+wget -O /etc/apt/trusted.gpg.d/proxlb.asc https://repo.gyptazy.com/repository.gpg
+apt-get update && apt-get -y install proxlb
+cp /etc/proxlb/proxlb_example.yaml /etc/proxlb/proxlb.yaml
+# Adjust the config to your needs
+vi /etc/proxlb/proxlb.yaml
+systemctl start proxlb
+```
+
+Afterwards, ProxLB is running in the background and balances your cluster by your defined balancing method (default: memory).
+
+#### Details
+ProxLB provides two different repositories:
+* https://repo.gyptazy.com/stable (only stable release)
+* https://repo.gyptazy.com/testing (bleeding edge - not recommended)
+
+The repository is signed and the GPG key can be found at:
+* https://repo.gyptazy.com/repository.gpg
+
+You can also simply import it by running:
+
+```
+# KeyID:  17169F23F9F71A14AD49EDADDB51D3EB01824F4C
+# UID:    gyptazy Solutions Repository <contact@gyptazy.com>
+# SHA256: 52c267e6f4ec799d40cdbdb29fa518533ac7942dab557fa4c217a76f90d6b0f3  repository.gpg
+
+wget -O /etc/apt/trusted.gpg.d/proxlb.asc https://repo.gyptazy.com/repository.gpg
+```
+
+*Note: The defined repositories `repo.gyptazy.com` and `repo.proxlb.de` are the same!*
+
+#### Debian Packages (.deb files)
+If you do not want to use the repository you can also find the debian packages as a .deb file on gyptazy's CDN at:
+* https://cdn.gyptazy.com/files/os/debian/proxlb/
+
+Afterwards, you can simply install the package by running:
+```bash
+dpkg -i proxlb_*.deb
+cp /etc/proxlb/proxlb_example.yaml /etc/proxlb/proxlb.yaml
+# Adjust the config to your needs
+vi /etc/proxlb/proxlb.yaml
+systemctl start proxlb
+```
+
+### Container Images / Docker
+Using the ProxLB container images is straight forward and only requires you to mount the config file.
+
+```bash
+# Pull the image
+docker pull cr.gyptazy.com/proxlb/proxlb:latest
+# Download the config
+wget -O proxlb.yaml https://raw.githubusercontent.com/gyptazy/ProxLB/refs/heads/main/config/proxlb_example.yaml
+# Adjust the config to your needs
+vi proxlb.yaml
+# Start the ProxLB container image with the ProxLB config
+docker run -it --rm -v $(pwd)/proxlb.yaml:/etc/proxlb/proxlb.yaml proxlb
+```
+
+*Note: ProxLB container images are officially only available at cr.proxlb.de and cr.gyptazy.com.*
+
+#### Overview of Images
+| Version | Image |
+|------|:------:|
+| latest | cr.gyptazy.com/proxlb/proxlb:latest |
+| v1.1.5 | cr.gyptazy.com/proxlb/proxlb:v1.1.5 |
+| v1.1.4 | cr.gyptazy.com/proxlb/proxlb:v1.1.4 |
+| v1.1.3 | cr.gyptazy.com/proxlb/proxlb:v1.1.3 |
+| v1.1.2 | cr.gyptazy.com/proxlb/proxlb:v1.1.2 |
+| v1.1.1 | cr.gyptazy.com/proxlb/proxlb:v1.1.1 |
+| v1.1.0 | cr.gyptazy.com/proxlb/proxlb:v1.1.0 |
+| v1.0.6 | cr.gyptazy.com/proxlb/proxlb:v1.0.6 |
+| v1.0.5 | cr.gyptazy.com/proxlb/proxlb:v1.0.5 |
+| v1.0.4 | cr.gyptazy.com/proxlb/proxlb:v1.0.4 |
+| v1.0.3 | cr.gyptazy.com/proxlb/proxlb:v1.0.3 |
+| v1.0.2 | cr.gyptazy.com/proxlb/proxlb:v1.0.2 |
+| v1.0.0 | cr.gyptazy.com/proxlb/proxlb:v1.0.0 |
+| v0.9.9 | cr.gyptazy.com/proxlb/proxlb:v0.9.9 |
 
 ### Source
+ProxLB can also easily be used from the provided sources - for traditional systems but also as a Docker/Podman container image.
 
-## Upgrading
+#### Traditional System
+Setting up and running ProxLB from the sources is simple and requires just a few commands. Ensure Python 3 and the Python dependencies are installed on your system, then run ProxLB using the following command:
+```bash
+git clone https://github.com/gyptazy/ProxLB.git
+cd ProxLB
+```
 
-### Upgrading from < 1.1.0
-Upgrading ProxLB is not supported due to a fundamental redesign introduced in version 1.1.x. With this update, ProxLB transitioned from a monolithic application to a pure Python-style project, embracing a more modular and flexible architecture. This shift aimed to improve maintainability and extensibility while keeping up with modern development practices. Additionally, ProxLB moved away from traditional ini-style configuration files and adopted YAML for configuration management. This change simplifies configuration handling, reduces the need for extensive validation, and ensures better type casting, ultimately providing a more streamlined and user-friendly experience.
+Afterwards simply adjust the config file to your needs:
+```bash
+vi config/proxlb.yaml
+```
 
-### Upgrading from >= 1.1.0
-Uprading within the current stable versions, starting from 1.1.0, will be possible in all supported ways.
+Start ProxLB by Python3 on the system:
+```bash
+python3 proxlb/main.py -c config/proxlb.yaml
+```
+
+#### Container Image
+Creating a container image of ProxLB is straightforward using the provided Dockerfile. The Dockerfile simplifies the process by automating the setup and configuration required to get ProxLB running in an Alpine container. Simply follow the steps in the Dockerfile to build the image, ensuring all dependencies and configurations are correctly applied. For those looking for an even quicker setup, a ready-to-use ProxLB container image is also available, eliminating the need for manual building and allowing for immediate deployment.
+
+```bash
+git clone https://github.com/gyptazy/ProxLB.git
+cd ProxLB
+docker build -t proxlb .
+```
+
+Afterwards simply adjust the config file to your needs:
+```bash
+vi config/proxlb.yaml
+```
+
+Finally, start the created container.
+```bash
+docker run -it --rm -v $(pwd)/proxlb.yaml:/etc/proxlb/proxlb.yaml proxlb
+```
 
 ## Usage / Configuration
 Running ProxLB is straightforward and versatile, as it only requires `Python3` and the `proxmoxer` library. This means ProxLB can be executed directly on a Proxmox node or on dedicated systems such as Debian, RedHat, or even FreeBSD, provided that the Proxmox API is accessible from the client running ProxLB. ProxLB can also run inside a Container - Docker or LXC - and is simply up to you.
@@ -136,46 +238,60 @@ See also: [#65: Host groups: Honour HA groups](https://github.com/gyptazy/ProxLB
 ### Options
 The following options can be set in the configuration file `proxlb.yaml`:
 
-| Section | Option | Example | Type | Description |
-|------|:------:|:------:|:------:|:------:|
-| `proxmox_api` |  |  | |  |
-| | hosts | ['virt01.example.com', '10.10.10.10', 'fe01::bad:code::cafe'] | `List` | List of Proxmox nodes. Can be IPv4, IPv6 or mixed. |
-| | user | root@pam | `Str` | Username for the API. |
-| | pass | FooBar | `Str` | Password for the API. (Recommended: Use API token authorization!) |
-| | token_id | proxlb | `Str` | Token ID of the user for the API. |
-| | token_secret | 430e308f-1337-1337-beef-1337beefcafe | `Str` | Secret of the token ID for the API. |
-| | ssl_verification | True | `Bool` | Validate SSL certificates (1) or ignore (0). (default: 1, type: bool) |
-| | timeout | 10 | `Int` | Timeout for the Proxmox API in sec. (default: 10) |
-| `proxmox_cluster` |  | | |  |
-| | maintenance_nodes | ['virt66.example.com'] | `List` | A list of Proxmox nodes that are defined to be in a maintenance. (default: []) |
-| | ignore_nodes | [] | `List` | A list of Proxmox nodes that are defined to be ignored.  (default: []) |
-| | overprovisioning | False | `Bool` | Avoids balancing when nodes would become overprovisioned. |
-| `balancing` |  | | |  |
-| | enable | True | `Bool` | Enables the guest balancing.  (default: True)|
-| | enforce_affinity | True | `Bool` | Enforcing affinity/anti-affinity rules but balancing might become worse.  (default: False) |
-| | parallel | False | `Bool` | If guests should be moved in parallel or sequentially. (default: False)|
-| | live | True | `Bool` | If guests should be moved live or shutdown.  (default: True)|
-| | with_local_disks | True | `Bool` | If balancing of guests should include local disks  (default: True)|
-| | balance_types | ['vm', 'ct'] | `List` | Defined the types of guests that should be honored.  (default: ['vm', 'ct']) |
-| | max_job_validation | 1800 | `Int` | How long a job validation may take in seconds. (default: 1800) |
-| | balanciness | 10 | `Int` | The maximum delta of resource usage between node with highest and lowest usage. (default: 10) |
-| | method | memory | `Str` | The balancing method that should be used.  (default: memory | choices: memory, cpu, disk)|
-| | mode | used | `Str` | The balancing mode that should be used.  (default: used | choices: used, assigned)|
-| `service` |  | | |  |
-| | daemon | False | `Bool` | If daemon mode should be activated  (default: False)|
-| | schedule | 12 | `Int` | How often rebalancing should occur in hours in daemon mode (default: 12)|
-| | log_level | INFO | `Str` | Defines the default log level that should be logged.  (default: INFO) |
+| Section | Option | Sub Option | Example | Type | Description |
+|---------|:------:|:----------:|:-------:|:----:|:-----------:|
+| `proxmox_api` |  |  |  |  |  |
+|  | hosts |  | ['virt01.example.com', '10.10.10.10', 'fe01::bad:code::cafe', 'virt01.example.com:443', '[fc00::1]', '[fc00::1]:443', 'fc00::1:8006'] | `List` | List of Proxmox nodes. Can be IPv4, IPv6 or mixed. You can specify custom ports. In case of IPv6 without brackets the port is considered after the last colon |
+|  | user |  | root@pam | `Str` | Username for the API. |
+|  | pass |  | FooBar | `Str` | Password for the API. (Recommended: Use API token authorization!) |
+|  | token_id |  | proxlb | `Str` | Token ID of the user for the API. |
+|  | token_secret |  | 430e308f-1337-1337-beef-1337beefcafe | `Str` | Secret of the token ID for the API. |
+|  | ssl_verification |  | True | `Bool` | Validate SSL certificates (1) or ignore (0). [values: `1` (default), `0`] |
+|  | timeout |  | 10 | `Int` | Timeout for the Proxmox API in sec. |
+|  | retries |  | 1 | `Int` | How often a connection attempt to the defined API host should be performed. |
+|  | wait_time |  | 1 | `Int` | How many seconds should be waited before performing another connection attempt to the API host. |
+| `proxmox_cluster` |  |  |  |  |  |
+|  | maintenance_nodes |  | ['virt66.example.com'] | `List` | A list of Proxmox nodes that are defined to be in a maintenance. |
+|  | ignore_nodes |  | [] | `List` | A list of Proxmox nodes that are defined to be ignored. |
+|  | overprovisioning |  | False | `Bool` | Avoids balancing when nodes would become overprovisioned. |
+| `balancing` |  |  |  |  |  |
+|  | enable |  | True | `Bool` | Enables the guest balancing.|
+|  | enforce_affinity |  | True | `Bool` | Enforcing affinity/anti-affinity rules but balancing might become worse. |
+|  | parallel |  | False | `Bool` | If guests should be moved in parallel or sequentially.|
+|  | parallel_jobs |  | 5 | `Int` | The amount if parallel jobs when migrating guests. (default: `5`)|
+|  | live |  | True | `Bool` | If guests should be moved live or shutdown.|
+|  | with_local_disks |  | True | `Bool` | If balancing of guests should include local disks.|
+|  | balance_types |  | ['vm', 'ct'] | `List` | Defined the types of guests that should be honored. [values: `vm`, `ct`]|
+|  | max_job_validation |  | 1800 | `Int` | How long a job validation may take in seconds. (default: 1800) |
+|  | balanciness |  | 10 | `Int` | The maximum delta of resource usage between node with highest and lowest usage. |
+|  | method |  | memory | `Str` | The balancing method that should be used.  [values: `memory` (default), `cpu`, `disk`]|
+|  | mode |  | used | `Str` | The balancing mode that should be used. [values: `used` (default), `assigned`] |
+| `service` |  |  |  |  |  |
+|  | daemon |  | True | `Bool` | If daemon mode should be activated. |
+|  | `schedule` |  |  | `Dict` | Schedule config block for rebalancing. |
+|  |  | interval | 12 | `Int` | How often rebalancing should occur in daemon mode.|
+|  |  | format | hours | `Str` | Sets the time format. [values: `hours` (default), `minutes`]|
+|  | `delay` |  |  | `Dict` | Schedule config block for an optional delay until the service starts. |
+|  |  | enable | False | `Bool` | If a delay time should be validated.|
+|  |  | time | 1 | `Int` | Delay time until the service starts after the initial execution.|
+|  |  | format | hours | `Str` | Sets the time format. [values: `hours` (default), `minutes`]|
+|  | log_level |  | INFO | `Str` | Defines the default log level that should be logged. [values: `INFO` (default), `WARNING`, `CRITICAL`, `DEBUG`] |
+
 
 An example of the configuration file looks like:
 ```
 proxmox_api:
   hosts: ['virt01.example.com', '10.10.10.10', 'fe01::bad:code::cafe']
   user: root@pam
-  #pass: crazyPassw0rd!
-  token_id: proxlb
-  token_secret: 430e308f-1337-1337-beef-1337beefcafe
-  ssl_verification: False
+  pass: crazyPassw0rd!
+  # API Token method
+  # token_id: proxlb
+  # token_secret: 430e308f-1337-1337-beef-1337beefcafe
+  ssl_verification: True
   timeout: 10
+  # API Connection retries
+  # retries: 1
+  # wait_time: 1
 
 proxmox_cluster:
   maintenance_nodes: ['virt66.example.com']
@@ -192,12 +308,18 @@ balancing:
   max_job_validation: 1800
   balanciness: 5
   method: memory
-  mode: assigned
+  mode: used
 
 service:
-  daemon: False
-  schedule: 12
-  log_level: DEBUG
+  daemon: True
+  schedule:
+    interval: 12
+    format: hours
+  delay:
+    enable: False
+    time: 1
+    format: hours
+  log_level: INFO
 ```
 
 ### Parameters
@@ -205,7 +327,7 @@ The following options and parameters are currently supported:
 
 | Option | Long Option | Description | Default |
 |------|:------:|------:|------:|
-| -c | --config | Path to a config file. | /etc/proxlb/proxlb.conf (default) |
+| -c | --config | Path to a config file. | /etc/proxlb/proxlb.yaml (default) |
 | -d | --dry-run | Performs a dry-run without doing any actions. | False |
 | -j | --json | Returns a JSON of the VM movement. | False |
 | -b | --best-node | Returns the best next node for a VM/CT placement (useful for further usage with Terraform/Ansible). | False |
@@ -242,11 +364,49 @@ As a result, ProxLB will try to place the VMs with the `plb_anti_affinity_ntp` t
 
 **Note:** While this ensures that ProxLB tries distribute these VMs across different physical hosts within the Proxmox cluster this may not always work. If you have more guests attached to the group than nodes in the cluster, we still need to run them anywhere. If this case occurs, the next one with the most free resources will be selected.
 
+### Ignore VMs
+<img align="left" src="https://cdn.gyptazy.com/images/proxlb-ignore-vm-movement.jpg"/> Guests, such as VMs or CTs, can also be completely ignored. This means, they won't be affected by any migration (even when (anti-)affinity rules are enforced). To ensure a proper resource evaluation, these guests are still collected and evaluated but simply skipped for balancing actions. Another thing is the implementation. While ProxLB might have a very restricted configuration file including the file permissions, this file is only read- and writeable by the Proxmox administrators. However, we might have user and groups who want to define on their own that their systems shouldn't be moved. Therefore, these users can simpy set a specific tag to the guest object - just like the (anti)affinity rules.
+
+To define a guest to be ignored from the balancing, users assign a tag with the prefix `plb_ignore_$TAG`:
+
+#### Example for Screenshot
+```
+plb_ignore_dev
+```
+
+As a result, ProxLB will not migrate this guest with the `plb_ignore_dev` tag to any other node.
+
+**Note:** Ignored guests are really ignored. Even by enforcing affinity rules this guest will be ignored.
+
+### Pin VMs to Specific Hypervisor Nodes
+<img align="left" src="https://cdn.gyptazy.com/images/proxlb-tag-node-pinning.jpg"/> Guests, such as VMs or CTs, can also be pinned to specific (and multiple) nodes in the cluster. This might be usefull when running applications with some special licensing requirements that are only fulfilled on certain nodes. It might also be interesting, when some physical hardware is attached to a node, that is not available in general within the cluster.
+
+To pin a guest to a specific cluster node, users assign a tag with the prefix `plb_pin_$nodename` to the desired guest:
+
+#### Example for Screenshot
+```
+plb_pin_node03
+```
+
+As a result, ProxLB will pin the guest `dev-vm01` to the node `virt03`.
+
+You can also repeat this step multiple times for different node names to create a potential group of allowed hosts where a the guest may be served on. In this case, ProxLB takes the node with the lowest used resources according to the defined balancing values from this group.
+
+**Note:** The given node names from the tag are validated. This means, ProxLB validated if the given node name is really part of the cluster. In case of a wrongly defined or unavailable node name it continous to use the regular processes to make sure the guest keeps running.
+
 ## Maintenance
 <img src="https://cdn.gyptazy.com/images/proxlb-rebalancing-demo.gif"/>
 
 The `maintenance_nodes` option allows operators to designate one or more Proxmox nodes for maintenance mode. When a node is set to maintenance, no new guest workloads will be assigned to it, and all existing workloads will be migrated to other available nodes within the cluster. This process ensures that (anti)-affinity rules and resource availability are respected, preventing disruptions while maintaining optimal performance across the infrastructure.
 
+### Adding / Removing Nodes from Maintenance
+Within the section `proxmox_cluster` you can define the key `maintenance_nodes` as a list object. Simply add/remove one or more nodes with their equal name in the cluster and restart the daemon.
+```
+proxmox_cluster:
+  maintenance_nodes: ['virt66.example.com']
+```
+Afterwards, all guest objects will be moved to other nodes in the cluster by ensuring the best balancing.
+
 ## Misc
 ### Bugs
 Bugs can be reported via the GitHub issue tracker [here](https://github.com/gyptazy/ProxLB/issues). You may also report bugs via email or deliver PRs to fix them on your own. Therefore, you might also see the contributing chapter.
@@ -272,4 +432,4 @@ Connect with us in our dedicated chat room for immediate support and live intera
 **Note:** Please always keep in mind that this is a one-man show project without any further help. This includes coding, testing, packaging and all the infrastructure around it to keep this project up and running.
 
 ### Author(s)
- * Florian Paul Azim Hoberg @gyptazy (https://gyptazy.com)
+ * Florian Paul Azim Hoberg @gyptazy (https://gyptazy.com)

config/proxlb_example.yaml

@@ -5,8 +5,11 @@ proxmox_api:
   # API Token method
   # token_id: proxlb
   # token_secret: 430e308f-1337-1337-beef-1337beefcafe
-  ssl_verification: False
+  ssl_verification: True
   timeout: 10
+  # API Connection retries
+  # retries: 1
+  # wait_time: 1
 
 proxmox_cluster:
   maintenance_nodes: ['virt66.example.com']
@@ -17,15 +20,24 @@ balancing:
   enable: True
   enforce_affinity: False
   parallel: False
+  # If running parallel job, you can define
+  # the amount of prallel jobs (default: 5)
+  parallel_jobs: 1
   live: True
   with_local_disks: True
   balance_types: ['vm', 'ct']
   max_job_validation: 1800
   balanciness: 5
   method: memory
-  mode: assigned
+  mode: used
 
 service:
-  daemon: False
-  schedule: 12
-  log_level: DEBUG
+  daemon: True
+  schedule:
+    interval: 12
+    format: hours
+  delay:
+    enable: False
+    time: 1
+    format: hours
+  log_level: INFO

debian/changelog

@@ -0,0 +1,54 @@
+proxlb (1.1.5) stable; urgency=medium
+
+  * Allow custom API ports instead of fixed tcp/8006. (Closes: #260)
+
+ -- Florian Paul Azim Hoberg <gyptazy@gyptazy.com>  Mon, 14 Jul 2025 11:07:34 +0000
+
+proxlb (1.1.4) stable; urgency=medium
+
+  * Allow pinning of guests to a group of nodes. (Closes: #245)
+  * Modified log levels to make output lighter at INFO level. (Closes: #255)
+  * ixed an issue where balancing was performed in combination of deactivated balancing and dry-run mode. (Closes: #248)
+
+ -- Florian Paul Azim Hoberg <gyptazy@gyptazy.com>  Fri, 27 Jun 2025 16:22:58 +0000
+
+proxlb (1.1.3) stable; urgency=medium
+
+  * Add relaod (SIGHUP) function to ProxLB to reload the configuration. (Closes: #189)
+  * Add optional wait time parameter to delay execution until the service takes action. (Closes: #239)
+  * Make the amount of parallel migrations configurable. (Closes: #241)
+  * Use the average CPU consumption of a guest within the last 60 minutes instead of the current CPU usage. (Closes: #94)
+  * Align maintenance mode with Proxmox HA maintenance mode. (Closes: #232)
+
+ -- Florian Paul Azim Hoberg <gyptazy@gyptazy.com>  Thu, 19 Jun 2025 09:10:43 +0000
+
+proxlb (1.1.2) stable; urgency=medium
+
+  * Add a configurable retry mechanism when connecting to the Proxmox API. (Closed: #157)
+  * Add 1-to-1 relationships between guest and hypervisor node to ping a guest on a node. (Closes #218)
+  * Force type cast cpu count of guests to int for some corner cases where a str got returned. (Closed #222)
+  * Fix systemd unit file to run after network on non PVE nodes. (Closes #137)
+
+ -- Florian Paul Azim Hoberg <gyptazy@gyptazy.com>  Mon, 13 May 2025 18:12:04 +0000
+
+proxlb (1.1.1) stable; urgency=medium
+
+  * Fix tag evluation for VMs for being ignored for further balancing. (Closes: #163)
+  * Improve logging verbosity of messages that had a wrong servity. (Closes: #165)
+  * Providing the API upstream error message when migration fails in debug mode (Closes: #205)
+  * Change the default behaviour of the daemon mode to active. (Closes: #176)
+  * Change the default banalcing mode to used instead of assigned. (Closes: #180)
+  * Set cpu_used to the cpu usage, which is a percent, times the total number of cores to get a number where guest cpu_used can be added to nodes cpu_used and be meaningful. (Closes: #195)
+  * Honor the value when balancing should not be performed and stop balancing. (Closes: #174)
+  * Allow the use of minutes instead of hours and only accept hours or minutes in the format. (Closes: #187)
+  * Remove hard coded memory usage from lowest usage node and use method and mode specified in configuration instead. (Closes: #197)
+  * Fix the guest type relationship in the logs when a migration job failed. (Closes: #204)
+  * Requery a guest if that running guest reports 0 cpu usage. (Closes: #200)
+
+ -- Florian Paul Azim Hoberg <gyptazy@gyptazy.com>  Sat, 20 Apr 2025 20:55:02 +0000
+
+proxlb (1.1.0) stable; urgency=medium
+
+  * Refactored code base of ProxLB. (Closes: #114)
+
+ -- Florian Paul Azim Hoberg <gyptazy@gyptazy.com>  Mon, 17 Mar 2025 18:55:02 +0000

debian/control

@@ -0,0 +1,12 @@
+Source: proxlb
+Maintainer: Florian Paul Azim Hoberg <gyptazy@gyptazy.com>
+Section: admin
+Priority: optional
+Standards-Version: 4.5.0
+Build-Depends: debhelper-compat (= 13), dh-python, python3-all, python3-setuptools
+
+Package: proxlb
+Architecture: all
+Depends: ${python3:Depends}, ${misc:Depends}, python3-requests, python3-urllib3, python3-proxmoxer, python3-yaml 
+Description: A DRS alike Load Balancer for Proxmox Clusters
+ An advanced DRS alike loadbalancer for Proxmox clusters that also supports maintenance modes and affinity/anti-affinity rules.

debian/install

@@ -0,0 +1,2 @@
+proxlb /usr/lib/python3/dist-packages/
+service/proxlb.service /lib/systemd/system/

debian/postinst

@@ -0,0 +1,16 @@
+#!/bin/bash
+set -e
+
+#DEBHELPER#
+if [ "$1" = "configure" ]; then
+    systemctl enable proxlb.service
+    systemctl restart proxlb.service || true
+
+    # Create the 'plb' user if it does not exist
+    if ! id "plb" &>/dev/null; then
+        useradd --system --home /var/lib/proxlb --create-home --shell /usr/sbin/nologin --group nogroup plb
+        echo "User 'plb' created."
+    else
+        echo "User 'plb' already exists, skipping creation."
+    fi
+fi

debian/prerm

@@ -0,0 +1,16 @@
+#!/bin/bash
+set -e
+
+#DEBHELPER#
+if [ "$1" = "remove" ]; then
+    systemctl stop proxlb.service || true
+    systemctl disable proxlb.service || true
+
+    # Remove the 'plb' user if it exists
+    if id "plb" &>/dev/null; then
+        userdel --remove plb
+        echo "User 'plb' removed."
+    else
+        echo "User 'plb' does not exist, skipping removal."
+    fi
+fi

debian/rules

@@ -0,0 +1,4 @@
+#!/usr/bin/make -f
+%:
+	dh $@ --with python3 --buildsystem=pybuild
+

debian/source/format

@@ -0,0 +1 @@
+3.0 (native)

docs/01_requirements.md

@@ -0,0 +1,65 @@
+# Table of Contents
+
+- [Requirements](#requirements)
+- [Where To Run?](#where-to-run)
+
+## Requirements
+ProxLB is a sophisticated load balancer designed to enhance the management and distribution of workloads within a Proxmox cluster. By fully utilizing the Proxmox API, ProxLB eliminates the need for additional SSH access, streamlining cluster management while maintaining robust security. This chapter outlines the general requirements necessary to deploy and operate ProxLB effectively.
+
+### Proxmox Cluster Requirements
+To use ProxLB, you must have an existing Proxmox cluster consisting of at least two nodes. While traditional load balancers often struggle to manage minimal node configurations, ProxLB is optimized to provide efficient load distribution even in a two-node environment. The more nodes present in the cluster, the better ProxLB can optimize resource usage and manage workloads.
+
+### ProxLB Package Requirements
+Next to the previously mentioned requirements, ProxLB also requires you to fit the following ones:
+* Python3.x
+* proxmoxer
+* requests
+* urllib3
+* pyyaml
+
+### Seamless API Integration
+ProxLB relies exclusively on the Proxmox API for all management tasks. This eliminates the need for direct SSH access, ensuring a cleaner and more secure interaction with the cluster. The API integration allows ProxLB to:
+
+- Monitor cluster health and node resource utilization
+- Migrate virtual machines (VMs) and containers as needed
+- Manage storage utilization and distribution
+- Implement load balancing policies
+
+### Authentication and Security Standards
+ProxLB fully supports Proxmox’s integrated user management system, providing robust authentication and access control. Key features include:
+
+- **Multi-Factor Authentication (MFA):** Enhances security by requiring multiple verification methods.
+- **API Key Support:** ProxLB can utilize API keys for authentication instead of traditional username/password combinations, minimizing exposure to credentials.
+- **Role-Based Access Control (RBAC):** Ensures administrators have fine-grained control over user permissions.
+
+### Flexible Storage Support
+ProxLB offers versatile storage management options, supporting both local and shared storage types. It efficiently balances storage workloads across the cluster using the following storage systems:
+
+- **Local Storage:** Direct-attached storage on each node.
+- **Shared Storage:** Includes options like iSCSI, NVMeOF, and NFS for centralized storage solutions.
+- **Ceph:** Integrated support for Ceph distributed storage, providing high availability and fault tolerance.
+
+### Network Infrastructure Requirements
+For optimal performance, ProxLB requires a reliable and high-speed network connection between the nodes in the cluster. Ensure that the network infrastructure meets the following criteria:
+
+- **Low Latency:** Essential for real-time load balancing and VM migration.
+- **Sufficient Bandwidth:** Adequate to handle storage access, data replication, and migration traffic.
+- **Redundant Network Paths:** Recommended for increased fault tolerance and uptime.
+
+### System Resource Allocation
+ProxLB itself requires minimal system resources to operate. However, for managing larger clusters or high workloads, ensure the node running ProxLB has adequate resources available:
+
+- **CPU:** A modern multi-core processor.
+- **Memory:** At least 2 GB of RAM.
+- **Storage:** Minimal disk space for configuration files and logs.
+
+
+## Where To Run?
+ProxLB can run on pretty anthing and only requires you to have a network connectivity to any of the Proxmox host's API (usually on tcp/8006).
+
+Therefore, you can simply run ProxLB on:
+* Bare-metal Systems
+* VMs (even inside the Proxmox cluster)
+* Docker/Podman Container
+* LXC Container
+* On a Proxmox node

docs/02_installation.md

@@ -0,0 +1,164 @@
+# Table of Contents
+
+- [Installation](#installation)
+  - [Requirements / Dependencies](#requirements--dependencies)
+  - [Debian Package](#debian-package)
+    - [Quick-Start](#quick-start)
+    - [Details](#details)
+    - [Debian Packages (.deb files)](#debian-packages-deb-files)
+  - [RedHat Package](#redhat-package)
+  - [Container Images / Docker](#container-images--docker)
+    - [Overview of Images](#overview-of-images)
+  - [Source](#source)
+    - [Traditional System](#traditional-system)
+    - [Container Image](#container-image)
+- [Upgrading](#upgrading)
+  - [Upgrading from < 1.1.0](#upgrading-from--110)
+  - [Upgrading from >= 1.1.0](#upgrading-from--110)
+
+
+## Installation
+### Requirements / Dependencies
+* Python3.x
+* proxmoxer
+* requests
+* urllib3
+* pyyaml
+
+The dependencies can simply be installed with `pip` by running the following command:
+```
+pip install -r requirements.txt
+```
+
+*Note: Distribution packages, such like the provided `.deb` package will automatically resolve and install all required dependencies by using already packaged version from the distribution's repository. By using the Docker (container) image or Debian packages, you do not need to take any care of the requirements listed here.*
+
+### Debian Package
+ProxLB is a powerful and flexible load balancer designed to work across various architectures, including `amd64`, `arm64`, `rv64` and many other ones that support Python. It runs independently of the underlying hardware, making it a versatile choice for different environments. This chapter covers the step-by-step process to install ProxLB on Debian-based systems, including Debian clones like Ubuntu.
+
+#### Quick-Start
+You can simply use this snippet to install the repository and to install ProxLB on your system.
+
+```bash
+echo "deb https://repo.gyptazy.com/stable /" > /etc/apt/sources.list.d/proxlb.list
+wget -O /etc/apt/trusted.gpg.d/proxlb.asc https://repo.gyptazy.com/repository.gpg
+apt-get update && apt-get -y install proxlb
+cp /etc/proxlb/proxlb_example.yaml /etc/proxlb/proxlb.yaml
+# Adjust the config to your needs
+vi /etc/proxlb/proxlb.yaml
+systemctl start proxlb
+```
+
+Afterwards, ProxLB is running in the background and balances your cluster by your defined balancing method (default: memory).
+
+#### Details
+ProxLB provides two different repositories:
+* https://repo.gyptazy.com/stable (only stable release)
+* https://repo.gyptazy.com/testing (bleeding edge - not recommended)
+
+The repository is signed and the GPG key can be found at:
+* https://repo.gyptazy.com/repository.gpg
+
+You can also simply import it by running:
+
+```
+# KeyID:  17169F23F9F71A14AD49EDADDB51D3EB01824F4C
+# UID:    gyptazy Solutions Repository <contact@gyptazy.com>
+# SHA256: 52c267e6f4ec799d40cdbdb29fa518533ac7942dab557fa4c217a76f90d6b0f3  repository.gpg
+
+wget -O /etc/apt/trusted.gpg.d/proxlb.asc https://repo.gyptazy.com/repository.gpg
+```
+
+*Note: The defined repositories `repo.gyptazy.com` and `repo.proxlb.de` are the same!*
+
+#### Debian Packages (.deb files)
+If you do not want to use the repository you can also find the debian packages as a .deb file on gyptazy's CDN at:
+* https://cdn.gyptazy.com/files/os/debian/proxlb/
+
+Afterwards, you can simply install the package by running:
+```bash
+dpkg -i proxlb_*.deb
+cp /etc/proxlb/proxlb_example.yaml /etc/proxlb/proxlb.yaml
+# Adjust the config to your needs
+vi /etc/proxlb/proxlb.yaml
+systemctl start proxlb
+```
+
+### RedHat Package
+There's currently no official support for RedHat based systems. However, there's a dummy .rpm package for such systems in the pipeline which can be found here:
+* https://github.com/gyptazy/ProxLB/actions/workflows/20-pipeline-build-rpm-package.yml
+
+
+### Container Images / Docker
+Using the ProxLB container images is straight forward and only requires you to mount the config file.
+
+```bash
+# Pull the image
+docker pull cr.gyptazy.com/proxlb/proxlb:latest
+# Download the config
+wget -O proxlb.yaml https://raw.githubusercontent.com/gyptazy/ProxLB/refs/heads/main/config/proxlb_example.yaml
+# Adjust the config to your needs
+vi proxlb.yaml
+# Start the ProxLB container image with the ProxLB config
+docker run -it --rm -v $(pwd)/proxlb.yaml:/etc/proxlb/proxlb.yaml proxlb
+```
+
+*Note: ProxLB container images are officially only available at cr.proxlb.de and cr.gyptazy.com.* 
+
+#### Overview of Images
+| Version | Image |
+|------|:------:|
+| latest | cr.gyptazy.com/proxlb/proxlb:latest |
+| v1.1.0 | cr.gyptazy.com/proxlb/proxlb:v1.1.0 |
+| v1.0.6 | cr.gyptazy.com/proxlb/proxlb:v1.0.6 |
+| v1.0.5 | cr.gyptazy.com/proxlb/proxlb:v1.0.5 |
+| v1.0.4 | cr.gyptazy.com/proxlb/proxlb:v1.0.4 |
+| v1.0.3 | cr.gyptazy.com/proxlb/proxlb:v1.0.3 |
+| v1.0.2 | cr.gyptazy.com/proxlb/proxlb:v1.0.2 |
+| v1.0.0 | cr.gyptazy.com/proxlb/proxlb:v1.0.0 |
+| v0.9.9 | cr.gyptazy.com/proxlb/proxlb:v0.9.9 |
+
+### Source
+ProxLB can also easily be used from the provided sources - for traditional systems but also as a Docker/Podman container image.
+
+#### Traditional System
+Setting up and running ProxLB from the sources is simple and requires just a few commands. Ensure Python 3 and the Python dependencies are installed on your system, then run ProxLB using the following command:
+```bash
+git clone https://github.com/gyptazy/ProxLB.git
+cd ProxLB
+```
+
+Afterwards simply adjust the config file to your needs:
+```bash
+vi config/proxlb.yaml
+```
+
+Start ProxLB by Python3 on the system:
+```bash
+python3 proxlb/main.py -c config/proxlb.yaml
+```
+
+#### Container Image
+Creating a container image of ProxLB is straightforward using the provided Dockerfile. The Dockerfile simplifies the process by automating the setup and configuration required to get ProxLB running in an Alpine container. Simply follow the steps in the Dockerfile to build the image, ensuring all dependencies and configurations are correctly applied. For those looking for an even quicker setup, a ready-to-use ProxLB container image is also available, eliminating the need for manual building and allowing for immediate deployment.
+
+```bash
+git clone https://github.com/gyptazy/ProxLB.git
+cd ProxLB
+docker build -t proxlb .
+```
+
+Afterwards simply adjust the config file to your needs:
+```bash
+vi config/proxlb.yaml
+```
+
+Finally, start the created container.
+```bash
+docker run -it --rm -v $(pwd)/proxlb.yaml:/etc/proxlb/proxlb.yaml proxlb
+```
+
+## Upgrading
+### Upgrading from < 1.1.0
+Upgrading ProxLB is not supported due to a fundamental redesign introduced in version 1.1.x. With this update, ProxLB transitioned from a monolithic application to a pure Python-style project, embracing a more modular and flexible architecture. This shift aimed to improve maintainability and extensibility while keeping up with modern development practices. Additionally, ProxLB moved away from traditional ini-style configuration files and adopted YAML for configuration management. This change simplifies configuration handling, reduces the need for extensive validation, and ensures better type casting, ultimately providing a more streamlined and user-friendly experience.
+
+### Upgrading from >= 1.1.0
+Uprading within the current stable versions, starting from 1.1.0, will be possible in all supported ways.

docs/03_configuration.md

@@ -0,0 +1,216 @@
+# Table of Contents
+
+1. [Authentication / User Accounts / Permissions](#authentication--user-accounts--permissions)
+    1. [Authentication](#authentication)
+    2. [Creating a Dedicated User](#creating-a-dedicated-user)
+    3. [Creating an API Token for a User](#creating-an-api-token-for-a-user)
+    4. [Required Permissions for a User](#required-permissions-for-a-user)
+2. [Configuration](#configuration)
+    1. [Affinity & Anti-Affinity Rules](#affinity--anti-affinity-rules)
+        1. [Affinity Rules](#affinity-rules)
+        2. [Anti-Affinity Rules](#anti-affinity-rules)
+        3. [Affinity / Anti-Affinity Enforcing](#affinity--anti-affinity-enforcing)
+        4. [Ignore VMs](#ignore-vms)
+        5. [Pin VMs to Hypervisor Nodes](#pin-vms-to-hypervisor-nodes)
+    2. [API Loadbalancing](#api-loadbalancing)
+    3. [Ignore Host-Nodes or Guests](#ignore-host-nodes-or-guests)
+    4. [IPv6 Support](#ipv6-support)
+    5. [Logging / Log-Level](#logging--log-level)
+    6. [Parallel Migrations](#parallel-migrations)
+    7. [Run as a Systemd-Service](#run-as-a-systemd-service)
+    8. [SSL Self-Signed Certificates](#ssl-self-signed-certificates)
+
+## Authentication / User Accounts / Permissions
+### Authentication
+ProxLB supports the traditional username and password authentication method, which is familiar to many users. This method requires users to provide their credentials (username and password) to gain access to the Proxmox system. While this method is straightforward and easy to implement, it has several security limitations. Username and password combinations can be vulnerable to brute force attacks, where an attacker systematically attempts various combinations until the correct one is found. If a user's credentials are compromised through phishing, malware, or other means, the attacker can gain unauthorized access to the system. Additionally, traditional authentication does not provide granular control over permissions and access levels, potentially exposing sensitive operations to unauthorized users.
+
+To enhance security, ProxLB supports API token authentication. API tokens are unique identifiers that are used to authenticate API requests. They offer several advantages over traditional username and password authentication. API tokens are more secure as they are typically long, random strings that are difficult to guess. They can be revoked and regenerated as needed, reducing the risk of unauthorized access. API tokens can be associated with specific user accounts that have only the required permissions, ensuring that users only have access to the resources and operations they need. Furthermore, API tokens can be used for automated scripts and applications, facilitating seamless integration with other systems and services.
+
+When Multi-Factor Authentication (MFA) or Two-Factor Authentication (2FA) is enabled in the Proxmox cluster, the system enforces the use of API tokens for authentication. This is because traditional username and password authentication is not considered secure enough in conjunction with MFA/2FA. To ensure the highest level of security when using API tokens, follow these best practices: Use dedicated user accounts for API tokens, each with only the necessary permissions. This limits the potential impact of a compromised token. Ensure that API tokens are long, random, and unique. Avoid using easily guessable patterns or sequences. Periodically regenerate and replace API tokens to minimize the risk of long-term exposure. Store API tokens securely, using environment variables or secure vaults. Avoid hardcoding tokens in source code or configuration files. Regularly monitor and audit the usage of API tokens to detect any suspicious activity or unauthorized access.
+
+### Creating a Dedicated User
+It is advisable to avoid using the default root@pam user for balancing tasks in ProxLB. Instead, creating a dedicated user account is recommended and can be done easily. You can create a new user through the GUI, API, or CLI. While the detailed roles required for balancing are outlined in the next chapter, you can also use the following CLI commands to create a user with the necessary roles to manage Virtual Machines (VMs) and Containers (CTs):
+
+```
+pveum role add proxlb --privs Datastore.Audit,Sys.Audit,VM.Audit,VM.Migrate
+pveum user add proxlb@pve --password <password>
+pveum acl modify / --roles proxlb --users proxlb@pve
+```
+
+*Note: The user management can also be done on the WebUI without invoking the CLI.*
+
+### Creating an API Token for a User
+Create an API token for user proxlb@pve with token ID proxlb and deactivated privilege separation:
+```
+pveum user token add proxlb@pve proxlb --privsep 0
+```
+
+Afterwards, you get the token secret returned. You can now add those entries to your ProxLB config. Make sure, that you also keep the `user` parameter, next to the new token parameters.
+
+> [!IMPORTANT]  
+> The parameter `pass` then needs to be **absent**! You should also take care about the privilege and authentication mechanism behind Proxmox. You might want or even might not want to use privilege separation and this is up to your personal needs and use case.
+
+| Proxmox API  | ProxLB Config  | Example  |
+|---|---|---|
+| User  | [user](https://github.com/gyptazy/ProxLB/blob/main/config/proxlb_example.yaml#L3) | proxlb@pve  |
+| Token ID  | [token_id](https://github.com/gyptazy/ProxLB/blob/main/config/proxlb_example.yaml#L6) | proxlb  |
+| Token Secret  | [token_secret](https://github.com/gyptazy/ProxLB/blob/main/config/proxlb_example.yaml#L7)  |  430e308f-1337-1337-beef-1337beefcafe |
+
+*Note: The API token configuration can also be done on the WebUI without invoking the CLI.*
+
+### Required Permissions for a User
+To ensure that ProxLB operates effectively and securely, it is essential to assign the appropriate permissions to the user accounts responsible for managing the load balancing tasks. The following permissions are the minimum required for a user to perform essential ProxLB operations:
+
+* `Datastore.Audit`: Grants the ability to audit and view datastore information.
+* `Sys.Audit`: Allows the user to audit and view system information.
+* `VM.Audit`: Enables the user to audit and view virtual machine details.
+* `VM.Migrate`: Provides the permission to migrate virtual machines.
+
+Assigning these permissions ensures that the user can access necessary information and perform critical operations related to load balancing without granting excessive privileges. This practice helps maintain a secure and efficient ProxLB environment.
+
+## Configuration
+
+### Affinity & Anti-Affinity Rules
+ProxLB provides an advanced mechanism to define affinity and anti-affinity rules, enabling precise control over virtual machine (VM) placement. These rules help manage resource distribution, improve high availability configurations, and optimize performance within a Proxmox Virtual Environment (PVE) cluster. By leveraging Proxmox’s integrated access management, ProxLB ensures that users can only define and manage rules for guests they have permission to access.
+
+ProxLB implements affinity and anti-affinity rules through a tag-based system within the Proxmox web interface. Each guest (virtual machine or container) can be assigned specific tags, which then dictate its placement behavior. This method maintains a streamlined and secure approach to managing VM relationships while preserving Proxmox’s inherent permission model.
+
+#### Affinity Rules
+<img align="left" src="https://cdn.gyptazy.com/images/proxlb-affinity-rules.jpg"/> Affinity rules are used to group certain VMs together, ensuring that they run on the same host whenever possible. This can be beneficial for workloads requiring low-latency communication, such as clustered databases or application servers that frequently exchange data.
+
+To define an affinity rule which keeps all guests assigned to this tag together on a node, users assign a tag with the prefix `plb_affinity_$TAG`:
+
+##### Example for Screenshot
+```
+plb_affinity_talos
+```
+
+As a result, ProxLB will attempt to place all VMs with the `plb_affinity_web` tag on the same host (see also the attached screenshot with the same node).
+
+#### Anti-Affinity Rules
+<img align="left" src="https://cdn.gyptazy.com/images/proxlb-anti-affinity-rules.jpg"/> Conversely, anti-affinity rules ensure that designated VMs do not run on the same physical host. This is particularly useful for high-availability setups, where redundancy is crucial. Ensuring that critical services are distributed across multiple hosts reduces the risk of a single point of failure.
+
+To define an anti-affinity rule that ensures to not move systems within this group to the same node, users assign a tag with the prefix:
+
+##### Example for Screenshot
+```
+plb_anti_affinity_ntp
+```
+
+As a result, ProxLB will try to place the VMs with the `plb_anti_affinity_ntp` tag on different hosts (see also the attached screenshot with the different nodes).
+
+**Note:** While this ensures that ProxLB tries distribute these VMs across different physical hosts within the Proxmox cluster this may not always work. If you have more guests attached to the group than nodes in the cluster, we still need to run them anywhere. If this case occurs, the next one with the most free resources will be selected.
+
+### Affinity / Anti-Affinity Enforcing
+When a cluster is already balanced and does not require further adjustments, enabling the enforce_affinity parameter ensures that affinity and anti-affinity rules are still respected. This parameter prioritizes the placement of guest objects according to these rules, even if it leads to slight resource imbalances or increased migration overhead. Regularly reviewing and updating these rules, along with monitoring cluster performance, helps maintain optimal performance and reliability. By carefully managing these aspects, you can create a cluster environment that meets your specific needs and maintains a good balance of resources.
+
+```
+balancing:
+  enforce_affinity: True
+```
+
+*Note: This may have impacts to the cluster. Depending on the created group matrix, the result may also be an unbalanced cluster.*
+
+### Ignore VMs / CTs
+<img align="left" src="https://cdn.gyptazy.com/images/proxlb-ignore-vm-movement.jpg"/> Guests, such as VMs or CTs, can also be completely ignored. This means, they won't be affected by any migration (even when (anti-)affinity rules are enforced). To ensure a proper resource evaluation, these guests are still collected and evaluated but simply skipped for balancing actions. Another thing is the implementation. While ProxLB might have a very restricted configuration file including the file permissions, this file is only read- and writeable by the Proxmox administrators. However, we might have user and groups who want to define on their own that their systems shouldn't be moved. Therefore, these users can simpy set a specific tag to the guest object - just like the (anti)affinity rules.
+
+To define a guest to be ignored from the balancing, users assign a tag with the prefix `plb_ignore_$TAG`:
+
+#### Example for Screenshot
+```
+plb_ignore_dev
+```
+
+As a result, ProxLB will not migrate this guest with the `plb_ignore_dev` tag to any other node.
+
+**Note:** Ignored guests are really ignored. Even by enforcing affinity rules this guest will be ignored.
+
+### Pin VMs to Specific Hypervisor Nodes
+<img align="left" src="https://cdn.gyptazy.com/images/proxlb-tag-node-pinning.jpg"/> Guests, such as VMs or CTs, can also be pinned to specific nodes in the cluster. This might be usefull when running applications with some special licensing requirements that are only fulfilled on certain nodes. It might also be interesting, when some physical hardware is attached to a node, that is not available in general within the cluster.
+
+To pin a guest to a specific cluster node, users assign a tag with the prefix `plb_pin_$nodename` to the desired guest:
+
+#### Example for Screenshot
+```
+plb_pin_node03
+```
+
+As a result, ProxLB will pin the guest `dev-vm01` to the node `virt03`.
+
+You can also repeat this step multiple times for different node names to create a potential group of allowed hosts where a the guest may be served on. In this case, ProxLB takes the node with the lowest used resources according to the defined balancing values from this group.
+
+**Note:** The given node names from the tag are validated. This means, ProxLB validated if the given node name is really part of the cluster. In case of a wrongly defined or unavailable node name it continous to use the regular processes to make sure the guest keeps running.
+
+### API Loadbalancing
+ProxLB supports API loadbalancing, where one or more host objects can be defined as a list. This ensures, that you can even operator ProxLB without further changes when one or more nodes are offline or in a maintenance. When defining multiple hosts, the first reachable one will be picked. You can speficy custom ports in the list. There are 4 ways of defining hosts with ports:
+1. Hostname of IPv4 without port (in this case the default 8006 will be used)
+2. Hostname or IPv4 with port
+3. IPv6 in brackets with optional port
+4. IPv6 without brackets, in this case the port is assumed after last colon
+
+```
+proxmox_api:
+  hosts: ['virt01.example.com', '10.10.10.10', 'fe01::bad:code::cafe', 'virt01.example.com:443', '[fc00::1]', '[fc00::1]:443', 'fc00::1:8006']
+```
+
+### Ignore Host-Nodes or Guests
+In managing a Proxmox environment, it's often necessary to exclude certain host nodes and guests from various operations. For host nodes, this exclusion can be achieved by specifying them in the ignore_nodes parameter within the proxmox_api chapter, effectively preventing any automated processes from interacting with these nodes. Guests, on the other hand, can be ignored by assigning them a specific tag that starts with or is equal to plb_ignore, ensuring they are omitted from any automated tasks or monitoring. By implementing these configurations, administrators can fine-tune their Proxmox management to focus only on relevant nodes and guests, optimizing operational efficiency and resource allocation.
+
+```
+proxmox_cluster:
+  ignore_nodes: ['node01', 'node02']
+```
+
+### IPv6 Support
+Yes, ProxLB fully supports IPv6.
+
+### Logging / Log-Level
+ProxLB supports systemd for seamless service management on Linux distributions. To enable this, create a proxLB.service file in /etc/systemd/system/ from `service/proxlb.service` within this repository.
+
+On systems without systemd, such as FreeBSD and macOS, ProxLB runs with similar configurations but logs to stdout and stderr. The logging level and verbosity can be set in the `service` section of the configuration file:
+
+```
+service:
+  log_level: DEBUG
+```
+
+ProxLB only support the following log levels:
+* INFO
+* WARNING
+* CRITICAL
+* DEBUG
+
+### Parallel Migrations
+By default, parallel migrations are deactivated. This means, that a guest object gets migrated and the migration job is being watched until the VM or CT got moved to a new node. However, this may take a lot of time and many environments are fast enough to handle the IO load for multiple guest objects. However, there are always corner cases and this depends on your setup. Parallel migrations can be enabled by setting `parallel` to `True` within the `balancing` chapter:
+
+```
+balancing:
+  parallel: False
+```
+
+### Run as a Systemd-Service
+The proxlb systemd unit orchestrates the ProxLB application. ProxLB can be used either as a one-shot solution or run periodically, depending on the configuration specified in the daemon chapter of its configuration file.
+
+```
+service:
+  daemon: False
+    schedule:
+      interval: 12
+      format: hours
+```
+
+In this configuration:
+* `daemon`: False indicates that the ProxLB application is not running as a daemon and will execute as a one-shot solution.
+* `schedule`: 12 defines the interval for the schedule, specifying how often rebalancing should be done if running as a daemon.
+* `format`: Defines the given format of schedule where you can choose between `hours` or `minutes`.
+
+### SSL Self-Signed Certificates
+If you are using SSL self-signed certificates or non-valid certificated in general and do not want to deal with additional trust levels, you may also disable the SSL validation. This may mostly be helpful for dev- & test labs.
+
+SSL certificate validation can be disabled in the `proxmox_api` section in the config file by setting:
+```
+proxmox_api:
+    ssl_verification: False
+```
+
+*Note: Disabling SSL certificate validation is not recommended.*

docs/99-faq.md

@@ -0,0 +1,24 @@
+## Table of Contents
+
+1. [GUI Integration](#gui-integration)
+    - [How to install pve-proxmoxlb-service-ui package](https://github.com/gyptazy/ProxLB/issues/44)
+2. [Proxmox HA Integration](#proxmox-ha-integration)
+    - [Host groups: Honour HA groups](https://github.com/gyptazy/ProxLB/issues/65)
+
+### GUI Integration
+<img align="left" src="https://cdn.gyptazy.com/images/proxlb-GUI-integration.jpg"/> ProxLB can also be accessed through the Proxmox Web UI by installing the optional `pve-proxmoxlb-service-ui` package, which depends on the proxlb package. For full Web UI integration, this package must be installed on all nodes within the cluster. Once installed, a new menu item - `Rebalancing`, appears in the cluster level under the HA section. Once installed, it offers two key functionalities:
+* Rebalancing VM workloads
+* Migrate VM workloads away from a defined node (e.g. maintenance preparation)
+
+**Note:** This package is currently discontinued and will be readded at a later time. See also: [#44: How to install pve-proxmoxlb-service-ui package](https://github.com/gyptazy/ProxLB/issues/44).
+
+### Proxmox HA Integration
+Proxmox HA (High Availability) groups are designed to ensure that virtual machines (VMs) remain running within a Proxmox cluster. HA groups define specific rules for where VMs should be started or migrated in case of node failures, ensuring minimal downtime and automatic recovery.
+
+However, when used in conjunction with ProxLB, the built-in load balancer for Proxmox, conflicts can arise. ProxLB operates with its own logic for workload distribution, taking into account affinity and anti-affinity rules. While it effectively balances guest workloads, it may re-shift and redistribute VMs in a way that does not align with HA group constraints, potentially leading to unsuitable placements.
+
+Due to these conflicts, it is currently not recommended to use both HA groups and ProxLB simultaneously. The interaction between the two mechanisms can lead to unexpected behavior, where VMs might not adhere to HA group rules after being moved by ProxLB.
+
+A solution to improve compatibility between HA groups and ProxLB is under evaluation, aiming to ensure that both features can work together without disrupting VM placement strategies.
+
+See also: [#65: Host groups: Honour HA groups](https://github.com/gyptazy/ProxLB/issues/65).

misc/01-replace-version.sh

@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
-VERSION="1.1.0-alpha"
+VERSION="1.1.4"
 
 sed -i "s/^__version__ = .*/__version__ = \"$VERSION\"/" "proxlb/utils/version.py"
 sed -i "s/version=\"[0-9]*\.[0-9]*\.[0-9]*\"/version=\"$VERSION\"/" setup.py
-echo "OK: Versions have been sucessfully set to $VERSION"
+echo "OK: Versions have been sucessfully set to $VERSION"

proxlb/main.py

@@ -13,6 +13,7 @@ __license__ = "GPL-3.0"
 
 
 import logging
+import signal
 from utils.logger import SystemdLogger
 from utils.cli_parser import CliParser
 from utils.config_parser import ConfigParser
@@ -32,6 +33,9 @@ def main():
     # Initialize logging handler
     logger = SystemdLogger(level=logging.INFO)
 
+    # Signal handler for SIGHUP
+    signal.signal(signal.SIGHUP, Helper.handler_sighup)
+
     # Parses arguments passed from the CLI
     cli_parser = CliParser()
     cli_args = cli_parser.parse_args()
@@ -44,6 +48,9 @@ def main():
     # Update log level from config and fallback to INFO if not defined
     logger.set_log_level(proxlb_config.get('service', {}).get('log_level', 'INFO'))
 
+    # Validate of an optional service delay
+    Helper.get_service_delay(proxlb_config)
+
     # Connect to Proxmox API & create API object
     proxmox_api = ProxmoxApi(proxlb_config)
 
@@ -51,10 +58,19 @@ def main():
     proxlb_config["proxmox_api"]["pass"] = "********"
 
     while True:
+
+        # Validate if reload signal was sent during runtime
+        # and reload the ProxLB configuration and adjust log level
+        if Helper.proxlb_reload:
+            logger.info("Reloading ProxLB configuration.")
+            proxlb_config = config_parser.get_config()
+            logger.set_log_level(proxlb_config.get('service', {}).get('log_level', 'INFO'))
+            Helper.proxlb_reload = False
+
         # Get all required objects from the Proxmox cluster
         meta = {"meta": proxlb_config}
         nodes = Nodes.get_nodes(proxmox_api, proxlb_config)
-        guests = Guests.get_guests(proxmox_api, nodes)
+        guests = Guests.get_guests(proxmox_api, nodes, meta)
         groups = Groups.get_groups(guests, nodes)
 
         # Merge obtained objects from the Proxmox cluster for further usage
@@ -71,9 +87,13 @@ def main():
         Helper.log_node_metrics(proxlb_data, init=False)
 
         # Perform balancing actions via Proxmox API
-        if not cli_args.dry_run:
-            Balancing(proxmox_api, proxlb_data)
+        if proxlb_data["meta"]["balancing"].get("enable", False):
+            if not cli_args.dry_run:
+                Balancing(proxmox_api, proxlb_data)
 
+        # Validate if the JSON output should be
+        # printed to stdout
+        Helper.print_json(proxlb_data, cli_args.json)
         # Validate daemon mode
         Helper.get_daemon_mode(proxlb_config)
 

proxlb/models/balancing.py

@@ -12,6 +12,7 @@ __license__ = "GPL-3.0"
 
 import proxmoxer
 import time
+from itertools import islice
 from utils.logger import SystemdLogger
 from typing import Dict, Any
 
@@ -48,26 +49,68 @@ class Balancing:
         Initializes the Balancing class with the provided ProxLB data.
 
         Args:
-            proxlb_data (dict): The data required for balancing VMs and CTs.
+            proxmox_api (object): The Proxmox API client instance used to interact with the Proxmox cluster.
+            proxlb_data (dict): A dictionary containing data related to the ProxLB load balancing configuration.
         """
-        for guest_name, guest_meta in proxlb_data["guests"].items():
+        def chunk_dict(data, size):
+            """
+            Splits a dictionary into chunks of a specified size.
+            Args:
+                data (dict): The dictionary to be split into chunks.
+                size (int): The size of each chunk.
+            Yields:
+                dict: A chunk of the original dictionary with the specified size.
+            """
+            logger.debug("Starting: chunk_dict.")
+            it = iter(data.items())
+            for chunk in range(0, len(data), size):
+                yield dict(islice(it, size))
 
-            if guest_meta["node_current"] != guest_meta["node_target"]:
-                guest_id = guest_meta["id"]
-                guest_node_current = guest_meta["node_current"]
-                guest_node_target = guest_meta["node_target"]
+        # Validate if balancing should be performed in parallel or sequentially.
+        # If parallel balancing is enabled, set the number of parallel jobs.
+        parallel_jobs = proxlb_data["meta"]["balancing"].get("parallel_jobs", 5)
+        if not proxlb_data["meta"]["balancing"].get("parallel", False):
+            parallel_jobs = 1
+            logger.debug("Balancing: Parallel balancing is disabled. Running sequentially.")
+        else:
+            logger.debug(f"Balancing: Parallel balancing is enabled. Running with {parallel_jobs} parallel jobs.")
 
-                # VM Balancing
-                if guest_meta["type"] == "vm":
-                    self.exec_rebalancing_vm(proxmox_api, proxlb_data, guest_name)
+        for chunk in chunk_dict(proxlb_data["guests"], parallel_jobs):
+            jobs_to_wait = []
 
-                # CT Balancing
-                elif guest_meta["type"] == "ct":
-                    self.exec_rebalancing_ct(proxmox_api, proxlb_data, guest_name)
+            for guest_name, guest_meta in chunk.items():
 
-                # Hopefully never reaching, but should be catched
+                # Check if the guest's target is not the same as the current node
+                if guest_meta["node_current"] != guest_meta["node_target"]:
+
+                    # Check if the guest is not ignored and perform the balancing
+                    # operation based on the guest type
+                    if not guest_meta["ignore"]:
+                        job_id = None
+
+                        # VM Balancing
+                        if guest_meta["type"] == "vm":
+                            job_id = self.exec_rebalancing_vm(proxmox_api, proxlb_data, guest_name)
+
+                        # CT Balancing
+                        elif guest_meta["type"] == "ct":
+                            job_id = self.exec_rebalancing_ct(proxmox_api, proxlb_data, guest_name)
+
+                        # Just in case we get a new type of guest in the future
+                        else:
+                            logger.critical(f"Balancing: Got unexpected guest type: {guest_meta['type']}. Cannot proceed guest: {guest_meta['name']}.")
+
+                        if job_id:
+                            jobs_to_wait.append((guest_name, guest_meta["node_current"], job_id))
+
+                    else:
+                        logger.debug(f"Balancing: Guest {guest_name} is ignored and will not be rebalanced.")
                 else:
-                    logger.critical(f"Balancing: Got unexpected guest type: {guest_meta['type']}. Cannot proceed guest: {guest_meta['name']}.")
+                    logger.debug(f"Balancing: Guest {guest_name} is already on the target node {guest_meta['node_target']} and will not be rebalanced.")
+
+            # Wait for all jobs in the current chunk to complete
+            for guest_name, node, job_id in jobs_to_wait:
+                self.get_rebalancing_job_status(proxmox_api, proxlb_data, guest_name, node, job_id)
 
     def exec_rebalancing_vm(self, proxmox_api: any, proxlb_data: Dict[str, Any], guest_name: str) -> None:
         """
@@ -106,13 +149,13 @@ class Balancing:
         }
 
         try:
-            logger.debug(f"Balancing: Starting to migrate guest {guest_name} of type VM.")
+            logger.info(f"Balancing: Starting to migrate VM guest {guest_name} from {guest_node_current} to {guest_node_target}.")
             job_id = proxmox_api.nodes(guest_node_current).qemu(guest_id).migrate().post(**migration_options)
-            job = self.get_rebalancing_job_status(proxmox_api, proxlb_data, guest_name, guest_node_current, job_id)
         except proxmoxer.core.ResourceException as proxmox_api_error:
-            logger.critical(f"Balancing: Failed to migrate guest {guest_name} of type CT due to some Proxmox errors. Please check if resource is locked or similar.")
-
+            logger.critical(f"Balancing: Failed to migrate guest {guest_name} of type VM due to some Proxmox errors. Please check if resource is locked or similar.")
+            logger.debug(f"Balancing: Failed to migrate guest {guest_name} of type VM due to some Proxmox errors: {proxmox_api_error}")
         logger.debug("Finished: exec_rebalancing_vm.")
+        return job_id
 
     def exec_rebalancing_ct(self, proxmox_api: any, proxlb_data: Dict[str, Any], guest_name: str) -> None:
         """
@@ -135,13 +178,13 @@ class Balancing:
         guest_node_target = proxlb_data["guests"][guest_name]["node_target"]
 
         try:
-            logger.debug(f"Balancing: Starting to migrate guest {guest_name} of type CT.")
+            logger.info(f"Balancing: Starting to migrate CT guest {guest_name} from {guest_node_current} to {guest_node_target}.")
             job_id = proxmox_api.nodes(guest_node_current).lxc(guest_id).migrate().post(target=guest_node_target, restart=1)
-            job = self.get_rebalancing_job_status(proxmox_api, proxlb_data, guest_name, guest_node_current, job_id)
         except proxmoxer.core.ResourceException as proxmox_api_error:
             logger.critical(f"Balancing: Failed to migrate guest {guest_name} of type CT due to some Proxmox errors. Please check if resource is locked or similar.")
-
+            logger.debug(f"Balancing: Failed to migrate guest {guest_name} of type CT due to some Proxmox errors: {proxmox_api_error}")
         logger.debug("Finished: exec_rebalancing_ct.")
+        return job_id
 
     def get_rebalancing_job_status(self, proxmox_api: any, proxlb_data: Dict[str, Any], guest_name: str, guest_current_node: str, job_id: int, retry_counter: int = 1) -> bool:
         """
@@ -159,35 +202,32 @@ class Balancing:
             bool: True if the job completed successfully, False otherwise.
         """
         logger.debug("Starting: get_rebalancing_job_status.")
-        # Parallel migrations can take a huge time and create a higher load, if not defined by an
-        # operator we will use a sequential mode by default
-        if not proxlb_data["meta"]["balancing"].get("parallel", False):
-            job = proxmox_api.nodes(guest_current_node).tasks(job_id).status().get()
+        job = proxmox_api.nodes(guest_current_node).tasks(job_id).status().get()
 
-            # Watch job id until it finalizes
-            if job["status"] == "running":
-                # Do not hammer the API while
-                # watching the job status
-                time.sleep(10)
-                retry_counter += 1
+        # Watch job id until it finalizes
+        if job["status"] == "running":
+            # Do not hammer the API while
+            # watching the job status
+            time.sleep(10)
+            retry_counter += 1
 
-                # Run recursion until we hit the soft-limit of maximum migration time for a guest
-                if retry_counter < proxlb_data["meta"]["balancing"].get("max_job_validation", 1800):
-                    logger.debug(f"Balancing: Job ID {job_id} (guest: {guest_name}) for migration is still running... (Run: {retry_counter})")
-                    self.get_rebalancing_job_status(proxmox_api, proxlb_data, guest_name, guest_current_node, job_id, retry_counter)
-                else:
-                    logger.warning(f"Balancing: Job ID {job_id} (guest: {guest_name}) for migration took too long. Please check manually.")
-                    logger.debug("Finished: get_rebalancing_job_status.")
-                    return False
+            # Run recursion until we hit the soft-limit of maximum migration time for a guest
+            if retry_counter < proxlb_data["meta"]["balancing"].get("max_job_validation", 1800):
+                logger.debug(f"Balancing: Job ID {job_id} (guest: {guest_name}) for migration is still running... (Run: {retry_counter})")
+                self.get_rebalancing_job_status(proxmox_api, proxlb_data, guest_name, guest_current_node, job_id, retry_counter)
+            else:
+                logger.warning(f"Balancing: Job ID {job_id} (guest: {guest_name}) for migration took too long. Please check manually.")
+                logger.debug("Finished: get_rebalancing_job_status.")
+                return False
 
-            # Validate job output for errors when finished
-            if job["status"] == "stopped":
+        # Validate job output for errors when finished
+        if job["status"] == "stopped":
 
-                if job["exitstatus"] == "OK":
-                    logger.debug(f"Balancing: Job ID {job_id} (guest: {guest_name}) was sucessfully.")
-                    logger.debug("Finished: get_rebalancing_job_status.")
-                    return True
-                else:
-                    logger.critical(f"Balancing: Job ID {job_id} (guest: {guest_name}) went into an error! Please check manually.")
-                    logger.debug("Finished: get_rebalancing_job_status.")
-                    return False
+            if job["exitstatus"] == "OK":
+                logger.debug(f"Balancing: Job ID {job_id} (guest: {guest_name}) was successfully.")
+                logger.debug("Finished: get_rebalancing_job_status.")
+                return True
+            else:
+                logger.critical(f"Balancing: Job ID {job_id} (guest: {guest_name}) went into an error! Please check manually.")
+                logger.debug("Finished: get_rebalancing_job_status.")
+                return False

proxlb/models/calculations.py

@@ -66,7 +66,7 @@ class Calculations:
     @staticmethod
     def set_node_assignments(proxlb_data: Dict[str, Any]) -> Dict[str, Any]:
         """
-        Set the assigned ressources of the nodes based on the current assigned
+        Set the assigned resources of the nodes based on the current assigned
         guest resources by their created groups as an initial base.
 
         Args:
@@ -82,6 +82,7 @@ class Calculations:
                 guest_node_current = proxlb_data["guests"][guest_name]["node_current"]
                 # Update Hardware assignments
                 # Update assigned values for the current node
+                logger.debug(f"set_node_assignment of guest {guest_name} on node {guest_node_current} with cpu_total: {proxlb_data['guests'][guest_name]['cpu_total']}, memory_total: {proxlb_data['guests'][guest_name]['memory_total']}, disk_total: {proxlb_data['guests'][guest_name]['disk_total']}.")
                 proxlb_data["nodes"][guest_node_current]["cpu_assigned"] += proxlb_data["guests"][guest_name]["cpu_total"]
                 proxlb_data["nodes"][guest_node_current]["memory_assigned"] += proxlb_data["guests"][guest_name]["memory_total"]
                 proxlb_data["nodes"][guest_node_current]["disk_assigned"] += proxlb_data["guests"][guest_name]["disk_total"]
@@ -119,10 +120,8 @@ class Calculations:
             if method_value_highest - method_value_lowest > balanciness:
                 proxlb_data["meta"]["balancing"]["balance"] = True
                 logger.debug(f"Guest balancing is required. Highest value: {method_value_highest}, lowest value: {method_value_lowest} balanced by {method} and {mode}.")
-                logger.critical(f"Guest balancing is required. Highest value: {method_value_highest}, lowest value: {method_value_lowest} balanced by {method} and {mode}.")
             else:
                 logger.debug(f"Guest balancing is ok. Highest value: {method_value_highest}, lowest value: {method_value_lowest} balanced by {method} and {mode}.")
-                logger.critical(f"Guest balancing is ok. Highest value: {method_value_highest}, lowest value: {method_value_lowest} balanced by {method} and {mode}.")
 
         else:
             logger.warning("No guests for balancing found.")
@@ -130,7 +129,7 @@ class Calculations:
         logger.debug("Finished: get_balanciness.")
 
     @staticmethod
-    def get_most_free_node(proxlb_data: Dict[str, Any], return_node: bool = False) -> Dict[str, Any]:
+    def get_most_free_node(proxlb_data: Dict[str, Any], return_node: bool = False, guest_node_relation_list: list = []) -> Dict[str, Any]:
         """
         Get the name of the Proxmox node in the cluster with the most free resources based on
         the user defined method (e.g.: memory) and mode (e.g.: used).
@@ -139,6 +138,8 @@ class Calculations:
             proxlb_data (Dict[str, Any]): The data holding all content of all objects.
             return_node (bool): The indicator to simply return the best node for further
                                 assignments.
+            guest_node_relation_list (list): A list of nodes that have a tag on the given
+                                             guest relationship for pinning.
 
         Returns:
             Dict[str, Any]: Updated meta data section of the node with the most free resources that should
@@ -147,9 +148,18 @@ class Calculations:
         logger.debug("Starting: get_most_free_node.")
         proxlb_data["meta"]["balancing"]["balance_next_node"] = ""
 
-        # Do not include nodes that are marked in 'maintenance'
+        # Filter and exclude nodes that are in maintenance mode
         filtered_nodes = [node for node in proxlb_data["nodes"].values() if not node["maintenance"]]
-        lowest_usage_node = min(filtered_nodes, key=lambda x: x["memory_used_percent"])
+
+        # Filter and include nodes that given by a relationship between guest and node. This is only
+        # used if the guest has a relationship to a node defined by "pin" tags.
+        if len(guest_node_relation_list) > 0:
+            filtered_nodes = [node for node in proxlb_data["nodes"].values() if node["name"] in guest_node_relation_list]
+
+        # Filter by the defined methods and modes for balancing
+        method = proxlb_data["meta"]["balancing"].get("method", "memory")
+        mode = proxlb_data["meta"]["balancing"].get("mode", "used")
+        lowest_usage_node = min(filtered_nodes, key=lambda x: x[f"{method}_{mode}_percent"])
         proxlb_data["meta"]["balancing"]["balance_reason"] = 'resources'
         proxlb_data["meta"]["balancing"]["balance_next_node"] = lowest_usage_node["name"]
 
@@ -207,13 +217,13 @@ class Calculations:
         None
         """
         logger.debug("Starting: relocate_guests.")
-        if proxlb_data["meta"]["balancing"]["balance"] or proxlb_data["meta"]["balancing"]["enforce_affinity"]:
+        if proxlb_data["meta"]["balancing"]["balance"] or proxlb_data["meta"]["balancing"].get("enforce_affinity", False):
 
             if proxlb_data["meta"]["balancing"].get("balance", False):
-                logger.debug("Balancing of guests will be performt. Reason: balanciness")
+                logger.debug("Balancing of guests will be performed. Reason: balanciness")
 
             if proxlb_data["meta"]["balancing"].get("enforce_affinity", False):
-                logger.debug("Balancing of guests will be performt. Reason: enforce affinity balancing")
+                logger.debug("Balancing of guests will be performed. Reason: enforce affinity balancing")
 
             for group_name in proxlb_data["groups"]["affinity"]:
 
@@ -225,6 +235,7 @@ class Calculations:
                 for guest_name in proxlb_data["groups"]["affinity"][group_name]["guests"]:
                     proxlb_data["meta"]["balancing"]["balance_next_guest"] = guest_name
                     Calculations.val_anti_affinity(proxlb_data, guest_name)
+                    Calculations.val_node_relationships(proxlb_data, guest_name)
                     Calculations.update_node_resources(proxlb_data)
 
         logger.debug("Finished: relocate_guests.")
@@ -248,10 +259,10 @@ class Calculations:
         None
         """
         logger.debug("Starting: val_anti_affinity.")
-        # Start by interating over all defined anti-affinity groups
+        # Start by iterating over all defined anti-affinity groups
         for group_name in proxlb_data["groups"]["anti_affinity"].keys():
 
-            # Validate if the provided guest ist included in the anti-affinity group
+            # Validate if the provided guest is included in the anti-affinity group
             if guest_name in proxlb_data["groups"]["anti_affinity"][group_name]['guests'] and not proxlb_data["guests"][guest_name]["processed"]:
                 logger.debug(f"Anti-Affinity: Guest: {guest_name} is included in anti-affinity group: {group_name}.")
 
@@ -278,6 +289,39 @@ class Calculations:
 
         logger.debug("Finished: val_anti_affinity.")
 
+    @staticmethod
+    def val_node_relationships(proxlb_data: Dict[str, Any], guest_name: str):
+        """
+        Validates and assigns guests to nodes based on defined relationships based on tags.
+
+        Parameters:
+        proxlb_data (Dict[str, Any]): The data holding all content of all objects.
+        guest_name (str): The name of the guest to be validated and assigned a node.
+
+        Returns:
+        None
+        """
+        logger.debug("Starting: val_node_relationships.")
+        proxlb_data["guests"][guest_name]["processed"] = True
+
+        if len(proxlb_data["guests"][guest_name]["node_relationships"]) > 0:
+            logger.debug(f"Guest '{guest_name}' has relationships defined to node(s): {','.join(proxlb_data['guests'][guest_name]['node_relationships'])}. Pinning to node.")
+
+            # Get the node with the most free resources of the group
+            guest_node_relation_list = proxlb_data["guests"][guest_name]["node_relationships"]
+            Calculations.get_most_free_node(proxlb_data, False, guest_node_relation_list)
+
+            # Validate if the specified node name is really part of the cluster
+            if proxlb_data["meta"]["balancing"]["balance_next_node"] in proxlb_data["nodes"].keys():
+                logger.debug(f"Guest '{guest_name}' has a specific relationship defined to node: {proxlb_data['meta']['balancing']['balance_next_node']} is a known hypervisor node in the cluster.")
+            else:
+                logger.warning(f"Guest '{guest_name}' has a specific relationship defined to node: {proxlb_data['meta']['balancing']['balance_next_node']} but this node name is not known in the cluster!")
+
+        else:
+            logger.debug(f"Guest '{guest_name}' does not have any specific node relationships.")
+
+        logger.debug("Finished: val_node_relationships.")
+
     @staticmethod
     def update_node_resources(proxlb_data):
         """

proxlb/models/guests.py

@@ -11,6 +11,7 @@ __license__ = "GPL-3.0"
 from typing import Dict, Any
 from utils.logger import SystemdLogger
 from models.tags import Tags
+import time
 
 logger = SystemdLogger()
 
@@ -34,7 +35,7 @@ class Guests:
         """
 
     @staticmethod
-    def get_guests(proxmox_api: any, nodes: Dict[str, Any]) -> Dict[str, Any]:
+    def get_guests(proxmox_api: any, nodes: Dict[str, Any], meta: Dict[str, Any]) -> Dict[str, Any]:
         """
         Get metrics of all guests in a Proxmox cluster.
 
@@ -61,10 +62,11 @@ class Guests:
             # resource metrics for rebalancing to ensure that we do not overprovisiong the node.
             for guest in proxmox_api.nodes(node).qemu.get():
                 if guest['status'] == 'running':
+
                     guests['guests'][guest['name']] = {}
                     guests['guests'][guest['name']]['name'] = guest['name']
-                    guests['guests'][guest['name']]['cpu_total'] = guest['cpus']
-                    guests['guests'][guest['name']]['cpu_used'] = guest['cpu']
+                    guests['guests'][guest['name']]['cpu_total'] = int(guest['cpus'])
+                    guests['guests'][guest['name']]['cpu_used'] = Guests.get_guest_cpu_usage(proxmox_api, node, guest['vmid'], guest['name'])
                     guests['guests'][guest['name']]['memory_total'] = guest['maxmem']
                     guests['guests'][guest['name']]['memory_used'] = guest['mem']
                     guests['guests'][guest['name']]['disk_total'] = guest['maxdisk']
@@ -77,7 +79,10 @@ class Guests:
                     guests['guests'][guest['name']]['affinity_groups'] = Tags.get_affinity_groups(guests['guests'][guest['name']]['tags'])
                     guests['guests'][guest['name']]['anti_affinity_groups'] = Tags.get_anti_affinity_groups(guests['guests'][guest['name']]['tags'])
                     guests['guests'][guest['name']]['ignore'] = Tags.get_ignore(guests['guests'][guest['name']]['tags'])
+                    guests['guests'][guest['name']]['node_relationships'] = Tags.get_node_relationships(guests['guests'][guest['name']]['tags'])
                     guests['guests'][guest['name']]['type'] = 'vm'
+
+                    logger.debug(f"Resources of Guest {guest['name']} (type VM) added: {guests['guests'][guest['name']]}")
                 else:
                     logger.debug(f'Metric for VM {guest["name"]} ignored because VM is not running.')
 
@@ -88,8 +93,8 @@ class Guests:
                 if guest['status'] == 'running':
                     guests['guests'][guest['name']] = {}
                     guests['guests'][guest['name']]['name'] = guest['name']
-                    guests['guests'][guest['name']]['cpu_total'] = guest['cpus']
-                    guests['guests'][guest['name']]['cpu_used'] = guest['cpu']
+                    guests['guests'][guest['name']]['cpu_total'] = int(guest['cpus'])
+                    guests['guests'][guest['name']]['cpu_used'] = Guests.get_guest_cpu_usage(proxmox_api, node, guest['vmid'], guest['name'])
                     guests['guests'][guest['name']]['memory_total'] = guest['maxmem']
                     guests['guests'][guest['name']]['memory_used'] = guest['mem']
                     guests['guests'][guest['name']]['disk_total'] = guest['maxdisk']
@@ -102,9 +107,47 @@ class Guests:
                     guests['guests'][guest['name']]['affinity_groups'] = Tags.get_affinity_groups(guests['guests'][guest['name']]['tags'])
                     guests['guests'][guest['name']]['anti_affinity_groups'] = Tags.get_anti_affinity_groups(guests['guests'][guest['name']]['tags'])
                     guests['guests'][guest['name']]['ignore'] = Tags.get_ignore(guests['guests'][guest['name']]['tags'])
+                    guests['guests'][guest['name']]['node_relationships'] = Tags.get_node_relationships(guests['guests'][guest['name']]['tags'])
                     guests['guests'][guest['name']]['type'] = 'ct'
+
+                    logger.debug(f"Resources of Guest {guest['name']} (type CT) added: {guests['guests'][guest['name']]}")
                 else:
                     logger.debug(f'Metric for CT {guest["name"]} ignored because CT is not running.')
 
         logger.debug("Finished: get_guests.")
         return guests
+
+    @staticmethod
+    def get_guest_cpu_usage(proxmox_api, node_name: str, vm_id: int, vm_name: str) -> float:
+        """
+        Retrieve the average CPU usage of a guest instance (VM/CT) over the past hour.
+
+        This method queries the Proxmox VE API for RRD (Round-Robin Database) data
+        related to CPU usage of a specific guest instance and calculates the average CPU usage
+        over the last hour using the "AVERAGE" consolidation function.
+
+        Args:
+            proxmox_api: An instance of the Proxmox API client.
+            node_name (str): The name of the Proxmox node hosting the VM.
+            vm_id (int): The unique identifier of the guest instance (VM/CT).
+            vm_name (str): The name of the guest instance (VM/CT).
+
+        Returns:
+            float: The average CPU usage as a fraction (0.0 to 1.0) over the past hour.
+                   Returns 0.0 if no data is available.
+        """
+        logger.debug("Finished: get_guest_cpu_usage.")
+        time.sleep(0.1)
+
+        try:
+            logger.debug(f"Getting RRD dara for guest: {vm_name}.")
+            guest_data_rrd = proxmox_api.nodes(node_name).qemu(vm_id).rrddata.get(timeframe="hour", cf="AVERAGE")
+        except Exception:
+            logger.error(f"Failed to retrieve RRD data for guest: {vm_name} (ID: {vm_id}) on node: {node_name}. Using 0.0 as CPU usage.")
+            logger.debug("Finished: get_guest_cpu_usage.")
+            return 0.0
+
+        cpu_usage = sum(entry.get("cpu", 0.0) for entry in guest_data_rrd) / len(guest_data_rrd)
+        logger.debug(f"CPU RRD data for guest: {vm_name}: {cpu_usage}")
+        logger.debug("Finished: get_guest_cpu_usage.")
+        return cpu_usage

proxlb/models/nodes.py

@@ -63,7 +63,7 @@ class Nodes:
                 nodes["nodes"][node["node"]]["maintenance"] = False
                 nodes["nodes"][node["node"]]["cpu_total"] = node["maxcpu"]
                 nodes["nodes"][node["node"]]["cpu_assigned"] = 0
-                nodes["nodes"][node["node"]]["cpu_used"] = node["cpu"]
+                nodes["nodes"][node["node"]]["cpu_used"] = node["cpu"] * node["maxcpu"]
                 nodes["nodes"][node["node"]]["cpu_free"] = (node["maxcpu"]) - (node["cpu"] * node["maxcpu"])
                 nodes["nodes"][node["node"]]["cpu_assigned_percent"] = nodes["nodes"][node["node"]]["cpu_assigned"] / nodes["nodes"][node["node"]]["cpu_total"] * 100
                 nodes["nodes"][node["node"]]["cpu_free_percent"] = nodes["nodes"][node["node"]]["cpu_free"] / node["maxcpu"] * 100
@@ -84,14 +84,14 @@ class Nodes:
                 nodes["nodes"][node["node"]]["disk_used_percent"] = nodes["nodes"][node["node"]]["disk_used"] / node["maxdisk"] * 100
 
                 # Evaluate if node should be set to maintenance mode
-                if Nodes.set_node_maintenance(proxlb_config, node["node"]):
+                if Nodes.set_node_maintenance(proxmox_api, proxlb_config, node["node"]):
                     nodes["nodes"][node["node"]]["maintenance"] = True
 
         logger.debug("Finished: get_nodes.")
         return nodes
 
     @staticmethod
-    def set_node_maintenance(proxlb_config: Dict[str, Any], node_name: str) -> Dict[str, Any]:
+    def set_node_maintenance(proxmox_api, proxlb_config: Dict[str, Any], node_name: str) -> Dict[str, Any]:
         """
         Set nodes to maintenance mode based on the provided configuration.
 
@@ -99,6 +99,7 @@ class Nodes:
         based on the configuration provided in proxlb_config.
 
         Args:
+            proxmox_api (any): The Proxmox API client instance.
             proxlb_config (Dict[str, Any]): A dictionary containing the ProxLB configuration, including maintenance nodes.
             node_name: (str): The current node name within the outer iteration.
 
@@ -107,11 +108,24 @@ class Nodes:
         """
         logger.debug("Starting: set_node_maintenance.")
 
+        # Evaluate maintenance mode by config
         if proxlb_config.get("proxmox_cluster", None).get("maintenance_nodes", None) is not None:
             if len(proxlb_config.get("proxmox_cluster", {}).get("maintenance_nodes", [])) > 0:
                 if node_name in proxlb_config.get("proxmox_cluster", {}).get("maintenance_nodes", []):
-                    logger.warning(f"Node: {node_name} has been set to maintenance mode.")
+                    logger.info(f"Node: {node_name} has been set to maintenance mode (by ProxLB config).")
                     return True
+                else:
+                    logger.debug(f"Node: {node_name} is not in maintenance mode by ProxLB config.")
+
+        # Evaluate maintenance mode by Proxmox HA
+        for ha_element in proxmox_api.cluster.ha.status.current.get():
+            if ha_element.get("status"):
+                if "maintenance mode" in ha_element.get("status"):
+                    if ha_element.get("node") == node_name:
+                        logger.info(f"Node: {node_name} has been set to maintenance mode (by Proxmox HA API).")
+                        return True
+                    else:
+                        logger.debug(f"Node: {node_name} is not in maintenance mode by Proxmox HA API.")
 
         logger.debug("Finished: set_node_maintenance.")
 
@@ -135,7 +149,7 @@ class Nodes:
         if proxlb_config.get("proxmox_cluster", None).get("ignore_nodes", None) is not None:
             if len(proxlb_config.get("proxmox_cluster", {}).get("ignore_nodes", [])) > 0:
                 if node_name in proxlb_config.get("proxmox_cluster", {}).get("ignore_nodes", []):
-                    logger.warning(f"Node: {node_name} has been set to be ignored. Not adding node!")
+                    logger.info(f"Node: {node_name} has been set to be ignored. Not adding node!")
                     return True
 
         logger.debug("Finished: set_node_ignore.")

proxlb/models/tags.py

@@ -139,7 +139,7 @@ class Tags:
             tags (List): A list holding all defined tags for a given guest.
 
         Returns:
-            Bool: Returns a bool that indicates wether to ignore a guest or not.
+            Bool: Returns a bool that indicates whether to ignore a guest or not.
         """
         logger.debug("Starting: get_ignore.")
         ignore_tag = False
@@ -151,3 +151,30 @@ class Tags:
 
         logger.debug("Finished: get_ignore.")
         return ignore_tag
+
+    @staticmethod
+    def get_node_relationships(tags: List[str]) -> str:
+        """
+        Get a node relationship tag for a guest from the Proxmox cluster by the API to pin
+        a guest to a node.
+
+        This method retrieves a relationship tag between a guest and a specific
+        hypervisor node to pin the guest to a specific node (e.g., for licensing reason).
+
+        Args:
+            tags (List): A list holding all defined tags for a given guest.
+
+        Returns:
+            Str: The related hypervisor node name.
+        """
+        logger.debug("Starting: get_node_relationships.")
+        node_relationship_tags = []
+
+        if len(tags) > 0:
+            for tag in tags:
+                if tag.startswith("plb_pin"):
+                    node_relationship_tag = tag.replace("plb_pin_", "")
+                    node_relationship_tags.append(node_relationship_tag)
+
+        logger.debug("Finished: get_node_relationships.")
+        return node_relationship_tags

proxlb/utils/helper.py

@@ -8,7 +8,9 @@ __copyright__ = "Copyright (C) 2025 Florian Paul Azim Hoberg (@gyptazy)"
 __license__ = "GPL-3.0"
 
 
+import json
 import uuid
+import re
 import sys
 import time
 import utils.version
@@ -39,6 +41,8 @@ class Helper:
         get_daemon_mode(proxlb_config: Dict[str, Any]) -> None:
             Checks if the daemon mode is active and handles the scheduling accordingly.
     """
+    proxlb_reload = False
+
     def __init__(self):
         """
         Initializes the general Helper clas.
@@ -115,12 +119,149 @@ class Helper:
             None
         """
         logger.debug("Starting: get_daemon_mode.")
-        if proxlb_config.get("service", {}).get("daemon", False):
-            sleep_seconds = proxlb_config.get("service", {}).get("schedule", 12) * 3600
-            logger.info(f"Daemon mode active: Next run in: {proxlb_config.get('service', {}).get('schedule', 12)} hours.")
+        if proxlb_config.get("service", {}).get("daemon", True):
+
+            # Validate schedule format which changed in v1.1.1
+            if type(proxlb_config["service"].get("schedule", None)) != dict:
+                logger.error("Invalid format for schedule. Please use 'hours' or 'minutes'.")
+                sys.exit(1)
+
+            # Convert hours to seconds
+            if proxlb_config["service"]["schedule"].get("format", "hours") == "hours":
+                sleep_seconds = proxlb_config.get("service", {}).get("schedule", {}).get("interval", 12) * 3600
+            # Convert minutes to seconds
+            elif proxlb_config["service"]["schedule"].get("format", "hours") == "minutes":
+                sleep_seconds = proxlb_config.get("service", {}).get("schedule", {}).get("interval", 720) * 60
+            else:
+                logger.error("Invalid format for schedule. Please use 'hours' or 'minutes'.")
+                sys.exit(1)
+
+            logger.info(f"Daemon mode active: Next run in: {proxlb_config.get('service', {}).get('schedule', {}).get('interval', 12)} {proxlb_config['service']['schedule'].get('format', 'hours')}.")
             time.sleep(sleep_seconds)
+
         else:
-            logger.debug("Daemon mode is not active.")
+            logger.debug("Successfully executed ProxLB. Daemon mode not active - stopping.")
+            print("Daemon mode not active - stopping.")
             sys.exit(0)
 
         logger.debug("Finished: get_daemon_mode.")
+
+    @staticmethod
+    def get_service_delay(proxlb_config: Dict[str, Any]) -> None:
+        """
+        Checks if a start up delay for the service is defined and waits to proceed until
+        the time is up.
+
+        Parameters:
+            proxlb_config (Dict[str, Any]): A dictionary containing the ProxLB configuration.
+
+        Returns:
+            None
+        """
+        logger.debug("Starting: get_service_delay.")
+        if proxlb_config.get("service", {}).get("delay", {}).get("enable", False):
+
+            # Convert hours to seconds
+            if proxlb_config["service"]["delay"].get("format", "hours") == "hours":
+                sleep_seconds = proxlb_config.get("service", {}).get("delay", {}).get("time", 1) * 3600
+            # Convert minutes to seconds
+            elif proxlb_config["service"]["delay"].get("format", "hours") == "minutes":
+                sleep_seconds = proxlb_config.get("service", {}).get("delay", {}).get("time", 60) * 60
+            else:
+                logger.error("Invalid format for service delay. Please use 'hours' or 'minutes'.")
+                sys.exit(1)
+
+            logger.info(f"Service delay active: First run in: {proxlb_config.get('service', {}).get('delay', {}).get('time', 1)} {proxlb_config['service']['delay'].get('format', 'hours')}.")
+            time.sleep(sleep_seconds)
+
+        else:
+            logger.debug("Service delay not active. Proceeding without delay.")
+
+        logger.debug("Finished: get_service_delay.")
+
+    @staticmethod
+    def print_json(proxlb_config: Dict[str, Any], print_json: bool = False) -> None:
+        """
+        Prints the calculated balancing matrix as a JSON output to stdout.
+
+        Parameters:
+            proxlb_config (Dict[str, Any]): A dictionary containing the ProxLB configuration.
+
+        Returns:
+            None
+        """
+        logger.debug("Starting: print_json.")
+        if print_json:
+            # Create a filtered list by stripping the 'meta' key from the proxlb_config dictionary
+            # to make sure that no credentials are leaked.
+            filtered_data = {k: v for k, v in proxlb_config.items() if k != "meta"}
+            print(json.dumps(filtered_data, indent=4))
+
+        logger.debug("Finished: print_json.")
+
+    @staticmethod
+    def handler_sighup(signum, frame):
+        """
+        Signal handler for SIGHUP.
+
+        This method is triggered when the process receives a SIGHUP signal.
+        It sets the `proxlb_reload` class variable to True to indicate that
+        configuration should be reloaded in the main loop.
+
+        Args:
+            signum (int): The signal number (expected to be signal.SIGHUP).
+            frame (frame object): Current stack frame (unused but required by signal handler signature).
+        """
+        logger.debug("Starting: handle_sighup.")
+        logger.debug("Got SIGHUP signal. Reloading...")
+        Helper.proxlb_reload = True
+        logger.debug("Finished: handle_sighup.")
+
+    @staticmethod
+    def get_host_port_from_string(host_object):
+        """
+        Parses a string containing a host (IPv4, IPv6, or hostname) and an optional port, and returns a tuple of (host, port).
+
+        Supported formats:
+        - Hostname or IPv4 without port: "example.com" or "192.168.0.1"
+        - Hostname or IPv4 with port: "example.com:8006" or "192.168.0.1:8006"
+        - IPv6 in brackets with optional port: "[fc00::1]" or "[fc00::1]:8006"
+        - IPv6 without brackets, port is assumed after last colon: "fc00::1:8006"
+
+        If no port is specified, port 8006 is used as the default.
+
+        Args:
+            host_object (str): A string representing a host with or without a port.
+
+        Returns:
+            tuple: A tuple (host: str, port: int)
+        """
+        logger.debug("Starting: get_host_port_from_string.")
+
+        # IPv6 (with or without port, written in brackets)
+        match = re.match(r'^\[(.+)\](?::(\d+))?$', host_object)
+        if match:
+            host = match.group(1)
+            port = int(match.group(2)) if match.group(2) else 8006
+            return host, port
+
+        # Count colons to identify IPv6 addresses without brackets
+        colon_count = host_object.count(':')
+
+        # IPv4 or hostname without port
+        if colon_count == 0:
+            return host_object, 8006
+
+        # IPv4 or hostname with port
+        elif colon_count == 1:
+            host, port = host_object.split(':')
+            return host, int(port)
+
+        # IPv6 (with or without port, assume last colon is port)
+        else:
+            parts = host_object.rsplit(':', 1)
+            try:
+                port = int(parts[1])
+                return parts[0], port
+            except ValueError:
+                return host_object, 8006

proxlb/utils/logger.py

@@ -9,6 +9,7 @@ __license__ = "GPL-3.0"
 
 
 import logging
+import sys
 try:
     from systemd.journal import JournalHandler
     SYSTEMD_PRESENT = True
@@ -82,17 +83,22 @@ class SystemdLogger:
         self.logger = logging.getLogger(name)
         self.logger.setLevel(level)
 
-        # Create a JournalHandler for systemd integration if this
-        # is supported on the underlying OS.
+        # Create a logging handler depending on the
+        # capabilities of the underlying OS where systemd
+        # logging is preferred.
         if SYSTEMD_PRESENT:
             # Add a JournalHandler for systemd integration
-            journal_handler = JournalHandler()
-            journal_handler.setLevel(level)
-            # Set a formatter to include the logger's name and log message
-            formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
-            journal_handler.setFormatter(formatter)
-            # Add handler to logger
-            self.logger.addHandler(journal_handler)
+            handler = JournalHandler()
+        else:
+            # Add a stdout handler as a fallback
+            handler = logging.StreamHandler(sys.stdout)
+
+        handler.setLevel(level)
+        # Set a formatter to include the logger's name and log message
+        formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
+        handler.setFormatter(formatter)
+        # Add handler to logger
+        self.logger.addHandler(handler)
 
     def set_log_level(self, level: str) -> None:
         """

proxlb/utils/proxmox_api.py

@@ -20,18 +20,20 @@ except ImportError:
     PROXMOXER_PRESENT = False
 import random
 import socket
-import sys
 try:
     import requests
     REQUESTS_PRESENT = True
 except ImportError:
     REQUESTS_PRESENT = False
+import sys
+import time
 try:
     import urllib3
     URLLIB3_PRESENT = True
 except ImportError:
     URLLIB3_PRESENT = False
 from typing import Dict, Any
+from utils.helper import Helper
 from utils.logger import SystemdLogger
 
 
@@ -94,6 +96,7 @@ class ProxmoxApi:
         """
         logger.debug("Starting: ProxmoxApi initialization.")
         self.proxmox_api = self.api_connect(proxlb_config)
+        self.test_api_user_permissions(self.proxmox_api)
         logger.debug("Finished: ProxmoxApi initialization.")
 
     def __getattr__(self, name):
@@ -115,7 +118,7 @@ class ProxmoxApi:
                 "token_id" and "token_secret" keys for API token authentication.
 
         Raises:
-            SystemExit: If both username/password and API token authentication methods are
+            SystemExit: If both pass/token_secret and API token authentication methods are
                         provided, the function will log a critical error message and terminate
                         the program.
 
@@ -130,17 +133,17 @@ class ProxmoxApi:
             sys.exit(1)
 
         proxlb_credentials = proxlb_config["proxmox_api"]
-        present_auth_user = "user" in proxlb_credentials
-        present_auth_token = "token_id" in proxlb_credentials
+        present_auth_pass = "pass" in proxlb_credentials
+        present_auth_secret = "token_secret" in proxlb_credentials
 
-        if present_auth_user and present_auth_token:
+        if present_auth_pass and present_auth_secret:
             logger.critical(f"Username/password and API token authentication are mutal exclusive. Please use only one!")
             print(f"Username/password and API token authentication are mutal exclusive. Please use only one!")
             sys.exit(1)
 
         logger.debug("Finished: validate_config.")
 
-    def api_connect_get_hosts(self, proxmox_api_endpoints: list) -> str:
+    def api_connect_get_hosts(self, proxlb_config, proxmox_api_endpoints: list) -> str:
         """
         Perform a connectivity test to determine a working host for the Proxmox API.
 
@@ -151,6 +154,7 @@ class ProxmoxApi:
         are found, one is chosen at random to distribute the load across the cluster.
 
         Args:
+            proxlb_config (Dict[str, Any]): A dictionary containing the ProxLB configuration.
             proxmox_api_endpoints (list): A list of Proxmox API endpoints to test.
 
         Returns:
@@ -174,26 +178,30 @@ class ProxmoxApi:
             logger.critical(f"No proxmox_api hosts are defined.")
             sys.exit(1)
 
-        # Get a suitable Proxmox API endpoint. Therefore, we check if we only have
-        # a single Proxmox API endpoint or multiple ones. If only one, we can return
-        # this one immediately. If this one does not work, the urllib will raise an
-        # exception during the connection attempt.
-        if len(proxmox_api_endpoints) == 1:
-            return proxmox_api_endpoints[0]
-
         # If we have multiple Proxmox API endpoints, we need to check each one by
         # doing a connection attempt for IPv4 and IPv6. If we find a working one,
         # we return that one. This allows us to define multiple endpoints in a cluster.
         validated_api_hosts = []
         for host in proxmox_api_endpoints:
-            validated = self.test_api_proxmox_host(host)
-            if validated:
-                validated_api_hosts.append(validated)
+
+            # Get or set a default value for a maximum of retries when connecting to
+            # the Proxmox API
+            api_connection_retries = proxlb_config["proxmox_api"].get("retries", 1)
+            api_connection_wait_time = proxlb_config["proxmox_api"].get("wait_time", 1)
+
+            for api_connection_attempt in range(api_connection_retries):
+                validated_api_host, api_port = self.test_api_proxmox_host(host)
+                if validated_api_host:
+                    validated_api_hosts.append(validated_api_host)
+                    break
+                else:
+                    logger.warning(f"Attempt {api_connection_attempt + 1}/{api_connection_retries} failed for host {host}. Retrying in {api_connection_wait_time} seconds...")
+                    time.sleep(api_connection_wait_time)
 
         if len(validated_api_hosts) > 0:
             # Choose a random host to distribute the load across the cluster
             # as a simple load balancing mechanism.
-            return random.choice(validated_api_hosts)
+            return random.choice(validated_api_hosts), api_port
 
         logger.critical("No valid Proxmox API hosts found.")
         print("No valid Proxmox API hosts found.")
@@ -221,6 +229,10 @@ class ProxmoxApi:
         """
         logger.debug("Starting: test_api_proxmox_host.")
 
+        # Validate for custom ports in API hosts which might indicate
+        # that an external loadbalancer will be used.
+        host, port = Helper.get_host_port_from_string(host)
+
         # Try resolving DNS to IP and log non-resolvable ones
         try:
             ip = socket.getaddrinfo(host, None, socket.AF_UNSPEC)
@@ -232,12 +244,12 @@ class ProxmoxApi:
         for address_type in ip:
             if address_type[0] == socket.AF_INET:
                 logger.debug(f"{host} is type ipv4.")
-                if self.test_api_proxmox_host_ipv4(host):
-                    return host
+                if self.test_api_proxmox_host_ipv4(host, port):
+                    return host, port
             elif address_type[0] == socket.AF_INET6:
                 logger.debug(f"{host} is type ipv6.")
-                if self.test_api_proxmox_host_ipv6(host):
-                    return host
+                if self.test_api_proxmox_host_ipv6(host, port):
+                    return host, port
             else:
                 return False
 
@@ -262,7 +274,7 @@ class ProxmoxApi:
         logger.debug("Starting: test_api_proxmox_host_ipv4.")
         sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
         sock.settimeout(timeout)
-        logger.warning(f"Warning: Host {host} ran into a timout when connectoing on IPv4 for tcp/{port}.")
+        logger.warning(f"Warning: Host {host} ran into a timeout when connecting on IPv4 for tcp/{port}.")
         result = sock.connect_ex((host, port))
 
         if result == 0:
@@ -295,7 +307,7 @@ class ProxmoxApi:
         logger.debug("Starting: test_api_proxmox_host_ipv6.")
         sock = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
         sock.settimeout(timeout)
-        logger.warning(f"Host {host} ran into a timout when connectoing on IPv6 for tcp/{port}.")
+        logger.warning(f"Host {host} ran into a timeout when connecting via IPv6 for tcp/{port}.")
         result = sock.connect_ex((host, port))
 
         if result == 0:
@@ -306,9 +318,39 @@ class ProxmoxApi:
         sock.close()
         logger.warning(f"Host {host} is unreachable on IPv6 for tcp/{port}.")
 
-        logger.debug("Finished: test_api_proxmox_host_ipv4.")
+        logger.debug("Finished: test_api_proxmox_host_ipv6.")
         return False
 
+    def test_api_user_permissions(self, proxmox_api: any):
+        """
+        Test the permissions of the current user/token used for the Proxmox API.
+
+        This method gets all assigned permissions for all API paths for the current
+        used user/token and validates them against the minimum required permissions.
+
+        Args:
+            proxmox_api (any): The Proxmox API client instance.
+        """
+        logger.debug("Starting: test_api_user_permissions.")
+        permissions_required = ["Datastore.Audit", "Sys.Audit", "VM.Audit", "VM.Migrate"]
+        permissions_available = []
+
+        # Get the permissions for the current user/token from API
+        permissions = proxmox_api.access.permissions.get()
+
+        # Get all available permissions of the current user/token
+        for path, permission in permissions.items():
+            for permission in permissions[path]:
+                permissions_available.append(permission)
+
+        # Validate if all required permissions are included within the available permissions
+        for required_permission in permissions_required:
+            if required_permission not in permissions_available:
+                logger.critical(f"Permission '{required_permission}' is missing. Please adjust the permissions for your user/token. See also: https://github.com/gyptazy/ProxLB/blob/main/docs/03_configuration.md#required-permissions-for-a-user")
+                sys.exit(1)
+
+        logger.debug("Finished: test_api_user_permissions.")
+
     def api_connect(self, proxlb_config: Dict[str, Any]) -> proxmoxer.ProxmoxAPI:
         """
         Establishes a connection to the Proxmox API using the provided configuration.
@@ -341,7 +383,7 @@ class ProxmoxApi:
         self.validate_config(proxlb_config)
 
         # Get a valid Proxmox API endpoint
-        proxmox_api_endpoint = self.api_connect_get_hosts(proxlb_config.get("proxmox_api", {}).get("hosts", []))
+        proxmox_api_endpoint, proxmox_api_port = self.api_connect_get_hosts(proxlb_config, proxlb_config.get("proxmox_api", {}).get("hosts", []))
 
         # Disable warnings for SSL certificate validation
         if not proxlb_config.get("proxmox_api").get("ssl_verification", True):
@@ -355,6 +397,7 @@ class ProxmoxApi:
             if proxlb_config.get("proxmox_api").get("token_secret", False):
                 proxmox_api = proxmoxer.ProxmoxAPI(
                     proxmox_api_endpoint,
+                    port=proxmox_api_port,
                     user=proxlb_config.get("proxmox_api").get("user", True),
                     token_name=proxlb_config.get("proxmox_api").get("token_id", True),
                     token_value=proxlb_config.get("proxmox_api").get("token_secret", True),
@@ -364,6 +407,7 @@ class ProxmoxApi:
             else:
                 proxmox_api = proxmoxer.ProxmoxAPI(
                     proxmox_api_endpoint,
+                    port=proxmox_api_port,
                     user=proxlb_config.get("proxmox_api").get("user", True),
                     password=proxlb_config.get("proxmox_api").get("pass", True),
                     verify_ssl=proxlb_config.get("proxmox_api").get("ssl_verification", True),
@@ -383,6 +427,5 @@ class ProxmoxApi:
             sys.exit(2)
 
         logger.info(f"API connection to host {proxmox_api_endpoint} succeeded.")
-
         logger.debug("Finished: api_connect.")
         return proxmox_api

proxlb/utils/version.py

@@ -3,5 +3,5 @@ __app_desc__ = "A DRS alike loadbalancer for Proxmox clusters."
 __author__ = "Florian Paul Azim Hoberg <gyptazy>"
 __copyright__ = "Copyright (C) 2025 Florian Paul Azim Hoberg (@gyptazy)"
 __license__ = "GPL-3.0"
-__version__ = "1.1.0-alpha"
+__version__ = "1.1.5"
 __url__ = "https://github.com/gyptazy/ProxLB"

service/proxlb.service

@@ -1,11 +1,13 @@
 [Unit]
 Description=ProxLB - A loadbalancer for Proxmox clusters
-After=network-online.target
-Wants=network-online.target
+After=network-online.target pveproxy.service
+Wants=network-online.target pveproxy.service
 
 [Service]
 ExecStart=python3 /usr/lib/python3/dist-packages/proxlb/main.py -c /etc/proxlb/proxlb.yaml
 User=plb
+ExecReload=/bin/kill -HUP $MAINPID
+KillMode=process
 
 [Install]
 WantedBy=multi-user.target

setup.py

@@ -2,7 +2,7 @@ from setuptools import setup
 
 setup(
     name="proxlb",
-    version="1.1.0-alpha",
+    version="1.1.5",
     description="A DRS alike loadbalancer for Proxmox clusters.",
     long_description="An advanced DRS alike loadbalancer for Proxmox clusters that also supports maintenance modes and affinity/anti-affinity rules.",
     author="Florian Paul Azim Hoberg",