Enhancement: Migrate ingress from nginx+loadbalancer service to true ingress #1411

Closed
opened 2026-04-05 16:27:09 +02:00 by MrUnknownDE · 0 comments
Owner

Originally created by @InputObject2 on 11/29/2023

Is your feature request related to a problem? Please describe.
Having nginx with a hardcoded config in a custom docker file is normally not a very kubernetes way of doing things.

Describe the solution you'd like
In the charts, create the ingress configuration to match various paths.

Describe alternatives you've considered
I've started working on this a bit and got it working as far as I can tell.

A configmap to hold the header configs:

custom-headers-configmap
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: custom-headers-configmap
  namespace: oneuptime
data:
  X-Real-IP: "$remote_addr"
  X-Forwarded-For: "$proxy_add_x_forwarded_for"
  X-Forwarded-Proto: "$scheme"
  Host: "$host"
  Upgrade: "$http_upgrade"
  Connection: "upgrade"

The main ingress that does no url rewrites (nginx-ingress requires different ingresses if you have different rewrite-targets):

main-ingress
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: oneuptime-main-no-rewrites
  annotations:
    nginx.ingress.kubernetes.io/proxy-buffers-number: "4"
    nginx.ingress.kubernetes.io/proxy-buffer-size: "256k"
    nginx.ingress.kubernetes.io/proxy-set-headers: "custom-headers-configmap"
    nginx.ingress.kubernetes.io/enable-websocket: "true"
    cert-manager.io/cluster-issuer: letsencrypt-production
    external-dns.alpha.kubernetes.io/hostname: oneuptime.example.com.
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - "oneuptime.example.com"
    secretName: tls-oneuptime
  rules:
  - host: "oneuptime.example.com"
    http:
      paths:
      - path: /
        pathType: Exact
        backend:
          service:
            name: oneuptime-status-page
            port:
              number: 3105
      - path: /status-page
        pathType: Prefix
        backend:
          service:
            name: oneuptime-status-page
            port:
              number: 3105
      - path: /accounts
        pathType: Prefix
        backend:
          service:
            name: oneuptime-accounts
            port:
              number: 3003
      - path: /otel-collector
        pathType: Prefix
        backend:
          service:
            name: oneuptime-otel-collector
            port:
              number: 4317
      - path: /ingestor
        pathType: Prefix
        backend:
          service:
            name: oneuptime-ingestor
            port:
              number: 3400
      - path: /dashboard
        pathType: Prefix
        backend:
          service:
            name: oneuptime-dashboard
            port:
              number: 3009
      - path: /admin
        pathType: Prefix
        backend:
          service:
            name: oneuptime-admin-dashboard
            port:
              number: 3158
      - path: /identity
        pathType: Prefix
        backend:
          service:
            name: oneuptime-identity
            port:
              number: 3087
      - path: /reference/
        pathType: Prefix
        backend:
          service:
            name: oneuptime-api-reference
            port:
              number: 1445
      - path: /file
        pathType: Prefix
        backend:
          service:
            name: oneuptime-file
            port:
              number: 3125
      - path: /api
        pathType: Prefix
        backend:
          service:
            name: oneuptime-dashboard-api
            port:
              number: 3002
      - path: /realtime
        pathType: Prefix
        backend:
          service:
            name: oneuptime-dashboard-api
            port:
              number: 3002
      - path: /analytics-api
        pathType: Prefix
        backend:
          service:
            name: oneuptime-dashboard-api
            port:
              number: 3002
      - path: /workflow
        pathType: Prefix
        backend:
          service:
            name: oneuptime-workflow
            port:
              number: 3099
      - path: /l/
        pathType: Prefix
        backend:
          service:
            name: oneuptime-link-shortener
            port:
              number: 3521
      - path: /workers
        pathType: Prefix
        backend:
          service:
            name: oneuptime-workers
            port:
              number: 3452

And then 4 different ingresses for the different rewrite-targets we have:

status-page-api-ingress
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: oneuptime-status-page-api
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /api/status-page/
    nginx.ingress.kubernetes.io/proxy-buffers-number: "4"
    nginx.ingress.kubernetes.io/proxy-buffer-size: "256k"
    nginx.ingress.kubernetes.io/proxy-set-headers: "custom-headers-configmap"
    nginx.ingress.kubernetes.io/enable-websocket: "true"
    cert-manager.io/cluster-issuer: letsencrypt-production
    external-dns.alpha.kubernetes.io/hostname: oneuptime.example.com.
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - "oneuptime.example.com"
    secretName: tls-oneuptime
  rules:
  - host: "oneuptime.example.com"
    http:
      paths:
      - path: /status-page-api/
        pathType: Prefix
        backend:
          service:
            name: oneuptime-dashboard-api
            port:
              number: 3002
status-page-sso-ingress
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: oneuptime-status-page-sso
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /status-page-sso/
    nginx.ingress.kubernetes.io/proxy-buffers-number: "4"
    nginx.ingress.kubernetes.io/proxy-buffer-size: "256k"
    nginx.ingress.kubernetes.io/proxy-set-headers: "custom-headers-configmap"
    nginx.ingress.kubernetes.io/enable-websocket: "true"
    cert-manager.io/cluster-issuer: letsencrypt-production
    external-dns.alpha.kubernetes.io/hostname: oneuptime.example.com.
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - "oneuptime.example.com"
    secretName: tls-oneuptime
  rules:
  - host: "oneuptime.example.com"
    http:
      paths:
      - path: /status-page-sso-api/
        pathType: Prefix
        backend:
          service:
            name: oneuptime-identity
            port:
              number: 3087
status-page-identity
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: oneuptime-status-page-identity
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /status-page/
    nginx.ingress.kubernetes.io/proxy-buffers-number: "4"
    nginx.ingress.kubernetes.io/proxy-buffer-size: "256k"
    nginx.ingress.kubernetes.io/proxy-set-headers: "custom-headers-configmap"
    nginx.ingress.kubernetes.io/enable-websocket: "true"
    cert-manager.io/cluster-issuer: letsencrypt-production
    external-dns.alpha.kubernetes.io/hostname: oneuptime.example.com.
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - "oneuptime.example.com"
    secretName: tls-oneuptime
  rules:
  - host: "oneuptime.example.com"
    http:
      paths:
      - path: /status-page-identity-api/
        pathType: Prefix
        backend:
          service:
            name: oneuptime-identity
            port:
              number: 3087
ingestor
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: oneuptime-ingestor
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /incoming-request/
    nginx.ingress.kubernetes.io/proxy-buffers-number: "4"
    nginx.ingress.kubernetes.io/proxy-buffer-size: "256k"
    nginx.ingress.kubernetes.io/proxy-set-headers: "custom-headers-configmap"
    nginx.ingress.kubernetes.io/enable-websocket: "true"
    cert-manager.io/cluster-issuer: letsencrypt-production
    external-dns.alpha.kubernetes.io/hostname: oneuptime.example.com.
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - "oneuptime.example.com"
    secretName: tls-oneuptime
  rules:
  - host: "oneuptime.example.com"
    http:
      paths:
      - path: /heartbeat
        pathType: Prefix
        backend:
          service:
            name: oneuptime-ingestor
            port:
              number: 3400

Additional context
I'd do the PR but I'm not sure where to put this in the charts and I'd rather ask first since maybe I'm missing some paths. All the buttons in the web UI work and the experience is butter-smooth as far as I can tell.

*Originally created by @InputObject2 on 11/29/2023* **Is your feature request related to a problem? Please describe.** Having nginx with a hardcoded config in a custom docker file is normally not a very kubernetes way of doing things. **Describe the solution you'd like** In the charts, create the ingress configuration to match various paths. **Describe alternatives you've considered** I've started working on this a bit and got it working as far as I can tell. A configmap to hold the header configs: <details> <summary>custom-headers-configmap</summary> ```yaml --- apiVersion: v1 kind: ConfigMap metadata: name: custom-headers-configmap namespace: oneuptime data: X-Real-IP: "$remote_addr" X-Forwarded-For: "$proxy_add_x_forwarded_for" X-Forwarded-Proto: "$scheme" Host: "$host" Upgrade: "$http_upgrade" Connection: "upgrade" ``` </details> The main ingress that does no url rewrites (nginx-ingress requires different ingresses if you have different rewrite-targets): <details> <summary>main-ingress</summary> ```yaml --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: oneuptime-main-no-rewrites annotations: nginx.ingress.kubernetes.io/proxy-buffers-number: "4" nginx.ingress.kubernetes.io/proxy-buffer-size: "256k" nginx.ingress.kubernetes.io/proxy-set-headers: "custom-headers-configmap" nginx.ingress.kubernetes.io/enable-websocket: "true" cert-manager.io/cluster-issuer: letsencrypt-production external-dns.alpha.kubernetes.io/hostname: oneuptime.example.com. spec: ingressClassName: nginx tls: - hosts: - "oneuptime.example.com" secretName: tls-oneuptime rules: - host: "oneuptime.example.com" http: paths: - path: / pathType: Exact backend: service: name: oneuptime-status-page port: number: 3105 - path: /status-page pathType: Prefix backend: service: name: oneuptime-status-page port: number: 3105 - path: /accounts pathType: Prefix backend: service: name: oneuptime-accounts port: number: 3003 - path: /otel-collector pathType: Prefix backend: service: name: oneuptime-otel-collector port: number: 4317 - path: /ingestor pathType: Prefix backend: service: name: oneuptime-ingestor port: number: 3400 - path: /dashboard pathType: Prefix backend: service: name: oneuptime-dashboard port: number: 3009 - path: /admin pathType: Prefix backend: service: name: oneuptime-admin-dashboard port: number: 3158 - path: /identity pathType: Prefix backend: service: name: oneuptime-identity port: number: 3087 - path: /reference/ pathType: Prefix backend: service: name: oneuptime-api-reference port: number: 1445 - path: /file pathType: Prefix backend: service: name: oneuptime-file port: number: 3125 - path: /api pathType: Prefix backend: service: name: oneuptime-dashboard-api port: number: 3002 - path: /realtime pathType: Prefix backend: service: name: oneuptime-dashboard-api port: number: 3002 - path: /analytics-api pathType: Prefix backend: service: name: oneuptime-dashboard-api port: number: 3002 - path: /workflow pathType: Prefix backend: service: name: oneuptime-workflow port: number: 3099 - path: /l/ pathType: Prefix backend: service: name: oneuptime-link-shortener port: number: 3521 - path: /workers pathType: Prefix backend: service: name: oneuptime-workers port: number: 3452 ``` </details> And then 4 different ingresses for the different rewrite-targets we have: <details> <summary>status-page-api-ingress</summary> ```yaml --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: oneuptime-status-page-api annotations: nginx.ingress.kubernetes.io/rewrite-target: /api/status-page/ nginx.ingress.kubernetes.io/proxy-buffers-number: "4" nginx.ingress.kubernetes.io/proxy-buffer-size: "256k" nginx.ingress.kubernetes.io/proxy-set-headers: "custom-headers-configmap" nginx.ingress.kubernetes.io/enable-websocket: "true" cert-manager.io/cluster-issuer: letsencrypt-production external-dns.alpha.kubernetes.io/hostname: oneuptime.example.com. spec: ingressClassName: nginx tls: - hosts: - "oneuptime.example.com" secretName: tls-oneuptime rules: - host: "oneuptime.example.com" http: paths: - path: /status-page-api/ pathType: Prefix backend: service: name: oneuptime-dashboard-api port: number: 3002 ``` </details> <details> <summary>status-page-sso-ingress</summary> ```yaml --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: oneuptime-status-page-sso annotations: nginx.ingress.kubernetes.io/rewrite-target: /status-page-sso/ nginx.ingress.kubernetes.io/proxy-buffers-number: "4" nginx.ingress.kubernetes.io/proxy-buffer-size: "256k" nginx.ingress.kubernetes.io/proxy-set-headers: "custom-headers-configmap" nginx.ingress.kubernetes.io/enable-websocket: "true" cert-manager.io/cluster-issuer: letsencrypt-production external-dns.alpha.kubernetes.io/hostname: oneuptime.example.com. spec: ingressClassName: nginx tls: - hosts: - "oneuptime.example.com" secretName: tls-oneuptime rules: - host: "oneuptime.example.com" http: paths: - path: /status-page-sso-api/ pathType: Prefix backend: service: name: oneuptime-identity port: number: 3087 ``` </details> <details> <summary>status-page-identity</summary> ```yaml --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: oneuptime-status-page-identity annotations: nginx.ingress.kubernetes.io/rewrite-target: /status-page/ nginx.ingress.kubernetes.io/proxy-buffers-number: "4" nginx.ingress.kubernetes.io/proxy-buffer-size: "256k" nginx.ingress.kubernetes.io/proxy-set-headers: "custom-headers-configmap" nginx.ingress.kubernetes.io/enable-websocket: "true" cert-manager.io/cluster-issuer: letsencrypt-production external-dns.alpha.kubernetes.io/hostname: oneuptime.example.com. spec: ingressClassName: nginx tls: - hosts: - "oneuptime.example.com" secretName: tls-oneuptime rules: - host: "oneuptime.example.com" http: paths: - path: /status-page-identity-api/ pathType: Prefix backend: service: name: oneuptime-identity port: number: 3087 ``` </details> <details> <summary>ingestor</summary> ```yaml --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: oneuptime-ingestor annotations: nginx.ingress.kubernetes.io/rewrite-target: /incoming-request/ nginx.ingress.kubernetes.io/proxy-buffers-number: "4" nginx.ingress.kubernetes.io/proxy-buffer-size: "256k" nginx.ingress.kubernetes.io/proxy-set-headers: "custom-headers-configmap" nginx.ingress.kubernetes.io/enable-websocket: "true" cert-manager.io/cluster-issuer: letsencrypt-production external-dns.alpha.kubernetes.io/hostname: oneuptime.example.com. spec: ingressClassName: nginx tls: - hosts: - "oneuptime.example.com" secretName: tls-oneuptime rules: - host: "oneuptime.example.com" http: paths: - path: /heartbeat pathType: Prefix backend: service: name: oneuptime-ingestor port: number: 3400 ``` </details> **Additional context** I'd do the PR but I'm not sure where to put this in the charts and I'd rather ask first since maybe I'm missing some paths. All the buttons in the web UI work and the experience is butter-smooth as far as I can tell.
MrUnknownDE added the enhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancementenhancement labels 2026-04-05 16:27:11 +02:00
Sign in to join this conversation.
No Label enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement enhancement
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github/oneuptime#1411