Delay between Probe and Monitoring Log / Incident #472

New Issue

MrUnknownDE · 2026-04-05T16:19:57+02:00

MrUnknownDE commented

2026-04-05 16:19:57 +02:00

Originally created by @Vincent-Bouffard-Ouidou on 5/25/2025

Describe the bug
Over time, the delay increases between what probe detect and when the monitoring logs are displayed.
For example, i'm monitoring my self-hosted Gitlab.
I've stopped the registry, probes (1 and 2) have detected that the status was 502, but nothing in monitoring logs or in incidents.
It was May 25 2025, 23:20 CEST but the latest log in monitoring logs is May 25 2025, 19:16 CEST
The incident was created about 5 hours after the downtime.

To Reproduce
Steps to reproduce the behavior:

Go to 'Monitoring Logs'
Click on 'View Summary'
See delay

Expected behavior
See in column "Monitored At" the latest monitor summary with no delay
Incident is created as soon as it's detected

Deployment Type
Self-hosted : 7.0.4255

Additional context
I have about 100 monitored, all website and ports are configured with an interval of 5 minutes.
The other monitors (SSL certificate) are configured with an interval of every day.
I have the feeling that clickhouse cannot process information in real time and it piles up over time.

It was working well to 7.0.3980, but after, i'm not sure.

*Originally created by @Vincent-Bouffard-Ouidou on 5/25/2025* **Describe the bug** Over time, the delay increases between what probe detect and when the monitoring logs are displayed. For example, i'm monitoring my self-hosted Gitlab. I've stopped the registry, probes (1 and 2) have detected that the status was 502, but nothing in monitoring logs or in incidents. It was May 25 2025, 23:20 CEST but the latest log in monitoring logs is May 25 2025, 19:16 CEST The incident was created about 5 hours after the downtime. **To Reproduce** Steps to reproduce the behavior: 1. Go to 'Monitoring Logs' 2. Click on 'View Summary' 3. See delay **Expected behavior** See in column "Monitored At" the latest monitor summary with no delay Incident is created as soon as it's detected **Deployment Type** Self-hosted : 7.0.4255 **Additional context** I have about 100 monitored, all website and ports are configured with an interval of 5 minutes. The other monitors (SSL certificate) are configured with an interval of every day. I have the feeling that clickhouse cannot process information in real time and it piles up over time. It was working well to 7.0.3980, but after, i'm not sure.

MrUnknownDE closed this issue

2026-04-05 16:19:57 +02:00

MrUnknownDE added the bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug bug labels 2026-04-05 16:19:58 +02:00

Sign in to join this conversation.

Branches Tags

master

release

snyk-fix-0aa85871afc5c8eee83a20407bd9e063

new-dash

probe-runner

better-logs

merge-ui

api-ref-refactor

safe-vm

tf-demo

dash-chunk

mult-slack

episode-resolve

feat/homepage-seo-v2

seo/compare-pages-homepage-optimize

tf-sets

vite-build

alert-grouping-product

oneuptime-fix-exception-5f43cb12

deprecate-telemetry-service

oneuptime-fix-exception-b77cfaf1

workflow-ui-update

chore/bump-version-9.2.5-1764859493

chore/bump-version-9.2.5-1764857051

state-change

postmortem

chore/bump-version-9.1.3-1764685359

pricing-compare

use-cases

dashboard-logs

incident-declared-at

agent-lib

scim-patch-fix

files-notes

dropdown-labels

revoke-token

refresh-sessions

oneuptime-ssl

telemetry-attr

nativewind

oneuptime-mobile-app

timepkr

cert-kube

new-log-ui

ms-temas-bot

span-improve

teams-integration

cert-manager

final-ms-teams

new-markdown

scim-update

probe-proxy-impl

probe-proxy-url

fb-whatsapp

whatsapp-integration

ms-teams-conn

copilot-plus

vs-code-copilot

ms-teams

server-bucket-size

sp-scim

scim

probe-queue

queue-work

keda-autoscaler

otel-keda

telemetry-queue

otel-queue

mobile-responsive

add-state-to-title

file-tf

snyk-fix-21d325191a83f6e495f02680309be6df

new-terraform-provider

tf-provider

msp-server

announcement-templates

optmize-esbuild

ms-teams-workspace-integration

run-as-non-root

fix/team-member-service-test-error

feature/msteams-integration-backend

optmize-dash

split-dashboard

feature/responsive-navbar

feat/mobile-responsive-sidemenu

mobile-navbar

mobile-ui

fix-recurring-events

fix-monitor-test

snyk-fix-6d5b446d98eb30ad7cc48664f26e8dd3

snyk-fix-5f2a25c13d925a6df331a8db1b4b6bc3

snyk-fix-82cf608c1e34aadeec1e40be2dd7692c

snyk-fix-28fb72b74f8e210b21994c9d4a749db4

snyk-fix-9dd32ce4170d36a0032a14eca7bcc974

snyk-fix-33cfbfb6119028a909c55ae0b6f42134

snyk-fix-0d6c626395070c0bf0647893fd4ad82c

telemetry-decorator

snyk-fix-9dcd00f3db77cda7a8a956ad4c744638

snyk-fix-39e89b36dc9e401d2beb27cf134cb8f8

manual-workflow

sp-api

snyk-fix-476babcac00c78674ae2231bc91de4f0

snyk-fix-aba76b601f35a212295c529b373457fa

slack-app

otel-collector-queue-size

scheduled-maintenance-feed

add-proxy-to-server-monitor

snyk-fix-129563dbc737d32bc6bb8e3311b8bb86

metrics-monitor

snyk-fix-b3faaf7e1b0131c254a19d461fdfa12f

snyk-fix-49a541156a8e299f9158ecc2e1ce3e4e

snyk-fix-6fed1c2110e88e596893df3351740515

snyk-fix-7f3c8e70773dfe11ea8caed8e34b2573

snyk-fix-2643b617b033d0cbf19291b6ad7776a4

snyk-fix-1219e1241120fcf13feb7bc0b54e18e3

merge-monitor-metrics

dashboarding

oneuptime-copilot-oneuptime-copilot-improve-comments-ZlPvy

oneuptime-copilot-oneuptime-copilot-improve-comments-vThJD

oneuptime-copilot-oneuptime-copilot-improve-comments-veTFM

snyk-fix-1230088cdc5e6714c07c4043881095db

sp-custom-cert

realtime

snyk-fix-2267fc7afa5dd1aadc03bda66d835a26

snyk-fix-82eb97bf3553f75fcdec629b2375d091

snyk-fix-0ff1eeedd26324c34c4b6446c78d93ab

snyk-fix-7a5cb77d5e30b278d6086cc6ec7563dd

snyk-fix-80a68103cb676782e141f18725deae71

oneuptime-copilot-oneuptime-copilot-add-spans-Dcefh

oneuptime-copilot-oneuptime-copilot-improve-comments-YRJKG

oneuptime-copilot-oneuptime-copilot-improve-comments-PBqVE

snyk-fix-6390d5be9d2dcbe507da6d19b87bb581

oneuptime-copilot-oneuptime-copilot-improve-comments-dnfTZ

copilot-main

error-tracking

snyk-fix-616369b0f1423d8113422a2b0ea1798a

snyk-fix-ecde2433ac879fc83fcb7fbaef849ca2

snyk-fix-1436684cb4b974b4b38ec0de7c27678b

snyk-fix-929b1416ccb03655ff400075fc126a5d

merge-dir

snyk-upgrade-bc55cc4a3bed3c919d5253c5ccb89569

oneuptime-copilot-oneuptime-copilot-improve-comments-NXaNT

oneuptime-copilot-oneuptime-copilot-improve-comments-MhFUX

oneuptime-copilot-oneuptime-copilot-improve-comments-eHoNt

oneuptime-copilot-oneuptime-copilot-improve-comments-BWpyo

landing-page

clickhouse-optimization

attributes-cache

snyk-upgrade-b1d7fc785c4b347f9540423872d4f7eb

snyk-upgrade-b12f9d5ede2ce54b392c16e1398efc0d

snyk-upgrade-90863abe750274199042129d216ec458

snyk-upgrade-24742a40cc66c0ff20127c49b31c5b92

status-page-reports

monitor-reports

oneuptime-app-oneuptime-app-improve-comments-wPkHZ

oneuptime-app-oneuptime-app-improve-comments-zheFq

oneuptime-app-oneuptime-app-improve-comments-yGblU

oneuptime-app-oneuptime-app-improve-comments-XvJyI

oneuptime-app-oneuptime-app-improve-comments-Iigpp

oneuptime-app-oneuptime-app-improve-comments-BLqEd

oneuptime-app-oneuptime-app-improve-comments-BSWXD

oneuptime-app-oneuptime-app-improve-comments-iDNIR

oneuptime-app-oneuptime-app-improve-comments-FPiCN

oneuptime-app-oneuptime-app-improve-comments-hfdRa

oneuptime-app-oneuptime-app-improve-comments-yEhKY

oneuptime-app-oneuptime-app-improve-comments-iOjcB

oneuptime-app-oneuptime-app-improve-comments-tFtyx

oneuptime-app-oneuptime-app-improve-comments-sBvxI

oneuptime-app-oneuptime-app-improve-comments-IXAhB

oneuptime-app-oneuptime-app-improve-comments-YtccK

oneuptime-app-oneuptime-app-improve-comments-QCnqA

oneuptime-app-oneuptime-app-improve-comments-NMjBi

oneuptime-app-oneuptime-app-improve-comments-lDSQP

oneuptime-app-oneuptime-app-improve-comments-YdAAE

oneuptime-app-oneuptime-app-improve-comments-JVpQw

oneuptime-app-oneuptime-app-improve-comments-DncIm

oneuptime-app-oneuptime-app-improve-comments-iBzpC

oneuptime-app-oneuptime-app-improve-comments-GkOnn

telemetry-backend

snyk-upgrade-818024a85e0cd5b743112ffb7f513123

snyk-upgrade-2f9ef5e24aa83032721fce51c3a0b0ca

oneuptime-app-oneuptime-app-refactor-code-nDOMw

oneuptime-app-oneuptime-app-refactor-code-nzaSs

oneuptime-app-oneuptime-app-refactor-code-TIxRv

oneuptime-app-oneuptime-app-refactor-code-Llyxi

oneuptime-app-oneuptime-app-refactor-code-lfIez

oneuptime-app-oneuptime-app-improve-comments-FLlpf

oneuptime-app-oneuptime-app-improve-comments-DaGfY

oneuptime-app-oneuptime-app-improve-comments-XaYkK

oneuptime-app-oneuptime-app-fix-grammar-and-spelling-CDFoR

ai-copilot-main

oneuptime-app-oneuptime-app-fix-grammar-and-spelling-hueYU

oneuptime-app-oneuptime-app-fix-grammar-and-spelling-HQfro

oneuptime-app-oneuptime-app-fix-grammar-and-spelling-JVpKF

oneuptime-app-oneuptime-app-fix-grammar-and-spelling-mSRcD

oneuptime-app-oneuptime-app-fix-grammar-and-spelling-NXSdG

oneuptime-app-oneuptime-app-fix-grammar-and-spelling-CYFVx

oneuptime-app-oneuptime-app-improve-comments-UsHIA

oneuptime-app-oneuptime-app-improve-comments-ZBFjb

oneuptime-app-oneuptime-app-improve-comments-ggYpS

oneuptime-app-oneuptime-app-improve-comments-YUFeh

eval-over-time

oneuptime-app-oneuptime-app-improve-comments-LkJYk

ai-main-70b

eslint

oneuptime-app-test-branch-11

oneuptime-app-test-branch-10

oneuptime-app-test-branch-7

oneuptime-app-test-branch-6

test-branch

nz/metric-detail

ext-postgres

perms-refactor

filter-refactor

group-by

greenlock-refactor

attributes

react-app

otel

snyk-upgrade-3677f1da0f20b9341d779856ea606cce

snyk-upgrade-324862099bddfa90e5e0e06eb934c162

renovate/node-16.x-lockfile

snyk-upgrade-ac717665d10b7db0ddb20de9bdaf435b

snyk-upgrade-7b7ea7a1c722d2b7f48e290063771f7b

snyk-upgrade-520b5a017b5e7985121ef90b57317068

renovate/stripe-react-stripe-js-1.x-lockfile

renovate/heroicons-react-2.x-lockfile

renovate/headlessui-react-1.x-lockfile

renovate/clickhouse-clickhouse-server-23.x

snyk-upgrade-8a1c1f353568dcbc7b2c3e4beec4a6b7

snyk-upgrade-affb0a1678b57ae5a6f3183f5e8d302d

renovate/vm2-3.x-lockfile

renovate/start-server-and-test-1.x-lockfile

renovate/chrome-launcher-0.x-lockfile

renovate/mikepenz-release-changelog-builder-action-3.x

renovate/npm-xmldom-vulnerability

monitor-groups

feature-flags

logging

admin-dashboard

update-deps

metered-plan

fix-status-page-refresh

owner-emails

owners-in-form

monitoring

fix-small-issues

single-docker-container

workflow-fix

staus-page-fix

slack_workflow

emails

fix/ping-monitor-ipv6

fix-typescript-errors

issue/160-add-setup-alert

143-fix-github-actions-jobs-dont-run-on-prs-from-forked-repo

142-github-actions-build-job-to-run-home-tests

147-form-containers

131-add-styling-for-account-in-topbar

chore/setup-sonar-cloud-#21

96-css-for-login-page

97-css-login-page

35-write-docker-build-job-for-helmchart

ts-migrate

monitor-box

feature-github-actions-3

feature-github-actions

feature-github-actions-2

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github/oneuptime#472