Multiple housekeeping jobs are created and numbers are doubled each day #575

New Issue

MrUnknownDE · 2026-04-05T16:45:57+02:00

MrUnknownDE commented

2026-04-05 16:45:57 +02:00

Originally created by @srinathman on 1/15/2026

NetBox Edition

NetBox Community

NetBox Version

v4.4.4.7

Python Version

3.12

Steps to Reproduce

It is a pretty recent install (~2months) with no other jobs running. We then noticed that the job queue had more than 100K jobs, being more than half of it in "Scheduled" state.

Here we can see how the number of jobs is increasing (double) every day:

netbox=> select to_char(created, 'YYYY-MM-DD') AS dt, count(1)  from core_job group by dt order by dt ;
     dt     | count 
------------+-------
 2025-12-01 |     1
 2025-12-02 |     1
 2025-12-04 |     2
 2025-12-05 |     1
 2025-12-06 |     1
 2025-12-07 |     1
 2025-12-08 |     1
 2025-12-09 |     1
 2025-12-10 |     1
 2025-12-11 |     1
 2025-12-12 |     1
 2025-12-13 |     1
 2025-12-14 |     1
 2025-12-15 |     1
 2025-12-16 |     1
 2025-12-17 |     1
 2025-12-18 |     1
 2025-12-19 |     1
 2025-12-20 |     1
 2025-12-21 |     1
 2025-12-22 |     1
 2025-12-23 |     1
 2025-12-24 |     1
 2025-12-25 |     2
 2025-12-26 |     2
 2025-12-27 |     2
 2025-12-28 |     4
 2025-12-29 |     8
 2025-12-30 |     8
 2025-12-31 |     8
 2026-01-01 |    16
 2026-01-02 |    16
 2026-01-03 |    32
 2026-01-04 |    64
 2026-01-05 |   128
 2026-01-06 |   256
 2026-01-07 |   512
 2026-01-08 |  1024
 2026-01-09 |  2024
 2026-01-10 |  4048
 2026-01-11 |  8048
 2026-01-12 | 16048
 2026-01-13 | 28048

All of them are Housekeeping jobs:

Worker logs shows:

# docker logs -n 1000 -f netbox-worker
03:35:38 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.383110s on worker d518a61fb780445485f3d6fa75e0bf95
03:35:38 default: Job OK (fd36e7a5-18c6-41a8-ae64-741c324bc086)
03:35:38 Result is kept for 500 seconds
03:35:38 default: handle(job=<Job: System Housekeeping>) (fd3879f2-ad80-42f3-ad41-2d68b642703b)
03:35:38 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.371754s on worker d518a61fb780445485f3d6fa75e0bf95
03:35:38 default: Job OK (fd3879f2-ad80-42f3-ad41-2d68b642703b)
03:35:38 Result is kept for 500 seconds
03:35:38 default: handle(job=<Job: System Housekeeping>) (fd3da052-6f49-4017-a363-aee6ff349918)
03:35:39 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.361040s on worker d518a61fb780445485f3d6fa75e0bf95
03:35:39 default: Job OK (fd3da052-6f49-4017-a363-aee6ff349918)
03:35:39 Result is kept for 500 seconds
03:35:39 default: handle(job=<Job: System Housekeeping>) (fd411d41-0875-4fd8-87a2-f4708024614b)
03:35:39 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.377983s on worker d518a61fb780445485f3d6fa75e0bf95
03:35:39 default: Job OK (fd411d41-0875-4fd8-87a2-f4708024614b)
03:35:39 Result is kept for 500 seconds
03:35:39 default: handle(job=<Job: System Housekeeping>) (fd41e5b6-fd05-496e-b875-cadfe9cf063c)
03:35:40 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.385645s on worker d518a61fb780445485f3d6fa75e0bf95
03:35:40 default: Job OK (fd41e5b6-fd05-496e-b875-cadfe9cf063c)
03:35:40 Result is kept for 500 seconds
03:35:40 default: handle(job=<Job: System Housekeeping>) (fd459af0-5e83-4871-836d-0fd3828535cb)
03:35:40 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.357940s on worker d518a61fb780445485f3d6fa75e0bf95
03:35:40 default: Job OK (fd459af0-5e83-4871-836d-0fd3828535cb)
03:35:40 Result is kept for 500 seconds
03:35:40 default: handle(job=<Job: System Housekeeping>) (fd47bacd-b0df-470d-84c0-70e94ff54530)
03:35:41 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.371687s on worker d518a61fb780445485f3d6fa75e0bf95
03:35:41 default: Job OK (fd47bacd-b0df-470d-84c0-70e94ff54530)
03:35:41 Result is kept for 500 seconds
03:35:41 default: handle(job=<Job: System Housekeeping>) (fd49cac4-869a-49bc-b5a8-a0c6d889acca)
03:35:41 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.364535s on worker d518a61fb780445485f3d6fa75e0bf95
03:35:41 default: Job OK (fd49cac4-869a-49bc-b5a8-a0c6d889acca)
03:35:41 Result is kept for 500 seconds
03:35:41 default: handle(job=<Job: System Housekeeping>) (fd55f4eb-76df-4a27-9c33-7f0233cc253a)
03:35:41 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.352494s on worker d518a61fb780445485f3d6fa75e0bf95
03:35:41 default: Job OK (fd55f4eb-76df-4a27-9c33-7f0233cc253a)
03:35:41 Result is kept for 500 seconds
03:35:42 default: handle(job=<Job: System Housekeeping>) (fd5b5e41-3811-4316-b073-b4bf2fbd8ab0)
03:35:42 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.364923s on worker d518a61fb780445485f3d6fa75e0bf95
03:35:42 default: Job OK (fd5b5e41-3811-4316-b073-b4bf2fbd8ab0)
03:35:42 Result is kept for 500 seconds
03:35:42 default: handle(job=<Job: System Housekeeping>) (fd5ded1e-b409-4b34-a263-18ffe384838e)
03:35:42 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.381298s on worker d518a61fb780445485f3d6fa75e0bf95
03:35:42 default: Job OK (fd5ded1e-b409-4b34-a263-18ffe384838e)
03:35:42 Result is kept for 500 seconds
03:35:42 default: handle(job=<Job: System Housekeeping>) (fd5ff950-0ade-464e-92d6-5847a077a66e)
03:35:43 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.390895s on worker d518a61fb780445485f3d6fa75e0bf95
03:35:43 default: Job OK (fd5ff950-0ade-464e-92d6-5847a077a66e)
03:35:43 Result is kept for 500 seconds
03:35:43 default: handle(job=<Job: System Housekeeping>) (fd62ae83-051b-4822-8f75-36f9bcd9f10e)
03:35:43 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.383343s on worker d518a61fb780445485f3d6fa75e0bf95
03:35:43 default: Job OK (fd62ae83-051b-4822-8f75-36f9bcd9f10e)
03:35:43 Result is kept for 500 seconds
03:35:43 default: handle(job=<Job: System Housekeeping>) (fd637660-e3ad-46b8-b190-c7e426568512)
03:35:44 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.355385s on worker d518a61fb780445485f3d6fa75e0bf95
03:35:44 default: Job OK (fd637660-e3ad-46b8-b190-c7e426568512)
03:35:44 Result is kept for 500 seconds
03:35:44 default: handle(job=<Job: System Housekeeping>) (fd6a3fbf-3fb0-40f4-bdc8-61c5baf5040a)
03:35:44 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.395517s on worker d518a61fb780445485f3d6fa75e0bf95
03:35:44 default: Job OK (fd6a3fbf-3fb0-40f4-bdc8-61c5baf5040a)
03:35:44 Result is kept for 500 seconds
03:35:44 default: handle(job=<Job: System Housekeeping>) (fd6b1b33-b699-427b-aa30-d3412b965edf)
03:35:45 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.376043s on worker d518a61fb780445485f3d6fa75e0bf95
03:35:45 default: Job OK (fd6b1b33-b699-427b-aa30-d3412b965edf)
03:35:45 Result is kept for 500 seconds
03:35:45 default: handle(job=<Job: System Housekeeping>) (fd72ace1-4b34-4c25-8703-5c02eaaa3fbc)
03:35:45 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.372542s on worker d518a61fb780445485f3d6fa75e0bf95
03:35:45 default: Job OK (fd72ace1-4b34-4c25-8703-5c02eaaa3fbc)
03:35:45 Result is kept for 500 seconds
03:35:45 default: handle(job=<Job: System Housekeeping>) (fd739125-a099-4165-b583-fd0c63f6e521)

This seems to be related to https://github.com/netbox-community/netbox/issues/20243

Expected Behavior

One housekeeping job per day.

Observed Behavior

It started with one job per day for a few days, but it suddenly started increasing and today we had 50K housekeeping jobs created.

*Originally created by @srinathman on 1/15/2026* ### NetBox Edition NetBox Community ### NetBox Version v4.4.4.7 ### Python Version 3.12 ### Steps to Reproduce It is a pretty recent install (~2months) with no other jobs running. We then noticed that the job queue had more than 100K jobs, being more than half of it in "Scheduled" state. Here we can see how the number of jobs is increasing (double) every day: ``` netbox=> select to_char(created, 'YYYY-MM-DD') AS dt, count(1) from core_job group by dt order by dt ; dt | count ------------+------- 2025-12-01 | 1 2025-12-02 | 1 2025-12-04 | 2 2025-12-05 | 1 2025-12-06 | 1 2025-12-07 | 1 2025-12-08 | 1 2025-12-09 | 1 2025-12-10 | 1 2025-12-11 | 1 2025-12-12 | 1 2025-12-13 | 1 2025-12-14 | 1 2025-12-15 | 1 2025-12-16 | 1 2025-12-17 | 1 2025-12-18 | 1 2025-12-19 | 1 2025-12-20 | 1 2025-12-21 | 1 2025-12-22 | 1 2025-12-23 | 1 2025-12-24 | 1 2025-12-25 | 2 2025-12-26 | 2 2025-12-27 | 2 2025-12-28 | 4 2025-12-29 | 8 2025-12-30 | 8 2025-12-31 | 8 2026-01-01 | 16 2026-01-02 | 16 2026-01-03 | 32 2026-01-04 | 64 2026-01-05 | 128 2026-01-06 | 256 2026-01-07 | 512 2026-01-08 | 1024 2026-01-09 | 2024 2026-01-10 | 4048 2026-01-11 | 8048 2026-01-12 | 16048 2026-01-13 | 28048 ``` All of them are Housekeeping jobs: <img width="1416" height="841" alt="Image" src="https://github.com/user-attachments/assets/7dd295ca-7742-409f-bc7f-cf9a2d7bd0e2" /> Worker logs shows: ``` # docker logs -n 1000 -f netbox-worker 03:35:38 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.383110s on worker d518a61fb780445485f3d6fa75e0bf95 03:35:38 default: Job OK (fd36e7a5-18c6-41a8-ae64-741c324bc086) 03:35:38 Result is kept for 500 seconds 03:35:38 default: handle(job=<Job: System Housekeeping>) (fd3879f2-ad80-42f3-ad41-2d68b642703b) 03:35:38 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.371754s on worker d518a61fb780445485f3d6fa75e0bf95 03:35:38 default: Job OK (fd3879f2-ad80-42f3-ad41-2d68b642703b) 03:35:38 Result is kept for 500 seconds 03:35:38 default: handle(job=<Job: System Housekeeping>) (fd3da052-6f49-4017-a363-aee6ff349918) 03:35:39 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.361040s on worker d518a61fb780445485f3d6fa75e0bf95 03:35:39 default: Job OK (fd3da052-6f49-4017-a363-aee6ff349918) 03:35:39 Result is kept for 500 seconds 03:35:39 default: handle(job=<Job: System Housekeeping>) (fd411d41-0875-4fd8-87a2-f4708024614b) 03:35:39 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.377983s on worker d518a61fb780445485f3d6fa75e0bf95 03:35:39 default: Job OK (fd411d41-0875-4fd8-87a2-f4708024614b) 03:35:39 Result is kept for 500 seconds 03:35:39 default: handle(job=<Job: System Housekeeping>) (fd41e5b6-fd05-496e-b875-cadfe9cf063c) 03:35:40 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.385645s on worker d518a61fb780445485f3d6fa75e0bf95 03:35:40 default: Job OK (fd41e5b6-fd05-496e-b875-cadfe9cf063c) 03:35:40 Result is kept for 500 seconds 03:35:40 default: handle(job=<Job: System Housekeeping>) (fd459af0-5e83-4871-836d-0fd3828535cb) 03:35:40 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.357940s on worker d518a61fb780445485f3d6fa75e0bf95 03:35:40 default: Job OK (fd459af0-5e83-4871-836d-0fd3828535cb) 03:35:40 Result is kept for 500 seconds 03:35:40 default: handle(job=<Job: System Housekeeping>) (fd47bacd-b0df-470d-84c0-70e94ff54530) 03:35:41 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.371687s on worker d518a61fb780445485f3d6fa75e0bf95 03:35:41 default: Job OK (fd47bacd-b0df-470d-84c0-70e94ff54530) 03:35:41 Result is kept for 500 seconds 03:35:41 default: handle(job=<Job: System Housekeeping>) (fd49cac4-869a-49bc-b5a8-a0c6d889acca) 03:35:41 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.364535s on worker d518a61fb780445485f3d6fa75e0bf95 03:35:41 default: Job OK (fd49cac4-869a-49bc-b5a8-a0c6d889acca) 03:35:41 Result is kept for 500 seconds 03:35:41 default: handle(job=<Job: System Housekeeping>) (fd55f4eb-76df-4a27-9c33-7f0233cc253a) 03:35:41 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.352494s on worker d518a61fb780445485f3d6fa75e0bf95 03:35:41 default: Job OK (fd55f4eb-76df-4a27-9c33-7f0233cc253a) 03:35:41 Result is kept for 500 seconds 03:35:42 default: handle(job=<Job: System Housekeeping>) (fd5b5e41-3811-4316-b073-b4bf2fbd8ab0) 03:35:42 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.364923s on worker d518a61fb780445485f3d6fa75e0bf95 03:35:42 default: Job OK (fd5b5e41-3811-4316-b073-b4bf2fbd8ab0) 03:35:42 Result is kept for 500 seconds 03:35:42 default: handle(job=<Job: System Housekeeping>) (fd5ded1e-b409-4b34-a263-18ffe384838e) 03:35:42 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.381298s on worker d518a61fb780445485f3d6fa75e0bf95 03:35:42 default: Job OK (fd5ded1e-b409-4b34-a263-18ffe384838e) 03:35:42 Result is kept for 500 seconds 03:35:42 default: handle(job=<Job: System Housekeeping>) (fd5ff950-0ade-464e-92d6-5847a077a66e) 03:35:43 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.390895s on worker d518a61fb780445485f3d6fa75e0bf95 03:35:43 default: Job OK (fd5ff950-0ade-464e-92d6-5847a077a66e) 03:35:43 Result is kept for 500 seconds 03:35:43 default: handle(job=<Job: System Housekeeping>) (fd62ae83-051b-4822-8f75-36f9bcd9f10e) 03:35:43 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.383343s on worker d518a61fb780445485f3d6fa75e0bf95 03:35:43 default: Job OK (fd62ae83-051b-4822-8f75-36f9bcd9f10e) 03:35:43 Result is kept for 500 seconds 03:35:43 default: handle(job=<Job: System Housekeeping>) (fd637660-e3ad-46b8-b190-c7e426568512) 03:35:44 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.355385s on worker d518a61fb780445485f3d6fa75e0bf95 03:35:44 default: Job OK (fd637660-e3ad-46b8-b190-c7e426568512) 03:35:44 Result is kept for 500 seconds 03:35:44 default: handle(job=<Job: System Housekeeping>) (fd6a3fbf-3fb0-40f4-bdc8-61c5baf5040a) 03:35:44 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.395517s on worker d518a61fb780445485f3d6fa75e0bf95 03:35:44 default: Job OK (fd6a3fbf-3fb0-40f4-bdc8-61c5baf5040a) 03:35:44 Result is kept for 500 seconds 03:35:44 default: handle(job=<Job: System Housekeeping>) (fd6b1b33-b699-427b-aa30-d3412b965edf) 03:35:45 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.376043s on worker d518a61fb780445485f3d6fa75e0bf95 03:35:45 default: Job OK (fd6b1b33-b699-427b-aa30-d3412b965edf) 03:35:45 Result is kept for 500 seconds 03:35:45 default: handle(job=<Job: System Housekeeping>) (fd72ace1-4b34-4c25-8703-5c02eaaa3fbc) 03:35:45 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.372542s on worker d518a61fb780445485f3d6fa75e0bf95 03:35:45 default: Job OK (fd72ace1-4b34-4c25-8703-5c02eaaa3fbc) 03:35:45 Result is kept for 500 seconds 03:35:45 default: handle(job=<Job: System Housekeeping>) (fd739125-a099-4165-b583-fd0c63f6e521) ``` This seems to be related to https://github.com/netbox-community/netbox/issues/20243 ### Expected Behavior One housekeeping job per day. ### Observed Behavior It started with one job per day for a few days, but it suddenly started increasing and today we had 50K housekeeping jobs created.

MrUnknownDE closed this issue

2026-04-05 16:45:57 +02:00

Sign in to join this conversation.

Branches Tags

main

21455-sql-indexes-audit

21357-register-model-actions

feature

20924-plugin-ui-components

21766-improve-test-coverage-for-sortable-table-columns

21157-export-template-public-models

21025-pre-render-config-contexts

21364-swagger

feature-ip-prefix-link

20911-dropdown-3

fix_module_substitution

21160-filterset

20911-dropdown-2

20044-elevation-stuck-lightmode

7604-filter-modifiers-v3

20660-script-load

19724-graphql

14884-script

19720-macaddress-interface-generic-relation

fix-19669-api-image-download

7604-filter-modifiers

19275-fixes-interface-bulk-edit

fix-17794-get_field_value_return_list

11507-show-aggregate-and-rir-on-api

9583-add_column_specific_search_field_to_tables

No Label netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox status: revisions needed status: revisions needed status: revisions needed status: revisions needed status: revisions needed status: revisions needed status: revisions needed status: revisions needed status: revisions needed status: revisions needed status: revisions needed status: revisions needed status: revisions needed type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github/netbox#575