RQ Jobs in non-model/default queues are kept when Jobs are deleted in DB and reinsert themselves #611

New Issue

MrUnknownDE · 2026-04-05T16:52:31+02:00

MrUnknownDE commented

2026-04-05 16:52:31 +02:00

Originally created by @moonrail on 1/12/2026

NetBox Edition

NetBox Community

NetBox Version

v4.5.0

Python Version

3.13

Steps to Reproduce

Use-Cases:

Canceling a queued/scheduled job
Re-Scheduling an existing Job (currently only possible by delete + enqueue)

Steps:

Have a plugin with a Job class inheriting from JobRunner
Enqueue a scheduled Job for this Job class and pass a queue_name that is
2.1 not the default queue
2.2 not the queue assigned via QUEUE_MAPPING, if any was defined
Delete the Job instance before its RQ Job runs, e.g. by using a scheduled Job while enqueueing or by not having enough workers to run it directly

Expected Behavior

Non-running RQ Job of deleted Job should be canceled and not start.

Observed Behavior

RQ Job of deleted Job stays queued/scheduled and is started by workers.

When that RQ Job starts, it uses the pickled Job Instance of its kwargs and saves it to the database, effectively re-inserting it with a new created datetime but under the same primary key.

This happens because feature of PR #19424 is not fully implemented - the delete logic in model Job is not passed this queue name and may not find the RQ Job to cancel it. It then silently ignores this issue without logging it and exits, leaving the RQ Job queued.

The delete logic only looks in one of two queues - either the queue defined by model label (app label is not differentiated) in QUEUE_MAPPINGS or otherwise in default:
3624b88c3f/netbox/core/models/jobs.py (L184-L193)

A workaround for plugins is to override the handle() function in the Job class and check if the job instance actually exists, before doing anything:

class SomeJob(JobRunner):
    @classmethod
    def handle(cls, job, *args, **kwargs):
        try:
            job.refresh_from_db()
        except Job.DoesNotExist:
            plugin_logger.info(f'{job} #{job.pk}/RQ Job {job.job_id} is not present in DB anymore, exit early')
            return

        return super().handle(job=job, *args, **kwargs)

PS: I do not know why the Job instance is pickled, but it seems odd - an RQ Job uses potentially outdated data as there is no refresh being done in current handle function and it leads to reinserting itself, as we can see. Instead of this it would be safer to just pass primary key, app label and model label to then fetch the Job instance on start.

*Originally created by @moonrail on 1/12/2026* ### NetBox Edition NetBox Community ### NetBox Version v4.5.0 ### Python Version 3.13 ### Steps to Reproduce Use-Cases: * Canceling a queued/scheduled job * Re-Scheduling an existing Job (currently only possible by delete + enqueue) Steps: 1. Have a plugin with a Job class inheriting from `JobRunner` 2. Enqueue a scheduled Job for this Job class and pass a `queue_name` that is 2.1 not the default queue 2.2 not the queue assigned via `QUEUE_MAPPING`, if any was defined 3. Delete the Job instance before its RQ Job runs, e.g. by using a scheduled Job while enqueueing or by not having enough workers to run it directly ### Expected Behavior Non-running RQ Job of deleted Job should be canceled and not start. ### Observed Behavior RQ Job of deleted Job stays queued/scheduled and is started by workers. When that RQ Job starts, it uses the pickled Job Instance of its kwargs and saves it to the database, effectively re-inserting it with a new created datetime but under the same primary key. This happens because feature of PR #19424 is not fully implemented - the delete logic in model `Job` is not passed this queue name and may not find the RQ Job to cancel it. It then silently ignores this issue without logging it and exits, leaving the RQ Job queued. The delete logic only looks in one of two queues - either the queue defined by model label (app label is not differentiated) in `QUEUE_MAPPINGS` or otherwise in `default`: https://github.com/netbox-community/netbox/blob/3624b88c3fa6cb677756823db1413081b3a6607a/netbox/core/models/jobs.py#L184-L193 A workaround for plugins is to override the `handle()` function in the Job class and check if the job instance actually exists, before doing anything: ```python class SomeJob(JobRunner): @classmethod def handle(cls, job, *args, **kwargs): try: job.refresh_from_db() except Job.DoesNotExist: plugin_logger.info(f'{job} #{job.pk}/RQ Job {job.job_id} is not present in DB anymore, exit early') return return super().handle(job=job, *args, **kwargs) ``` PS: I do not know why the Job instance is pickled, but it seems odd - an RQ Job uses potentially outdated data as there is no refresh being done in current handle function and it leads to reinserting itself, as we can see. Instead of this it would be safer to just pass primary key, app label and model label to then fetch the Job instance on start.

MrUnknownDE closed this issue

2026-04-05 16:52:31 +02:00

Sign in to join this conversation.

Branches Tags

main

21455-sql-indexes-audit

21357-register-model-actions

feature

20924-plugin-ui-components

21766-improve-test-coverage-for-sortable-table-columns

21157-export-template-public-models

21025-pre-render-config-contexts

21364-swagger

feature-ip-prefix-link

20911-dropdown-3

fix_module_substitution

21160-filterset

20911-dropdown-2

20044-elevation-stuck-lightmode

7604-filter-modifiers-v3

20660-script-load

19724-graphql

14884-script

19720-macaddress-interface-generic-relation

fix-19669-api-image-download

7604-filter-modifiers

19275-fixes-interface-bulk-edit

fix-17794-get_field_value_return_list

11507-show-aggregate-and-rir-on-api

9583-add_column_specific_search_field_to_tables

No Label netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low severity: low status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug type: bug

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github/netbox#611