Defer object serialization for events pipeline #496

New Issue

MrUnknownDE · 2026-04-05T16:37:02+02:00

MrUnknownDE commented

2026-04-05 16:37:02 +02:00

Originally created by @jeremystretch on 1/22/2026

NetBox Version

v4.5.1

Python Version

3.12

Area(s) of Concern

User Interface
REST API
GraphQL API
Python ORM
Other

Details

When an object is created or modified in NetBox, its changes are captured and queued for event processing. The flow is outlined below. (Note that although we're focusing on adds & changes here for simplicity, a very similar flow happens for object deletions as well.)

handle_changed_object() is triggered by the post_save() signal.
An ObjectChange record is created and saved to the database. This contains only a minimal serialized copy of the object (foreign keys are note resolved to related objects).
enqueue_event() is called to add the instance to the request's event queue.
The instance is serialized by serialize_for_event() to its complete JSON representation (using the its REST API serializer). This representation and associated metadata is added to the events queue.
Once the response has been generated, the events queue is flushed to the handlers registered in EVENTS_PIPELINE. (By default, this includes only extras.events.process_event_queue.)
process_event_queue() iterates through the events in the queue and executed any applicable event rules for each.

The JSON serialization performed at step 4 can be quite expensive, as the object's complete representation (including related objects) is assembled from the database. This occurs even if the serialized data is ultimately never needed, i.e. because there are no event rules registered for the object type.

We can skip the serialization process and its related database queries by deferring it until process_event_queue() executes and finds at least one suitable event rule. This would be a breaking change as event['data'] would presumably no longer be defined in the queue for other processors, but we might be able to devise a backward-compatible solution.

*Originally created by @jeremystretch on 1/22/2026* ### NetBox Version v4.5.1 ### Python Version 3.12 ### Area(s) of Concern - [x] User Interface - [x] REST API - [x] GraphQL API - [ ] Python ORM - [ ] Other ### Details When an object is created or modified in NetBox, its changes are captured and queued for event processing. The flow is outlined below. (Note that although we're focusing on adds & changes here for simplicity, a very similar flow happens for object deletions as well.) 1. [`handle_changed_object()`](https://github.com/netbox-community/netbox/blob/077d9b11294028b5e444a4c088058d18008533db/netbox/core/signals.py#L86) is triggered by the `post_save()` signal. 2. An ObjectChange record is created and saved to the database. This contains only a minimal serialized copy of the object (foreign keys are note resolved to related objects). 3. [`enqueue_event()`](https://github.com/netbox-community/netbox/blob/077d9b11294028b5e444a4c088058d18008533db/netbox/extras/events.py#L54) is called to add the instance to the request's event queue. 4. The instance is serialized by [`serialize_for_event()`](https://github.com/netbox-community/netbox/blob/077d9b11294028b5e444a4c088058d18008533db/netbox/extras/events.py#L26) to its _complete_ JSON representation (using the its REST API serializer). This representation and associated metadata is added to the events queue. 5. Once the response has been generated, the events queue is [flushed](https://github.com/netbox-community/netbox/blob/077d9b11294028b5e444a4c088058d18008533db/netbox/extras/events.py#L199) to the handlers registered in [`EVENTS_PIPELINE`](https://netboxlabs.com/docs/netbox/configuration/miscellaneous/#events_pipeline). (By default, this includes only `extras.events.process_event_queue`.) 6. [`process_event_queue()`](https://github.com/netbox-community/netbox/blob/077d9b11294028b5e444a4c088058d18008533db/netbox/extras/events.py#L169) iterates through the events in the queue and executed any applicable event rules for each. The JSON serialization performed at step 4 can be quite expensive, as the object's complete representation (including related objects) is assembled from the database. This occurs even if the serialized data is ultimately never needed, i.e. because there are no event rules registered for the object type. We can skip the serialization process and its related database queries by deferring it until `process_event_queue()` executes and finds at least one suitable event rule. This would be a breaking change as `event['data']` would presumably no longer be defined in the queue for other processors, but we might be able to devise a backward-compatible solution.

MrUnknownDE closed this issue

2026-04-05 16:37:02 +02:00

Sign in to join this conversation.

Branches Tags

main

21455-sql-indexes-audit

21357-register-model-actions

feature

20924-plugin-ui-components

21766-improve-test-coverage-for-sortable-table-columns

21157-export-template-public-models

21025-pre-render-config-contexts

21364-swagger

feature-ip-prefix-link

20911-dropdown-3

fix_module_substitution

21160-filterset

20911-dropdown-2

20044-elevation-stuck-lightmode

7604-filter-modifiers-v3

20660-script-load

19724-graphql

14884-script

19720-macaddress-interface-generic-relation

fix-19669-api-image-download

7604-filter-modifiers

19275-fixes-interface-bulk-edit

fix-17794-get_field_value_return_list

11507-show-aggregate-and-rir-on-api

9583-add_column_specific_search_field_to_tables

No Label complexity: high complexity: high complexity: high complexity: high complexity: high complexity: high complexity: high netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox netbox status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted status: accepted type: performance type: performance type: performance type: performance type: performance type: performance type: performance type: performance type: performance type: performance type: performance type: performance type: performance type: performance type: performance type: performance type: performance type: performance type: performance type: performance

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github/netbox#496