Fixes #21231: Cache table existence for ObjectType checks #515

New Issue

MrUnknownDE · 2026-04-05T16:38:50+02:00

MrUnknownDE commented

2026-04-05 16:38:50 +02:00

Originally created by @pheus on 1/20/2026

Fixes: #21231

Summary

This PR introduces a process-level cache for the core_objecttype table existence check within ObjectTypeManager.get_for_model(), based on the incredible findings of @jetseverschuren-novoserve.

Currently, NetBox performs database introspection on every get_for_model() call to determine if it should fall back to ContentType (a check retained for backward compatibility during v4.4 migrations). This hits pg_catalog.pg_class repeatedly, causing significant performance degradation on data-heavy pages and API endpoints.

Proposed Changes:

Added a _table_exists boolean flag to ObjectTypeManager.
The manager now performs introspection at most once per worker process.
Once the table is detected, the flag is set to True, bypassing all future introspection calls for that process.
The logic explicitly avoids caching a False result to ensure safety during active migrations or upgrades.

Performance Impact:
Benchmarks identified by the community show a reduction in lookup time from ~44ms to ~3ms per call in fully migrated environments by eliminating the high-frequency introspection queries.

Testing:
I have verified this solution by successfully migrating an empty v3.7.8 database to the most recent commit in main. The fallback logic correctly handles the pre-migration state and transitions seamlessly once the core_objecttype table is provisioned.

Special thanks to @jetseverschuren-novoserve for identifying this bottleneck and providing the detailed analysis!

*Originally created by @pheus on 1/20/2026* ### Fixes: #21231 ### Summary This PR introduces a process-level cache for the `core_objecttype` table existence check within `ObjectTypeManager.get_for_model()`, based on the incredible findings of `@jetseverschuren-novoserve`. Currently, NetBox performs database introspection on every `get_for_model()` call to determine if it should fall back to `ContentType` (a check retained for backward compatibility during v4.4 migrations). This hits `pg_catalog.pg_class` repeatedly, causing significant performance degradation on data-heavy pages and API endpoints. **Proposed Changes:** - Added a `_table_exists` boolean flag to `ObjectTypeManager`. - The manager now performs introspection at most once per worker process. - Once the table is detected, the flag is set to `True`, bypassing all future introspection calls for that process. - The logic explicitly avoids caching a `False` result to ensure safety during active migrations or upgrades. **Performance Impact:** Benchmarks identified by the community show a reduction in lookup time from ~44ms to ~3ms per call in fully migrated environments by eliminating the high-frequency introspection queries. **Testing:** I have verified this solution by successfully migrating an empty v3.7.8 database to the most recent commit in `main`. The fallback logic correctly handles the pre-migration state and transitions seamlessly once the `core_objecttype` table is provisioned. Special thanks to @jetseverschuren-novoserve for identifying this bottleneck and providing the detailed analysis!

MrUnknownDE closed this issue

2026-04-05 16:38:50 +02:00

Sign in to join this conversation.

Branches Tags

main

21455-sql-indexes-audit

21357-register-model-actions

feature

20924-plugin-ui-components

21766-improve-test-coverage-for-sortable-table-columns

21157-export-template-public-models

21025-pre-render-config-contexts

21364-swagger

feature-ip-prefix-link

20911-dropdown-3

fix_module_substitution

21160-filterset

20911-dropdown-2

20044-elevation-stuck-lightmode

7604-filter-modifiers-v3

20660-script-load

19724-graphql

14884-script

19720-macaddress-interface-generic-relation

fix-19669-api-image-download

7604-filter-modifiers

19275-fixes-interface-bulk-edit

fix-17794-get_field_value_return_list

11507-show-aggregate-and-rir-on-api

9583-add_column_specific_search_field_to_tables

No Label

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github/netbox#515