Optimize CipherSyncData for very large vaults #1801

New Issue

MrUnknownDE · 2026-04-06T02:25:38+02:00

MrUnknownDE commented

2026-04-06 02:25:38 +02:00

Originally created by @BlackDex on 1/11/2023

As mentioned in #3111, using a very very large vault causes some issues. Mainly because of a SQLite limit, but, it could also cause issue on MariaDB/MySQL or PostgreSQL. It also uses a lot of memory, and memory allocations.

This PR solves this by removing the need of all the cipher_uuid's just to gather the correct attachments.

It will use the user_uuid and org_uuid's to get all attachments linked to both, weither the user has access to them or not. This isn't an issue, since the matching is done per cipher and the attachment data is only returned if there is a matching cipher to where the user has access to.

I also modified some code to be able to use ::with_capacity(n) where possible. This prevents re-allocations if the Vec increases size, which will happen a lot if there are a lot of ciphers.

According to my tests measuring the time it takes to sync, it seems to have lowered the duration a bit more.

Fixes #3111

*Originally created by @BlackDex on 1/11/2023* As mentioned in #3111, using a very very large vault causes some issues. Mainly because of a SQLite limit, but, it could also cause issue on MariaDB/MySQL or PostgreSQL. It also uses a lot of memory, and memory allocations. This PR solves this by removing the need of all the cipher_uuid's just to gather the correct attachments. It will use the user_uuid and org_uuid's to get all attachments linked to both, weither the user has access to them or not. This isn't an issue, since the matching is done per cipher and the attachment data is only returned if there is a matching cipher to where the user has access to. I also modified some code to be able to use `::with_capacity(n)` where possible. This prevents re-allocations if the `Vec` increases size, which will happen a lot if there are a lot of ciphers. According to my tests measuring the time it takes to sync, it seems to have lowered the duration a bit more. Fixes #3111

MrUnknownDE closed this issue

2026-04-06 02:25:38 +02:00

Sign in to join this conversation.

No Label

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github/vaultwarden#1801