Scope attachment deduplication to ingestion source

Browse Source

Previously, attachment deduplication was handled globally by enforcing a unique constraint on the content hash (contentHashSha256) in the `attachments` table. This caused an issue where an attachment from one ingestion source would be incorrectly linked if the same attachment was processed by a different source.

This commit refactors the deduplication logic to be scoped on a per-ingestion-source basis.

Changes:
-   **Schema:** The `attachments` table schema has been updated to include a nullable `ingestionSourceId` column. A composite unique index has been added on `(ingestionSourceId, contentHashSha256)` to enforce per-source uniqueness. The `ingestionSourceId` is nullable to ensure backward compatibility with existing databases.
-   **Ingestion Logic:** The `IngestionService` has been updated to provide the `ingestionSourceId` when inserting attachment records. The `onConflictDoUpdate` clause now targets the new composite key, ensuring that attachments are only considered duplicates if they have the same hash and originate from the same ingestion source.

...

This commit is contained in:

Wayne

2025-10-06 00:04:06 +02:00

parent 826fd6f965

commit 2a3d6846d8

3 changed files with 0 additions and 3 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

1

packages/backend/tsconfig.tsbuildinfo

File diff suppressed because one or more lines are too long

1

packages/enterprise/tsconfig.tsbuildinfo

File diff suppressed because one or more lines are too long

1

packages/types/tsconfig.tsbuildinfo

File diff suppressed because one or more lines are too long